parkinson, Parkinson's research notebook

The tasks

Replication. A prediction reached p < 0.05 in its discovery half of the donors. Will it hold in the other half? The analysts see only the discovery-half donors. The split is rebuilt from the seed written down when the prediction was registered, so the answer can't leak in.

Follow-up. The swarm proposed a new prediction after a failure. Will it point the same way, at p < 0.05, in the Kamath et al. 2022 cohort? The analysts see only the first atlas.

Each answer is a probability, scored with the Brier score: the average squared distance between the probability and what happened (0 is perfect; always answering the base rate is the bar to beat). AUC says how well the probabilities rank the hits above the misses (0.5 is chance).

power        chance the effect clears the next test, given its size and the donors
confounder   share of the effect left after age, sex and post-mortem interval
composition  share of the effect left after the cell group's share of nuclei
cell type    how well the gene is measured in this group
axis         whether the gene moves the expected way on the other pathology measure
skeptic      the worst p after dropping one donor; the effect without the most severe

Replication

129 tasks: 86 train genes, 43 dev genes. 2 held (2%). Base rate on train: 0.012. Scores below are on the dev genes only; lower Brier is better, AUC 0.5 is chance.

model	Brier	AUC	dev tasks
always the base rate	0.023	0.500	43
six analysts, equal weight	0.438	0.238	43
power alone	0.487	0.143	43
confounder alone	0.826	0.750	43
composition alone	0.714	0.262	43
cell type alone	0.724	0.643	43
axis alone	0.023	0.500	43
skeptic alone	0.493	0.036	43
tuned on train genes	0.023	0.214	43

fitted weights (train genes)
  intercept    -4.63
  power        +0.09
  confounder   +0.04
  composition  +0.09
  cell type    +0.00
  axis         +0.00
  skeptic      +0.02

Follow-ups

238 tasks: 163 train genes, 75 dev genes. 50 held (21%). Base rate on train: 0.190. Scores below are on the dev genes only; lower Brier is better, AUC 0.5 is chance.

model	Brier	AUC	dev tasks
always the base rate	0.193	0.500	75
six analysts, equal weight	0.441	0.392	75
power alone	0.571	0.416	75
confounder alone	0.738	0.459	75
composition alone	0.724	0.585	75
cell type alone	0.502	0.484	75
axis alone	0.263	0.491	75
skeptic alone	0.575	0.414	75
tuned on train genes	0.181	0.599	75

fitted weights (train genes)
  intercept    -0.79
  power        -0.38
  confounder   -0.14
  composition  -0.08
  cell type    +0.29
  axis         +0.08
  skeptic      -0.48

Where the tuned analysts were most wrong

GPR37L1 in microglia, Lewy stagesaid 17% it would holdit held
LAMTOR3 in oligodendrocytes, Lewy stagesaid 18% it would holdit held
SLC30A7 in oligodendrocytes, Lewy stagesaid 18% it would holdit held
ABCB9 in oligodendrocytes, Lewy stagesaid 18% it would holdit held
KCNG2 in OPCs, Lewy stagesaid 18% it would holdit held
ABL2 in OPCs, Lewy stagesaid 18% it would holdit held

Sealed, then scored on the future

A score on held-out genes can still flatter a method that was chosen after looking. So the tuned weights are frozen, hashed, and scored only on predictions registered after that moment. The agent makes new predictions every few minutes and the swarm keeps proposing follow-ups, so this test set grows on its own.

sealed      2026-10-23T13:45:11Z
hash        dbbc59373b0341489c8aa4fcb8d3e32b88f905302587afb433ff2a3399b78324
trained on  101 replication tasks, base rate 0.050

after the seal   replication: 129 tasks, Brier 0.016 (base rate 0.016), AUC 0.531
                 follow-up:   82 tasks, Brier 0.105, AUC 0.561

Rules