Virtual Embryo Mutant perturbation prediction
The question
Participants will predict a held-out knockout: the expression and 3D position of the cells in a mutant embryo, given wild-type development plus one observed knockout to learn the shape of a perturbation response from.
A conditional knockout line is months of work per gene, so no lab can make one for every gene it wants to understand. The three knockouts here are ordered by difficulty: training and validation use very specific genes, while the hidden test gene is broadly expressed and its effect correspondingly diffuse. Validation answers become available by explicit opt-in during P3, as for the other tasks. Access permanently ends P2 submission eligibility; hidden test answers remain private.
Scoring is on the perturbation effect, not the absolute state. Predicting no change at all, simply returning the matched wild type, already reaches 0.956 absolute pseudobulk correlation, so nothing but the wild-type-to-mutant change carries any information about whether a model understood the knockout.
Modality
3D MERFISH from conditionally knocked-out embryos with matched wild-type controls at the same stage. Same schema as Task 2: 500-gene panel, log-normalised expression, per-cell 3D coordinates, cell-type annotation.
Splits
During P3, eligible participants can explicitly request the P2 validation answers for model development. Receiving access or submitting a first final P3 entry permanently stops further P2 submissions: for the entire ordinary team, or the individual Yukon member. Access does not consume a P3 attempt. The live Validation leaderboard remains separate from the immutable Frozen P2 standings captured at the P3 opening cutoff. Hidden test ground truth is never distributed.
- Train
- Mab21l2 KO @ E9.5
- A clear phenotype from a very specific gene.
- Validation
- Gata4 KO @ E8.75
- Also specific, at a different stage. Scored through the leaderboard until the final phase, when participants can explicitly request the answers and permanently end their P2 submission eligibility.
- Test
- β-catenin KO @ E8.75
- Broadly expressed, so its effect is diffuse: the hardest of the three. Hidden until the competition closes.
- Reference
- Matched WT @ E8.75 and E9.5
- Every DE metric is computed against the matched wild type, not against a preceding stage.
Input and output
- Input
- Wild-type development across the window, plus one observed knockout with its matched control. The model must predict the held-out mutant given the gene identity and the matched wild-type stage data.
- Output
- An AnnData file for the mutant condition. .X on the 500-gene panel plus obsm["spatial_3D"]. Same contract as Task 2.
The exact file contract and validation rules →
How it is scored
Five metrics rank Task 3: response gene recovery 30%, response direction 25%, response magnitude 25%, cell-state distribution 20%. Tissue shape is deliberately not scored here. It would compare absolute post-knockout geometry, while the meaningful question is whether the perturbation changed shape correctly, and no shift-based spatial term exists yet to sit beside the shift-based expression one.
- 30%
- Response gene recovery
Which genes the knockout actually moved.
- 25%
- Response direction
Whether they moved the way the real knockout moves them.
- 25%
- Response magnitude
How strong the predicted phenotype is, scored on the absolute value.
- 20%
- Cell-state distribution
The same question Tasks 1 and 2 ask, computed the same way. Directly against the target, with no wild-type-relative construction.
Every metric, with its derivation →
What the starter kit ships
Transparent reference points, not competitive upper bounds: a floor, a simple method, and a dynamics reference model, all with the exact evaluation code.
- wt_identity: the floor. Doing nothing.
- shift_transfer. Transfer the observed knockout’s pseudobulk response to the target gene.
- gene_ko. Adds gene-regulatory-network conditioning on the perturbed gene.
- perturb_ode: an integrated perturbation field rather than a fixed transferred shift.
- dynode_flow. Reuses the Task 2 checkpoint through its perturbation path, with no separate training.
Why it’s hard
- Absolute expression barely moves: doing nothing already scores 0.956 absolute pseudobulk Pearson. Only the wild-type-to-mutant change carries signal, which is why the panel scores the change and not the absolute state.
- One observed perturbation in training. A model has to separate the wild-type developmental trajectory from the genetic effect with very little supervision.
- The hidden test gene is broadly expressed, so its effect is diffuse rather than concentrated in a specific programme: the opposite of what the training knockout looks like.
- On the published references every model gets the direction of the response wrong, so this is not a task where a small architectural tweak is expected to close the gap.
Common pitfalls
- Copying the training knockout’s delta onto the test gene. It recovers a share of the effect and gets the direction wrong, which the panel scores explicitly.
- Optimising absolute expression. The floor already wins that comparison, by construction.
- Reading the shape group as a ranking signal here. For Task 3 it is diagnostic for now: it compares absolute post-knockout shape, while the meaningful question is whether the perturbation changed shape correctly, and no shift-based spatial term exists yet.
- Ignoring the stage difference between the training knockout and the evaluation ones.
Other tasks
Reach the organisers at [email protected].