Evaluation

T1 — Temporal gene-expression distribution prediction. single-cell RNA · predicts expression.

Task

Participants will predict the gene-expression distribution of a whole embryo at a developmental stage the model has never seen, given the stages before it. The prediction is a population of cells — progenitors, differentiating cells and terminal types in their real proportions — not an average cell. A model that collapses onto the mean profile forfeits most of the panel.

Staging an embryo is destructive and each time point is a different animal, so a stage that was not collected cannot be recovered by re-running the experiment; and the gap between collected stages is often where the interesting transitions happen. Here nothing sits between the validation and test stages at all, so interpolation has nothing to lean on.

To keep the task tractable we release two fully observed earlier stages of real dissociated single cells — RNA across the whole transcriptome, not the 500-gene MERFISH panel Tasks 2 and 3 use. Participants may additionally train on external public single-cell data; only the evaluation stages are fixed, and any external source must be disclosed with the submission.

Three populations of dissociated cells side by side, each a swarm of separate cells inside a density boundary. The E8.5 and E9.5 swarms are filled; the E12.5 boundary is drawn in violet and is empty. — Three different embryos, not one embryo followed over time — profiling destroys the embryo, so there is no cell-level correspondence between stages. What is predicted is the distribution at E12.5, and no morphology is involved.

Three different embryos, not one embryo followed over time — profiling destroys the embryo, so there is no cell-level correspondence between stages. What is predicted is the distribution at E12.5, and no morphology is involved.

Splits

Validation and test ground truth are both withheld. A validation submission returns a leaderboard score, not the answers — those are released only when the final test set is.

Train: E8.5 · E9.5; The two real single-cell stages before the target. There is no E9.25 in this release.
Validation: E10.5; Scored through the leaderboard; the answers are not distributed.
Test: E12.5; Hidden. Scored only in the final phase.
Outside the split: E7.75; Ships with the release, usable as background, not part of the split.

No observed stage sits between E10.5 and E12.5, so temporal interpolation has nothing to lean on; and only about a third of target cells have a same-type predecessor to extrapolate a velocity from. Full task page →