Do the cell-state findings survive population holdout?

The conclusion flips if the implicated state disappears only in population holdout while detection and annotation performance remain stable under site and chemistry holdouts. That pattern supports population specificity, not a broadly transferable cell-state association.

Are populations represented across multiple sites and chemistries, so a population holdout can separate population specificity from site or chemistry effects?

I'd check whether gene rankings change across holdouts even when cell-state labels persist, since the decision here is which genes to prioritize, and stable labels alone don't answer that.

For an implicated state represented in few cohorts, I'd inspect its separation from other states before and after mapping. A retained annotation alone doesn't tell us whether those cells still occupy a distinct neighborhood. The scIB benchmark provides a concrete example: Harmony kept several isolated cell populations together while overlapping them with one another, and its isolated-label F1 score captured that weakness. [Luecken et al.](https://www.nature.com/articles/s41592-021-01336-8) For the proposed holdouts, a compact diagnostic would show the implicated state's isolated-label separation score in each fold, using evaluation labels established independently of the mapping being assessed. Loss of separation despite retained labels would flag a mapping issue to investigate before interpreting population specificity. This would complement the gene-ranking check already suggested, while leaving the author's stated uncertainty about population, site, and chemistry overlap unresolved.

I'd also assess population contrasts within cell types across holdouts, since [this benchmark](https://pmc.ncbi.nlm.nih.gov/articles/PMC12636214/) found that scIB incompletely captures variation within cell types.