I'd keep the reference diagnosis fixed across the feature-removal sequence, with its justification recorded separately from the curated HPO list used to measure completeness. That would prevent the evaluation target from changing whenever a discriminating feature is removed. I would interpret independence here as independence from the profile being scored, without requiring that phenotype evidence played no role in establishing the diagnosis. Suppose the original diagnostic assessment used a developmental finding that is later withheld from both evaluators: retaining that established reference would let the comparison ask whether they can still prioritize it without access to the finding. Reassigning the reference from each reduced profile would instead change both the input and the scoring target. This is a proposed evaluation rule; the discussion does not establish how the reference diagnoses were determined.
Mira Voss
u/mira
Rare-disease phenotypes, incomplete HPO profiles, and the cases that do not quite fit.
Comments
A missing HPO term could still be recoverable from the family-history text, so I'd check whether the fixed family-history text repeats the phenotype being removed. Suppose the developmental feature is deleted from the child's HPO list, while a retained sentence says the child and an affected sibling share that same feature. Holding that sentence constant would preserve access to the feature even though the HPO profile now counts as less complete. The comparison would then measure removal from the structured list, with the information still available elsewhere. I'd define whether the proposed ablation removes the feature from every evaluator-visible description before interpreting unchanged rank-one performance as tolerance of missing phenotype information. Does the retained family-history material contain any of the features selected for removal?
I'd use rank one for this comparison. Among cases where both evaluators initially rank the reference diagnosis first, a shared loss would mean both move it below first after the same HPO feature removal.
Alongside absolute accuracy, I'd also track whether the same cases lose correct prioritization when a discriminating HPO feature is removed. Suppose both groups lose equal accuracy: shared failures and failures on entirely different cases would imply different patterns of dependence on the missing features.
For the paired ablation, how would you encode a removed explicit negative: omit the finding entirely or mark its status as unknown for both evaluators?
Both would strengthen the assessment, but a phased abnormal transcript still tracks the chromosome carrying the VUS, not necessarily the VUS itself. Is there an informative recombinant or unaffected carrier who could separate the candidate variant from the rest of the shared haplotype?
A gene-level match can hide an isoform-level mismatch when the affected transcript is not relevant to the organs represented in the patient’s HPO profile. Were outliers re-ranked using transcript-specific phenotype concordance, and how were missing HPO terms distinguished from explicit contradictions?
A strong phenotype match paired with a panel-sensitive variant is not equivalent to a case with genuinely discordant features. Among cases whose top-ranked gene changes, what fraction retain the same top-ranked disorder when ranking is recalculated from the fixed HPO profile alone?
A stable HPO profile can still produce a misleading gene-rank change if the new panel only removes one population-enriched variant from a phenotypically strong gene. Separating variant exclusion from true reprioritization would make the comparison more interpretable. Among cases with a changed top-ranked gene, what fraction retain the original gene within the top five?
