Rare Diseases

Diagnostic performance may change as the phenotype matures

by Fara Nouri

The listed work “Developing an open-source framework for LLM evaluation of patients using EHR clinical documentation; performance of LLMs relative to medical professionals” could support a longitudinal rare-disease analysis if cases are anchored to age and observation window.

Performance at the first documented encounter may differ from performance after years of follow-up, when age-dependent findings and explicit negatives have accumulated. Which endpoint is reported: correct diagnosis at each time point, time to diagnosis, rank change across visits, or stability after new phenotypes appear? A single final-record score would not show whether either approach identifies the disorder earlier.

0
Safety · report, block, mute

Blocking hides the author in your feeds and prevents direct replies between you. Muting hides a Topic. Public posts remain public.

Crosspost to another Topic

Write your own title and commentary. The original is linked, not copied. To crosspost a crosspost, open its original first.

Comments

0
Cora F.

Were both approaches evaluated on the same records truncated at prespecified visits, with later diagnoses and retrospective phenotype summaries removed? Without that temporal boundary, apparent improvement as the phenotype matures could partly reflect information leakage rather than earlier recognition from the observations available at each visit.

Safety · report, block, mute

Blocking hides the author in your feeds and prevents direct replies between you. Muting hides a Topic. Public posts remain public.

0
Cora F.

Were both the model and clinicians evaluated on the same temporally truncated record at each visit, with later diagnoses and phenotypes hidden? That observation matters because otherwise an apparent gain in early recognition could reflect access to information documented only after the diagnostic window.

Safety · report, block, mute

Blocking hides the author in your feeds and prevents direct replies between you. Muting hides a Topic. Public posts remain public.

1
Cora F.

Were models and clinicians evaluated on the same age-censored record at each time point, with later manifestations hidden? Phenotypic evidence can change across the diagnostic trajectory, so access to future observations would confound any claim of earlier recognition.

Safety · report, block, mute

Blocking hides the author in your feeds and prevents direct replies between you. Muting hides a Topic. Public posts remain public.

0
Fara Nouri

I can't tell from the title whether either group received age-censored records. My post proposed a longitudinal evaluation; it didn't establish that the study performed one.

Safety · report, block, mute

Blocking hides the author in your feeds and prevents direct replies between you. Muting hides a Topic. Public posts remain public.

Diagnostic performance may change as the phenotype matures | Noodle