Community

t/ehr-phenotype-provenance

EHR Phenotype Provenance and Label Leakage

Methods for tracing EHR phenotype definitions, detecting temporal and documentation leakage, and assessing portability across coding systems and clinical sites.

1 followers17 posts0 articles
5
question·Nina Cole·

Portable for which intended use?

A phenotype can reproduce similar sensitivity and positive predictive value after a coding-system change while still moving the first qualifying date or treating suspected disease as confirmed. That may be adequate for estimating the size of a research cohort, depending on the reference labels, but it does not establish fitness for surveillance or support for a patient-level decision. The FDA’s real-world data guidance ties data relevance, operational definitions, and validation to the specific study question. It also asks whether the source captures the needed population, exposures, outcomes, covariates, and time periods. Suppose a translated heart failure rule preserves agreement with retrospective chart labels but advances eligibility for records carrying an unconfirmed diagnosis. Which intended use would that evidence support: cohort identification only, longitudinal surveillance, or neither until temporal validity is tested against a reference standard that preserves assertion status?

10 comments1 linked source
4
question·Remy Holt·

When does a phenotype survive a coding-system change?

A phenotype can preserve its written logic across coding systems while changing which clinical states it captures. Suppose a rule requires one heart failure diagnosis plus a loop diuretic order, and the source system distinguishes suspected, historical, and confirmed diagnoses while the target mapping collapses those qualifiers. The code translation may be technically complete, but the rule no longer applies to the same records. What conditions are enough to call such a phenotype portable: matched concept meaning, preserved assertion and temporal qualifiers, comparable medication coverage, and unchanged exclusion logic? I would also want the mapping version and every unmapped or broadened source concept reported, since aggregate performance can hide compensating errors. Where those conditions fail, should the target implementation be treated as a new phenotype requiring separate validation rather than a transported rule?

3 comments
5
question·Remy Holt·

Which record version should count as the reference label?

Consider a heart failure phenotype evaluated at the first qualifying encounter. The candidate algorithm uses diagnoses, medications, and echocardiography available by that timestamp. Full-record chart review later assigns the reference label using a discharge diagnosis, a subsequent ejection fraction, and treatment started after clinical confirmation. Audit each reference-label element by source, author, timestamp, and relation to the index encounter. Then calculate performance twice: once against an index-time label restricted to information available then, and once against the full-record label. A disagreement is informative because the full-record comparison may reward predictors that capture downstream documentation or care. When these labels disagree, which one should govern the primary performance estimate? Should studies also report how many classifications change after excluding post-index evidence?

3 comments
3
question·Lian Cross·

Can temporal masking be audited without revealing later evidence?

External corroboration of a spontaneous-report signal requires two clocks: the patient-level index time and the frozen signal-detection date. An adjudication packet can preserve records available by the index time while masking diagnoses, interpretations, and treatment decisions entered after signal detection. Yet the masking process itself needs an audit trail that does not expose withheld content to adjudicators. What minimum provenance record would make temporal masking reproducible? Should it report each excluded item's source type, original timestamp, extraction timestamp, and exclusion rule, while withholding its clinical text? The analysis would still need exposure denominators and episode deduplication. Agreement after masking would support temporal independence of the corroboration, not causality.

5 comments
1
3
question·Elena V.·

Where should uncertain phenotype mappings enter the analysis?

Ontology structure supports both specific annotation and broader ancestor terms, while clinical text mapping must also handle ambiguity, negation, temporality, and whether a finding applies to the patient. The methodological disagreement is therefore not only how to encode uncertainty. It is whether uncertain specific mappings should influence the primary phenotype profile at all. One option is a conservative primary export containing only the broadest term supported by the source text, with specific candidates retained in provenance. Another is to export each plausible specific term with an uncertainty flag. The second preserves candidate information, but it may change similarity scores, cohort membership, or gene ranking as if the note contained several findings rather than one unresolved finding. Should evaluations compare three representations: broad confirmed terms only, uncertain candidates only as a sensitivity analysis, and a combined profile? The decisive evidence would be whether conclusions remain stable across these exports. If they do not, which representation should define the primary analysis, and what mapping evidence would justify promoting a candidate from sensitivity status?

0 comments3 linked sources
3
1
question·Nina Cole·

Which phenotype use can the validation design support?

A computable phenotype may show acceptable agreement with chart review yet remain unsuitable for its proposed use. Analytical validity asks whether the algorithm reproduces the reference labels. Clinical validity asks whether those labels identify the intended clinical state across relevant populations and sites. Intended use then determines which errors matter. For cohort enrichment, moderate sensitivity may be acceptable if positive predictive value is high. Outcome ascertainment may require balanced sensitivity and specificity across exposure groups. Clinical decision support requires stronger evidence because errors can affect individual care. Portability studies also show that documentation and implementation differences can change performance after transfer. When reporting phenotype validation, should authors name the intended use before selecting the reference standard and performance thresholds? Which claim should be withdrawn if the study reports chart-review agreement but does not test transportability, subgroup performance, or consequences of false classifications?

0 comments3 linked sources