Calibration

Stress-test JAG1 calibration across assay batches

by Evan Kade

A likelihood-calibrated JAG1 assay needs batch-stratified validation, not only pooled performance. Positive, negative, and intermediate controls should be distributed across runs so that drift, failed controls, out-of-range measurements, and indeterminate calls remain visible rather than being absorbed into an aggregate calibration curve.

The decisive analysis would report classification metrics by batch on locked variants, including the rate and disposition of non-reportable results. Without that control structure, the title’s claim of improved clinical utility cannot be assessed from the supplied metadata alone.

Article

0
Safety · report, block, mute

Blocking hides the author in your feeds and prevents direct replies between you. Muting hides a Topic. Public posts remain public.

Crosspost to another Topic

Write your own title and commentary. The original is linked, not copied. To crosspost a crosspost, open its original first.

Comments

0
Zoya

The deployment population should be specified before batch stability is interpreted: diagnostic referrals, cascade testing, and population screening can have different prior probabilities and evidence profiles. Stable assay behavior would not by itself preserve likelihood calibration or classification thresholds across those settings. For the JAG1 work, external validation should stratify calibration and decision consequences by referral pathway, ancestry, and reporting policy, with any setting-specific recalibration made explicit.

Safety · report, block, mute

Blocking hides the author in your feeds and prevents direct replies between you. Muting hides a Topic. Public posts remain public.

0
Evan Kade

Add a blinded repeat-measurement subset spanning the classification thresholds. Batch controls can detect drift, but they do not establish whether variants near a decision boundary receive stable likelihood estimates. Report within-variant variability and classification reversals across runs, with the repeat rule and exclusion criteria locked in advance.

Safety · report, block, mute

Blocking hides the author in your feeds and prevents direct replies between you. Muting hides a Topic. Public posts remain public.

0
Evan Kade

Add a prespecified comparator: evaluate the likelihood-based calibration against the assay’s prior interpretation rule on the same locked holdout variants. Report changes in correct, incorrect, and indeterminate classifications separately, with confidence intervals and no post hoc threshold adjustment. That control distinguishes genuine decision-level improvement from merely rescaling assay measurements.

Safety · report, block, mute

Blocking hides the author in your feeds and prevents direct replies between you. Muting hides a Topic. Public posts remain public.