Evan Kade

I should distinguish functional evidence-bin crossings from final classification reversals in the proposed JAG1 replicate check, holding other variant evidence fixed to isolate the effect of assay variation.

I should have specified that failed and indeterminate JAG1 runs remain separate outcomes, so their inclusion in the denominator doesn't imply classification stability.

Without prespecified target-population weights, keep the locked estimates stratum-specific and make no pooled transportability claim.

Estimate that count by jointly resampling likelihood weights on locked variants, with failed and indeterminate replicates retained when reporting the fraction of unchanged classifications.

The transport control should be a locked external variant set sampled across prespecified sites or time periods, with each stratum retaining failed, indeterminate, and out-of-range assay results. Apply the unchanged likelihood mapping and thresholds, then report stratum-specific calibration and threshold-crossing reversals before standardizing to the target referral distribution. This separates assay drift from referral-mix dependence and local recalibration.

Independence alone is not enough if the holdout reproduces the development set’s referral mix. A stronger control would preserve site- or time-stratified cohorts and apply the locked mapping separately within each stratum, reporting calibration, threshold crossings, and non-reportable calls. That design would distinguish transportability from good average performance driven by the dominant subgroup.

Add a blinded repeat-measurement subset spanning the classification thresholds. Batch controls can detect drift, but they do not establish whether variants near a decision boundary receive stable likelihood estimates. Report within-variant variability and classification reversals across runs, with the repeat rule and exclusion criteria locked in advance.

Add a prespecified comparator: evaluate the likelihood-based calibration against the assay’s prior interpretation rule on the same locked holdout variants. Report changes in correct, incorrect, and indeterminate classifications separately, with confidence intervals and no post hoc threshold adjustment. That control distinguishes genuine decision-level improvement from merely rescaling assay measurements.