Article

Confidence Is Not Probability: Verification-Based Confidence Prediction for LLM-Generated Clinical Summaries

2026-07-28

Abstract excerpt

<h4>Objective: </h4> Token-level confidence estimates from large language models (LLMs) do not reflect whether a generated clinical summary is factually correct. We investigate whether verification-based confidence prediction improves upon token-level estimates and whether LLM-as-judge labels can be assumed reliable for clinical summarization evaluation. Methods. Using MIMIC-IV, we construct 500 clinical summariza...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: b8cf0f75-79f2-5158-809e-32a9ff9901c6
DOI: 10.20944/preprints202607.1999.v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Confidence Is Not Probability: Verification-Based Confidence Prediction for LLM-Generated Clinical SummariesDOI 10.20944/preprints202607.1999.v1Read

A Systematic Exploration of LLM Behavior for EHR PhenotypingSemantic proximity Read

A statistical framework for evaluating the repeatability and reproducibility of large language modelsSemantic proximity Read

A Framework to Assess Clinical Safety and Hallucination Rates of LLMs for Medical Text SummarisationSemantic proximity Read

High Consistency, Limited Accuracy: Evaluating Large Language Models for Binary Medical DiagnosisSemantic proximity Read

Automated Evaluation of Large Language Model Response Concordance with Human Specialist Responses on Physician-to-Physician eConsult CasesSemantic proximity Read

Evaluating Large Language Models on Medical Evidence SummarizationSemantic proximity Read

The use of a large language models to create plain language summaries of evidence reviews in healthcare: a feasibility studySemantic proximity Read

CLEVER: Clinical Large Language Model Evaluationby Expert ReviewSemantic proximity Read

A dataset for evaluating clinical research claims in large language modelsSemantic proximity Read

Confidence-linked and uncertainty-based staged framework for phenotype validation using large language models.Semantic proximity Read

Evaluating the AI Potential as a Safety Net for Diagnosis: A Novel Benchmark of Large Language Models in Correcting Diagnostic ErrorsSemantic proximity Read

Developing an open-source framework for LLM evaluation of patients using EHR clinical documentation; performance of LLMs relative to medical professionalsSemantic proximity Read