Article

High Consistency, Limited Accuracy: Evaluating Large Language Models for Binary Medical Diagnosis

2025-12-09

Abstract excerpt

<h4>ABSTRACT</h4> <h4>Background</h4> Large Language Models (LLMs) have demonstrated impressive capabilities in medical knowledge tasks, achieving 60-80% accuracy on licensing examinations. However, their reliability and consistency in clinical diagnosis—critical for clinical trustworthiness–remain incompletely characterized. <h4>Objective</h4> To systematically evaluate the consistency and diagnostic accuracy...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: cd409f2b-6571-57ce-8f3e-ff599ce9bb33
DOI: 10.64898/2025.12.08.25341823

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

High Consistency, Limited Accuracy: Evaluating Large Language Models for Binary Medical DiagnosisDOI 10.64898/2025.12.08.25341823Read

ChatGPT vs DeepSeek: A Comparative Study of Diagnostic Accuracy and Clinical Reasoning in Rare and Complex DiseasesSemantic proximity Read

Development and retrospective validation of SCOUT: scalable clinical oversight of large language models via uncertainty triangulationSemantic proximity Read

Diagnostic Accuracy of Large Language Models for Rare Diseases: A Systematic Review and Meta-AnalysisSemantic proximity Read

Evaluating the AI Potential as a Safety Net for Diagnosis: A Novel Benchmark of Large Language Models in Correcting Diagnostic ErrorsSemantic proximity Read

Reasoning Beyond Accuracy: Expert Evaluation of Large Language Models in Diagnostic PathologySemantic proximity Read

Novel Insights into the Application of Large Language Models in the Diagnosis and Treatment of Complex Cardiovascular Diseases: A Comparative StudySemantic proximity Read

Understanding Clinical Reasoning Variability in Medical Large Language Models: A Mechanistic Interpretability StudySemantic proximity Read

A Systematic Exploration of LLM Behavior for EHR PhenotypingSemantic proximity Read

A Novel Framework for Evaluating the Clinical Reasoning Process of Large Language Models: A Comparative Study in NephrologySemantic proximity Read

Grounded large language models for diagnostic prediction in real-world emergency department settingsSemantic proximity Read

An interdisciplinary, randomized, single-blind evaluation of state-of-the-art large language models for their implications and risks in medical diagnosis and managementSemantic proximity Read

Diagnostic Performance of Large Language Models and Radiologists in Case-Based Radiology QuestionsSemantic proximity Read