Article

Evaluating Large Language Model Diagnostic Performance on JAMA Clinical Challenges via a Multi-Agent Conversational Framework

2025-08-24

Abstract excerpt

<h4>Background & Objective</h4> Standard clinical LLM benchmarks use multiple-choice vignettes that present all information up front, unlike real encounters where clinicians iteratively elicit histories and objective data. We hypothesized that such formats inflate LLM performance and mask weaknesses in diagnostic reasoning. We developed and evaluated a multi-AI agent conversational framework that converts JAMA Cl...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: b59a3ee6-ffc7-512e-ae0a-4bde3c7b6691
DOI: 10.1101/2025.08.20.25334087

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Evaluating Large Language Model Diagnostic Performance on JAMA Clinical Challenges via a Multi-Agent Conversational FrameworkDOI 10.1101/2025.08.20.25334087Read

One is Not Enough: Multi-Agent Conversation Framework Enhances Rare Disease Diagnostic Capabilities of Large Language ModelsSemantic proximity Read

Comparative Performance of agentic AI and Physicians in Taking Clinical History across Leading Large Language Models (LLMs)Semantic proximity Read

AI vs Human Performance in Conversational Hospital-Based Neurological DiagnosisSemantic proximity Read

MedClarify: An information-seeking AI agent for medical diagnosis with case-specific follow-up questionsSemantic proximity Read

Evaluating the AI Potential as a Safety Net for Diagnosis: A Novel Benchmark of Large Language Models in Correcting Diagnostic ErrorsSemantic proximity Read

From Tool to Teammate: A Randomized Controlled Trial of Clinician-AI Collaborative Workflows for DiagnosisSemantic proximity Read

Multidisciplinary large language model agent teams for precision oncology enhance complex gynecologic oncology decision supportSemantic proximity Read

Evaluating a Large Reasoning Model’s Performance on Open-Ended Medical ScenariosSemantic proximity Read

Neurosymbolic Multi-Agent Artificial Intelligence versus General-Purpose Large Language Models for Clinical Decision Support in Ileus and VolvulusSemantic proximity Read

Artificial Intelligence - Large Language Models (LLMs) and Agentic AI in Clinical Decision Support in Outpatient, General Practice and Acute Settings: A Systematic Review of Diagnostic Accuracy, Comparative Efficacy, Clinical Outcomes and Implementation FactorsSemantic proximity Read

Assessing the Utility of ChatGPT Throughout the Entire Clinical WorkflowSemantic proximity Read

AI Agents in Clinical Medicine: A Systematic ReviewSemantic proximity Read