Article

Collaborative intelligence in AI: Evaluating the performance of a council of AIs on the USMLE

2025-02-20

Abstract excerpt

The variability in responses generated by Large Language Models (LLMs) like OpenAI’s GPT-4 poses challenges in ensuring consistent accuracy on medical knowledge assessments, such as the United States Medical Licensing Exam (USMLE). This study introduces a novel multi-agent framework—referred to as a "Council of AIs"—to enhance LLM performance through collaborative decision-making. The Council consists of multiple...

Topics

Open a Topic to create a Post that cites this publication.

Genomics and Rare Diseases

Identifiers and source

Literature Corpus work: 53820eea-1d0b-58dd-ae80-608d4edbabd3
DOI: 10.1101/2025.02.17.25322388

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Collaborative intelligence in AI: Evaluating the performance of a council of AIs on the USMLEDOI 10.1101/2025.02.17.25322388Read

Multidisciplinary large language model agent teams for precision oncology enhance complex gynecologic oncology decision supportSemantic proximity Read

Evaluating Large Language Model Diagnostic Performance on JAMA Clinical Challenges via a Multi-Agent Conversational FrameworkSemantic proximity Read

Evaluating Capabilities of Large Language Models: Performance of GPT4 on Surgical Knowledge AssessmentsSemantic proximity Read

Performance of ChatGPT on free-response, clinical reasoning examsSemantic proximity Read

Evaluation of ChatGPT-4o’s and DeepSeek R1’s responses to urological problems: A comparative studySemantic proximity Read

From Tool to Teammate: A Randomized Controlled Trial of Clinician-AI Collaborative Workflows for DiagnosisSemantic proximity Read

Comparative Performance of Large Language Model-Based Chatbots, Dental Students, and Postgraduate Specialty Students in Text-Based Oral and Maxillofacial Radiology QuestionsSemantic proximity Read

Comparative Performance of General-Purpose Large Language Models and a Specialized Clinical AI Tool on OKAP-Style Ophthalmology QuestionsSemantic proximity Read

ChatGPT takes the FCPS exam in Internal MedicineSemantic proximity Read

Med-ICE: Enhancing Factual Accuracy in Medical AI through Autonomous Multi-Agent ConsensusSemantic proximity Read

Performance of Advanced Large Language Models (GPT-4o, GPT-4, Gemini 1.5 Pro, Claude 3 Opus) on Japanese Medical Licensing Examination: A Comparative StudySemantic proximity Read

Clinical Knowledge and Reasoning Abilities of AI Large Language Models in Anesthesiology: A Comparative Study on the ABA ExamSemantic proximity Read