Article

General purpose large language models match human performance on gastroenterology board exam self-assessments

2023-09-25

Abstract excerpt

<h4>Introduction</h4> While general-purpose large language models(LLMs) were able to pass USMLE-style examinations, their ability to perform in a specialized context, like gastroenterology, is unclear. In this study, we assessed the performance of three widely available LLMs: PaLM-2, GPT-3.5, and GPT-4 on the most recent ACG self-assessment(2022), utilizing both a basic and a prompt-engineered technique. <h4>Metho...

Topics

Open a Topic to create a Post that cites this publication.

Uncategorized literature

Identifiers and source

Literature Corpus work: f643a893-8ed4-520f-8c05-8e585de28b88
DOI: 10.1101/2023.09.21.23295918

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

General purpose large language models match human performance on gastroenterology board exam self-assessmentsDOI 10.1101/2023.09.21.23295918Read

Multimodal Large Language Model Passes Specialty Board Examination and Surpasses Human Test-Taker Scores: A Comparative Analysis Examining the Stepwise Impact of Model Prompting Strategies on PerformanceSemantic proximity Read

Performance of Advanced Large Language Models (GPT-4o, GPT-4, Gemini 1.5 Pro, Claude 3 Opus) on Japanese Medical Licensing Examination: A Comparative StudySemantic proximity Read

Large language models for generating medical examinations: systematic reviewSemantic proximity Read

Large language models for generating medical examinations: systematic reviewSemantic proximity Read

Across Generations, Sizes, and Types, Large Language Models Poorly Report Self-Confidence in Gastroenterology Clinical Reasoning TasksSemantic proximity Read

Evaluating the Performance of Large Language Models on a Neurology Board-Style ExaminationSemantic proximity Read

Performance of GPT-4 on the American College of Radiology In-Service ExaminationSemantic proximity Read

Textbook-Level Medical Knowledge in Large Language Models: A Comparative Evaluation Using the Japanese National Medical ExaminationSemantic proximity Read

Evaluating Large Language Models for Colonoscopy Preparation Assistance: Correctness and Diversity in Synthetic DialoguesSemantic proximity Read

Board-Level Performance of Leading Open-Weight Vision-Language Models on the Japanese Diagnostic Radiology Board Examination: Reasoning, Image-Input, and Language EffectsSemantic proximity Read

Cardiology Knowledge Assessment of Retrieval-Augmented Open versus Proprietary Large Language ModelsSemantic proximity Read

QUEST-AI: A System for Question Generation, Verification, and Refinement using AI for USMLE-Style ExamsSemantic proximity Read