Article

Textbook-Level Medical Knowledge in Large Language Models: A Comparative Evaluation Using the Japanese National Medical Examination

2025-09-12

Abstract excerpt

<h4>Study aims and objectives</h4> This study aimed to evaluate the performance of four reasoning-enhanced large language models (LLMs)—GPT-5, Grok-4, Claude Opus 4.1, and Gemini 2.5 Pro—on the Japanese National Medical Examination (JNME). <h4>Methods</h4> We evaluated LLM performance using the 2019 and 2025 JNME (n = 793). Questions were entered into each model with chain-of-thought prompting enabled. Accuracy...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: b43d3508-d8c7-5f78-920f-4790189f7640
DOI: 10.1101/2025.09.10.25335398

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Textbook-Level Medical Knowledge in Large Language Models: A Comparative Evaluation Using the Japanese National Medical ExaminationDOI 10.1101/2025.09.10.25335398Read

Performance of Advanced Large Language Models (GPT-4o, GPT-4, Gemini 1.5 Pro, Claude 3 Opus) on Japanese Medical Licensing Examination: A Comparative StudySemantic proximity Read

Performance of reasoning large language models on nephrology multiple-choice questionsSemantic proximity Read

Comparative Performance of General-Purpose Large Language Models and a Specialized Clinical AI Tool on OKAP-Style Ophthalmology QuestionsSemantic proximity Read

Comparative accuracy of ChatGPT-o1, DeepSeek R1, and Gemini 2.0 in answering general primary care questionsSemantic proximity Read

Evaluating the Performance of Large Language Models on a Neurology Board-Style ExaminationSemantic proximity Read

Performance of o1 pro and GPT-4 in self-assessment questions for nephrology board renewalSemantic proximity Read

Board-Level Performance of Leading Open-Weight Vision-Language Models on the Japanese Diagnostic Radiology Board Examination: Reasoning, Image-Input, and Language EffectsSemantic proximity Read

Evaluating Chat GPT-4o’s Comparative Performance Over GPT-4 in Japanese Medical Licensing Examination and Its Clinical Partnership PotentialSemantic proximity Read

Comparison of Large Language Models’ Performance on Neurosurgical Board Examination QuestionsSemantic proximity Read

General purpose large language models match human performance on gastroenterology board exam self-assessmentsSemantic proximity Read

Language-dependent variability in large language model performance on pharmaceutical knowledge tasksSemantic proximity Read

Benchmarking the Confidence of Large Language Models in Clinical QuestionsSemantic proximity Read