Article

Performance of Advanced Large Language Models (GPT-4o, GPT-4, Gemini 1.5 Pro, Claude 3 Opus) on Japanese Medical Licensing Examination: A Comparative Study

2024-07-09

Abstract excerpt

<h4>Purpose</h4> This study aims to evaluate the accuracy of medical knowledge in the most advanced LLMs (GPT-4o, GPT-4, Gemini 1.5 Pro, and Claude 3 Opus) as of 2024. It is the first to evaluate these LLMs using a non-English medical licensing exam. The insights from this study will guide educators, policymakers, and technical experts in the effective use of AI in medical education and clinical diagnosis. <h4>Me...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: acfe3ab0-25df-59eb-a4e2-5cabf85cf9be
DOI: 10.1101/2024.07.09.24310129

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Performance of Advanced Large Language Models (GPT-4o, GPT-4, Gemini 1.5 Pro, Claude 3 Opus) on Japanese Medical Licensing Examination: A Comparative StudyDOI 10.1101/2024.07.09.24310129Read

Textbook-Level Medical Knowledge in Large Language Models: A Comparative Evaluation Using the Japanese National Medical ExaminationSemantic proximity Read

Evaluating Chat GPT-4o’s Comparative Performance Over GPT-4 in Japanese Medical Licensing Examination and Its Clinical Partnership PotentialSemantic proximity Read

Multimodal Large Language Model Passes Specialty Board Examination and Surpasses Human Test-Taker Scores: A Comparative Analysis Examining the Stepwise Impact of Model Prompting Strategies on PerformanceSemantic proximity Read

Comparative Performance of General-Purpose Large Language Models and a Specialized Clinical AI Tool on OKAP-Style Ophthalmology QuestionsSemantic proximity Read

Comparison of Large Language Models’ Performance on Neurosurgical Board Examination QuestionsSemantic proximity Read

Evaluation of Large Language Models in Thailand’s National Medical Licensing ExaminationSemantic proximity Read

General purpose large language models match human performance on gastroenterology board exam self-assessmentsSemantic proximity Read

Can ChatGPT Pass the 2023 Japanese National Medical Licensing Examination?Semantic proximity Read

Performance of ChatGPT on Chinese National Medical Licensing Examinations: A Five-Year Examination Evaluation Study for Physicians, Pharmacists and NursesSemantic proximity Read

Performance of Generative Pretrained Transformer on the National Medical Licensing Examination in JapanSemantic proximity Read

Evaluating the Performance of Large Language Models on a Neurology Board-Style ExaminationSemantic proximity Read

Board-Level Performance of Leading Open-Weight Vision-Language Models on the Japanese Diagnostic Radiology Board Examination: Reasoning, Image-Input, and Language EffectsSemantic proximity Read