Article

Language-dependent variability in large language model performance on pharmaceutical knowledge tasks

2026-03-27

Abstract excerpt

<title>Abstract</title> <p>Large language models (LLMs) are increasingly being considered for applications in healthcare systems. However, their reliability across languages has not been fully examined. In this study, we investigated language-dependent differences in LLM performance on pharmaceutical knowledge questions using datasets derived from Japanese professional examinations. A total of 32 open-weight LLMs...

Topics

Open a Topic to create a Post that cites this publication.

Genomics and Rare Diseases

Identifiers and source

Literature Corpus work: 2e4ff75f-2d97-5e26-bf68-233751e76107
DOI: 10.21203/rs.3.rs-9217016/v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Language-dependent variability in large language model performance on pharmaceutical knowledge tasksDOI 10.21203/rs.3.rs-9217016/v1Read

Proprietary and Open-Source Large Language Models on the Korean Pharmacist Licensing Examination: A Comparative Benchmarking StudySemantic proximity Read

Performance of Advanced Large Language Models (GPT-4o, GPT-4, Gemini 1.5 Pro, Claude 3 Opus) on Japanese Medical Licensing Examination: A Comparative StudySemantic proximity Read

Textbook-Level Medical Knowledge in Large Language Models: A Comparative Evaluation Using the Japanese National Medical ExaminationSemantic proximity Read

Comparative Performance of General-Purpose Large Language Models and a Specialized Clinical AI Tool on OKAP-Style Ophthalmology QuestionsSemantic proximity Read

Clinical Knowledge and Reasoning Abilities of AI Large Language Models in Anesthesiology: A Comparative Study on the ABA ExamSemantic proximity Read

Psychometric Alignment Between Human and Artificial Intelligence Performance in Cardiology Residency In-Service ExaminationsSemantic proximity Read

A Systematic Review of Large Language Models in Medical Specialties: Applications, Challenges and Future DirectionsSemantic proximity Read

Evaluating the Performance of Large Language Models on a Neurology Board-Style ExaminationSemantic proximity Read

Benchmarking the Confidence of Large Language Models in Clinical QuestionsSemantic proximity Read

Comparative Performance of Large Language Model-Based Chatbots, Dental Students, and Postgraduate Specialty Students in Text-Based Oral and Maxillofacial Radiology QuestionsSemantic proximity Read

Comparing the Dental Knowledge of Large Language ModelsSemantic proximity Read

Comparison of Large Language Models’ Performance on Neurosurgical Board Examination QuestionsSemantic proximity Read