Article

Comparing Five Generative AI Chatbots’ Answers to LLM-Generated Clinical Questions with Medical Information Scientists’ Evidence Summaries

2025-09-27

Abstract excerpt

<h4>Objective</h4> To compare answers to clinical questions between five publicly available large language model (LLM) chatbots and information scientists. <h4>Methods</h4> LLMs were prompted to provide 45 PICO (patient, intervention, comparison, outcome) questions addressing treatment, prognosis, and etiology. Each question was answered by a medical information scientist and submitted to five LLM tools: ChatGPT...

Topics

Open a Topic to create a Post that cites this publication.

Genomics and Rare Diseases

Identifiers and source

Literature Corpus work: 46948221-7cc3-5764-8b78-e69b2f7c6c0a
DOI: 10.1101/2025.09.24.25336199

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Comparing Five Generative AI Chatbots’ Answers to LLM-Generated Clinical Questions with Medical Information Scientists’ Evidence SummariesDOI 10.1101/2025.09.24.25336199Read

Evaluating Eight Retrieval-Augmented Generation (RAG) Large Language Models’ Responses to Clinical Questions: A Comparative StudySemantic proximity Read

Comparison between ChatGPT 3.0 and Google Gemini Regarding Medicine KnowledgeSemantic proximity Read

Comparative Performance of Large Language Model-Based Chatbots, Dental Students, and Postgraduate Specialty Students in Text-Based Oral and Maxillofacial Radiology QuestionsSemantic proximity Read

Utility of Large Language Models for Health Care Professionals and Patients in Navigating Hematopoietic Stem Cell Transplantation: Comparison of the Performance of ChatGPT-3.5, ChatGPT-4, and BardSemantic proximity Read

Comparative accuracy of ChatGPT-o1, DeepSeek R1, and Gemini 2.0 in answering general primary care questionsSemantic proximity Read

Performance of Large Language Models as a Tool for Primary Care Consultations: Evaluation StudySemantic proximity Read

Protocol For Human Evaluation of Artificial Intelligence Chatbots in Clinical ConsultationsSemantic proximity Read

Evaluation of Prompts to Simplify Cardiovascular Disease Information Using a Large Language ModelSemantic proximity Read

Does ChatGPT Provide Appropriate and Equitable Medical Advice?: A Vignette-Based, Clinical Evaluation Across Care ContextsSemantic proximity Read

Human versus Artificial Intelligence: ChatGPT-4 Outperforming Bing, Bard, ChatGPT-3.5, and Humans in Clinical Chemistry Multiple-Choice QuestionsSemantic proximity Read

Arkangel AI, OpenEvidence, ChatGPT, Medisearch: are they objectively up to medical standards? A real-life assessment of LLMs in healthcareSemantic proximity Read

ChatGPT: Transforming Healthcare with AISemantic proximity Read