Article

Automating Evaluation of LLM-generated Responses to Patient Questions about Rare Diseases

2025-10-07

Abstract excerpt

<h4>Objectives</h4> Patients with rare diseases often struggle to find accurate medical information, and large language model (LLM)-based chatbots may help meet this need. However, evaluating LLM-generated free-text answers typically requires physician review, which is time-consuming and difficult to scale. This study compared traditional natural language processing (NLP) metrics to emerging LLM-based evaluation...

Topics

Open a Topic to create a Post that cites this publication.

Genomics and Rare Diseases

Identifiers and source

Literature Corpus work: e411adb2-7967-54e3-b1a7-8d52e061007d
DOI: 10.1101/2025.10.06.25337181

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Automating Evaluation of LLM-generated Responses to Patient Questions about Rare DiseasesDOI 10.1101/2025.10.06.25337181Read

Comparative Performance of Large Language Model-Based Chatbots, Dental Students, and Postgraduate Specialty Students in Text-Based Oral and Maxillofacial Radiology QuestionsSemantic proximity Read

Comparative Analysis of Large Language Models against the NHS 111 Online Triaging for Emergency OphthalmologySemantic proximity Read

Automated Evaluation of Large Language Model Response Concordance with Human Specialist Responses on Physician-to-Physician eConsult CasesSemantic proximity Read

Performance Evaluation of Large Language Model Managing Clinical Trial-Related Adverse EventsSemantic proximity Read

Performance of ChatGPT-4 and Bard Chatbots in Responding to Common Patient Questions on Prostate Cancer 177Lu-PSMA-617 TherapySemantic proximity Read

Entity-centric evaluation of large language model responses for medical question-answering tasksSemantic proximity Read

Comparing Five Generative AI Chatbots’ Answers to LLM-Generated Clinical Questions with Medical Information Scientists’ Evidence SummariesSemantic proximity Read

Comparison of Large Language Models in Answering Immuno-Oncology Questions: A Cross-Sectional StudySemantic proximity Read

Physician Evaluations of Large Language Model-Generated Responses to Medical Questions by Region and Years in Practice: A preliminary studySemantic proximity Read

Performance of Next-Generation AI Chatbots in Gynecological Knowledge Assessment: A Comparative Pilot Study of ChatGPT-5, Gemini-3, DeepSeek-V3.2, and Claude-4.5-OpusSemantic proximity Read

Utility of Large Language Models for Health Care Professionals and Patients in Navigating Hematopoietic Stem Cell Transplantation: Comparison of the Performance of ChatGPT-3.5, ChatGPT-4, and BardSemantic proximity Read

Human versus Artificial Intelligence: ChatGPT-4 Outperforming Bing, Bard, ChatGPT-3.5, and Humans in Clinical Chemistry Multiple-Choice QuestionsSemantic proximity Read