Article

Evaluating a Large Language Model’s Ability to Answer Clinicians’ Requests for Evidence Summaries

2024-05-03

Abstract excerpt

<h4>Objective</h4> This study investigated the performance of a generative artificial intelligence (AI) tool using GPT-4 in answering clinical questions in comparison with medical librarians’ gold-standard evidence syntheses. <h4>Methods</h4> Questions were extracted from an in-house database of clinical evidence requests previously answered by medical librarians. Questions with multiple parts were subdivided into...

Topics

Open a Topic to create a Post that cites this publication.

Genomics and Rare Diseases

Identifiers and source

Literature Corpus work: 3859b543-f38b-549e-be1c-bf067ae916de
DOI: 10.1101/2024.05.01.24306691

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Evaluating a Large Language Model’s Ability to Answer Clinicians’ Requests for Evidence SummariesDOI 10.1101/2024.05.01.24306691Read

Evaluating Eight Retrieval-Augmented Generation (RAG) Large Language Models’ Responses to Clinical Questions: A Comparative StudySemantic proximity Read

ACCORD-AI: an exploratory evaluation of large language models for drafting consensus voting itemsSemantic proximity Read

Comparing Five Generative AI Chatbots’ Answers to LLM-Generated Clinical Questions with Medical Information Scientists’ Evidence SummariesSemantic proximity Read

Developing artificial intelligence tools for institutional review board pre-review: A pilot study on ChatGPT’s accuracy and reproducibilitySemantic proximity Read

Generative Artificial Intelligence for Clinical Communication: Implications for Non-Pharmacological Interventions in Health CareSemantic proximity Read

Generative AI Guidelines in Korean Medical Journals: A Survey Using Human-AI CollaborationSemantic proximity Read

Performance of Large Language Models as a Tool for Primary Care Consultations: Evaluation StudySemantic proximity Read

Artificial Intelligence’s Contribution to Biomedical Literature Search: Revolutionizing or Complicating?Semantic proximity Read

AI-Generated Responses to Patient's Messages: Effectiveness, Feasibility and ImplementationSemantic proximity Read

Characterizing large language model generative artificial intelligence variability in the production of objective structured clinical examination stationsSemantic proximity Read

Will generative AI help solve systematic literature reviews? Evidence from a 2-year research programmeSemantic proximity Read

Reproducible Generative AI Evaluation for Healthcare: A Clinician-in-the-Loop ApproachSemantic proximity Read