Article

Evaluating Large Language Models on Medical Evidence Summarization

2023-04-24

Abstract excerpt

Recent advances in large language models (LLMs) have demonstrated remarkable successes in zero- and few-shot performance on various downstream tasks, paving the way for applications in high-stakes domains. In this study, we systematically examine the capabilities and limitations of LLMs, specifically GPT-3.5 and ChatGPT, in performing zero-shot medical evidence summarization across six clinical domains. We conduct...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 920965c4-bf60-5881-94b3-a3e3f39444f8
DOI: 10.1101/2023.04.22.23288967

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Evaluating Large Language Models on Medical Evidence SummarizationDOI 10.1101/2023.04.22.23288967Read

Almanac: Retrieval-Augmented Language Models for Clinical MedicineSemantic proximity Read

Zero-Shot Large Language Models for Long Clinical Text Summarization with Temporal ReasoningSemantic proximity Read

Patient Centric Summarization of Radiology Findings using Large Language ModelsSemantic proximity Read

Development and Evaluation of a Digital Scribe: Conversation Summarization Pipeline for Emergency Department Counseling Sessions towards Reducing Documentation BurdenSemantic proximity Read

Automating Evaluation of AI Text Generation in Healthcare with a Large Language Model (LLM)-as-a-JudgeSemantic proximity Read

CLEVER: Clinical Large Language Model Evaluationby Expert ReviewSemantic proximity Read

MulMed: Addressing Multiple Medical Tasks Utilizing Large Language ModelsSemantic proximity Read

Large Language Models Are Poor Clinical Decision-Makers: A Comprehensive BenchmarkSemantic proximity Read

Confidence Is Not Probability: Verification-Based Confidence Prediction for LLM-Generated Clinical SummariesSemantic proximity Read

A Framework to Assess Clinical Safety and Hallucination Rates of LLMs for Medical Text SummarisationSemantic proximity Read

Evaluating Large Language Models for Drafting Emergency Department Discharge SummariesSemantic proximity Read

A Comparative Investigation of Zero-shot Prompting and Fine-tuning for Clinical Note SummarizationSemantic proximity Read