Article

Automating Evaluation of AI Text Generation in Healthcare with a Large Language Model (LLM)-as-a-Judge

2025-04-22

Abstract excerpt

Electronic Health Records (EHRs) store vast amounts of clinical information that are difficult for healthcare providers to summarize and synthesize relevant details to their practice. To reduce cognitive load on providers, generative AI with Large Language Models have emerged to automatically summarize patient records into clear, actionable insights and offload the cognitive burden for providers. However, LLM summ...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 7a2d0b89-c684-544e-a367-47be8b3f8eb3
DOI: 10.1101/2025.04.22.25326219

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Automating Evaluation of AI Text Generation in Healthcare with a Large Language Model (LLM)-as-a-JudgeDOI 10.1101/2025.04.22.25326219Read

Large Language Model-Powered Administrative AI Agents for Healthcare Documentation Automation: A GPT-4 Framework for Clinical WorkflowsSemantic proximity Read

Retrieval-Augmented Generation (RAG) in Healthcare: A Comprehensive ReviewSemantic proximity Read

MRQC-LLM: A Novel Large Language Model Framework for Enhancing Medical Record Quality ControlSemantic proximity Read

Can Electronic care planning using AI Summarization Yield equal Documentation Quality? (EASY eDocQ)Semantic proximity Read

Comparative Performance of agentic AI and Physicians in Taking Clinical History across Leading Large Language Models (LLMs)Semantic proximity Read

CLEVER: Clinical Large Language Model Evaluationby Expert ReviewSemantic proximity Read

A Review of Large Language Models in Medical Education, Clinical Decision Support, and Healthcare AdministrationSemantic proximity Read

Evaluating Large Language Models for Drafting Emergency Department Discharge SummariesSemantic proximity Read

Towards Inpatient Discharge Summary Automation via Large Language Models: A Multidimensional Evaluation with a HIPAA-Compliant Instance of GPT-4o and Clinical Expert AssessmentSemantic proximity Read

A Framework to Assess Clinical Safety and Hallucination Rates of LLMs for Medical Text SummarisationSemantic proximity Read

Verifiable Summarization of Electronic Health Records Using Large Language Models to Support Chart ReviewSemantic proximity Read

Automated Evaluation of Large Language Model Response Concordance with Human Specialist Responses on Physician-to-Physician eConsult CasesSemantic proximity Read