Article

Entity-centric evaluation of large language model responses for medical question-answering tasks

2025-11-14

Abstract excerpt

<h4>Objective</h4> Develop a metric for evaluating the clinical alignment and informativeness of large language model (LLM)-generated responses in medical question-answering (QA) tasks. <h4>Materials and methods</h4> We propose EntQA, an entity-centric metric that extracts biomedical entities from patient backgrounds, diagnostic questions and LLM responses using a biomedical named entity recognition model, follo...

Topics

Open a Topic to create a Post that cites this publication.

Genomics and Rare Diseases

Identifiers and source

Literature Corpus work: 87c40071-b840-52b4-bf49-2810b1d25630
DOI: 10.1101/2025.11.12.25340106

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Entity-centric evaluation of large language model responses for medical question-answering tasksDOI 10.1101/2025.11.12.25340106Read

CLEVER: Clinical Large Language Model Evaluationby Expert ReviewSemantic proximity Read

Auditing frontier general-purpose large language models in biomedical tasks: reasoning gains, extraction limits, and benchmark reliabilitySemantic proximity Read

Evaluation of Large Language Model Performance on the Biomedical Language Understanding and Reasoning Benchmark: Comparative StudySemantic proximity Read

Enhancing LLM Accuracy and Reducing Hallucinations using Query Refinement Technique and Knowledge GraphsSemantic proximity Read

Automated Evaluation of Large Language Model Response Concordance with Human Specialist Responses on Physician-to-Physician eConsult CasesSemantic proximity Read

CARDBiomedBench: A Benchmark for Evaluating Large Language Model Performance in Biomedical ResearchSemantic proximity Read

Systematic Review of Large Language Models for Patient Care: Current Applications and ChallengesSemantic proximity Read

MedScope: A Lightweight Benchmark of Open-Source Large Language Models for Medical Question AnsweringSemantic proximity Read

Grounded large language models for diagnostic prediction in real-world emergency department settingsSemantic proximity Read

Automating Evaluation of LLM-generated Responses to Patient Questions about Rare DiseasesSemantic proximity Read

The Economics of Accuracy for Medical Reasoning with Large Language ModelsSemantic proximity Read

Developing an open-source framework for LLM evaluation of patients using EHR clinical documentation; performance of LLMs relative to medical professionalsSemantic proximity Read