Article

A Clinically-Informed Framework for Evaluating Vision-Language Models in Radiology Report Generation: Taxonomy of Errors and Risk-Aware Metric

2025-07-14

Abstract excerpt

Recent advances in vision-language models (VLMs) have enabled automatic radiology report generation, yet current evaluation methods remain limited to general-purpose NLP metrics or coarse classification-based clinical scores. In this study, we propose a clinically informed evaluation framework for VLM-generated radiology reports that goes beyond traditional performance measures. We define a taxonomy of 12 radiolog...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 71d106c3-3ae8-57e4-a4c7-8e25fcac34fc
DOI: 10.1101/2025.07.13.25331222

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

A Clinically-Informed Framework for Evaluating Vision-Language Models in Radiology Report Generation: Taxonomy of Errors and Risk-Aware MetricDOI 10.1101/2025.07.13.25331222Read

RadStructAgent: Agentic Structuring of Free-Text Chest X-ray Reports with Small Language ModelsSemantic proximity Read

Development of a Human Evaluation Framework and Correlation with Automated Metrics for Natural Language Generation of Medical DiagnosesSemantic proximity Read

Automating Radiology Report Generation: A Systematic Review of Deep Learning MethodsSemantic proximity Read

Expert-level validation of AI-generated medical text with scalable language modelsSemantic proximity Read

Language-dependent diagnostic safety of medical AI systems: a cross-lingual benchmarking and prospective clinical studySemantic proximity Read

Consensus, dissensus and synergy between clinicians and specialist foundation models in radiology report generationSemantic proximity Read

A statistical framework for evaluating the repeatability and reproducibility of large language modelsSemantic proximity Read

DISCERN: A Clinical Impact-aware Framework for Radiology Report ComparisonSemantic proximity Read

A Framework to Assess Clinical Safety and Hallucination Rates of LLMs for Medical Text SummarisationSemantic proximity Read

Large Language Models in Radiology Reporting—A Systematic Review of Performance, Limitations, and Clinical ImplicationsSemantic proximity Read

An Open-source Fine-tuned Large Language Model for Radiological Impression Generation: A Multi-reader Performance StudySemantic proximity Read

Enhancing Doctor-Patient Communication Using Large Language Models for Pathology Report InterpretationSemantic proximity Read