Article

Developing an open-source framework for LLM evaluation of patients using EHR clinical documentation; performance of LLMs relative to medical professionals

2026-08-24

Abstract excerpt

<h4>Background: </h4> Large language models (LLMs) have shown increasing capability in medical knowledge tasks, yet how they perform in extracting structured clinical information from real-world clinical documentation remains uncertain. We evaluated the performance of LLMs relative to medical professionals in extracting SNOMED-coded clinical information from openly available Ear, Nose and Throat (ENT) EHRs from MT...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 0a07895b-92f4-59f9-928d-fca1806d7e78
DOI: 10.64898/2026.08.21.26361031

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Developing an open-source framework for LLM evaluation of patients using EHR clinical documentation; performance of LLMs relative to medical professionalsDOI 10.64898/2026.08.21.26361031Read

The Transformative Potential of Large Language Models in Mining Electronic Health Records DataSemantic proximity Read

On the limitations of large language models in clinical diagnosisSemantic proximity Read

How well it works: Benchmarking performance of GPT models on medical natural language processing tasksSemantic proximity Read

General-purpose large language models can achieve physician-level accuracy in complex medical data extractionSemantic proximity Read

Extraction of Human Phenotype Ontology (HPO) Concepts from Clinical Notes Utilizing Large Language Models (LLM) with Model Context Protocol (MCP)Semantic proximity Read

A Systematic Exploration of LLM Behavior for EHR PhenotypingSemantic proximity Read

High Consistency, Limited Accuracy: Evaluating Large Language Models for Binary Medical DiagnosisSemantic proximity Read

Enhancing Doctor-Patient Communication Using Large Language Models for Pathology Report InterpretationSemantic proximity Read

Systematic benchmarking demonstrates large language models have not reached the diagnostic accuracy of traditional rare-disease decision support toolsSemantic proximity Read

Systematic Literature Review of LLM-Large Language Model in Medical: Digital Health, Technology and ApplicationsSemantic proximity Read

Prompt Engineering Enables Open-Source LLMs to Match Proprietary Models in Diagnostic Accuracy for Annotation of Radiology ReportsSemantic proximity Read

Large language models for automatable real-world performance monitoring of diagnostic decision support systems: a comparison to manual doctor panel review in a prospective clinical studySemantic proximity Read