Article

FHIR-AgentEval: A Modular Sandbox for Benchmarking Clinical LLM Agents with an Evaluation of Memory-Augmented Configurations

2026-02-09

Abstract excerpt

<title>Abstract</title> <p>Healthcare data exchange increasingly relies on HL7 FHIR, but FHIR's implementation complexity creates barriers for clinical workflows. Large language model (LLM) agents could bridge this gap by translating natural language requests into structured FHIR operations, yet their reliability remains unproven. We present FHIR-AgentEval, an extensible evaluation sandbox comprising 43 modular t...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: d2dad0fa-06cd-5e8b-ad65-bb8f6439e0f7
DOI: 10.21203/rs.3.rs-8746188/v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

FHIR-AgentEval: A Modular Sandbox for Benchmarking Clinical LLM Agents with an Evaluation of Memory-Augmented ConfigurationsDOI 10.21203/rs.3.rs-8746188/v1Read

Beyond Benchmarks: Dynamic, Automatic And Systematic Red-Teaming Agents For Trustworthy Medical Language ModelsSemantic proximity Read

Retrieval-Augmented Generation in Biomedicine: A Survey of Technologies, Datasets, and Clinical ApplicationsSemantic proximity Read

AM-RAF: An Agentic, Retrieval-Augmented, Uncertainty-Gated Framework for Trustworthy Clinical Decision Support in Resource-Constrained Health SystemsSemantic proximity Read

MRQC-LLM: A Novel Large Language Model Framework for Enhancing Medical Record Quality ControlSemantic proximity Read

AI Agents in Clinical Medicine: A Systematic ReviewSemantic proximity Read

Implementation of Large Language Models in Electronic Health RecordsSemantic proximity Read

An auditable, retrieval-grounded agent architecture improves medical accuracy and conversational safety in language modelsSemantic proximity Read

Research through Evaluation for Large Language Model in Patient-Clinician CommunicationsSemantic proximity Read

Improved precision oncology question-answering using agentic LLMSemantic proximity Read

Hard to Halt: Automation Bias in Agent-Driven Sequencing Prior Authorization WorkflowsSemantic proximity Read

A Systematic Review of Testing and Evaluation of Healthcare Applications of Large Language Models (LLMs)Semantic proximity Read

A ReAct Agentic AI System for Natural Language Querying and Statistical Analysis of The Cancer Genome Atlas Clinical DataSemantic proximity Read