Article

Retrospective Quality Analysis of a Clinical RAG Chatbot: Observable Signals and Lessons Learned

2026-01-27

Abstract excerpt

Retrieval-augmented generation (RAG) is increasingly adopted to ground clinical conversational agents in external knowledge sources, yet many deployed prototypes lack the observability required for standard RAG evaluation. In particular, retrieved documents and grounding context are often not logged, preventing direct assessment of retrieval quality and faithfulness. We report a post-hoc evaluation of EMSy , a cl...

Topics

Open a Topic to create a Post that cites this publication.

Genomics and Rare Diseases

Identifiers and source

Literature Corpus work: 20ec9213-dd13-56cf-a3c3-7b528a86bed3
DOI: 10.64898/2026.01.26.26344757

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Retrospective Quality Analysis of a Clinical RAG Chatbot: Observable Signals and Lessons LearnedDOI 10.64898/2026.01.26.26344757Read

Toward Trustworthy Chatbots: A Protocol for Red Teaming for Health Related ConversationsSemantic proximity Read

From Concept to Clinic: Real World Evidence for Autonomous AI Deployment in Primary Care TelemedicineSemantic proximity Read

Protocol For Human Evaluation of Artificial Intelligence Chatbots in Clinical ConsultationsSemantic proximity Read

CLINIKIOSK-An innovative technology to expand health care accessSemantic proximity Read

Implementation of Large Language Models in Electronic Health RecordsSemantic proximity Read

Mitigating Hallucinations in Large Language Models: A Comparative Study of RAG-enhanced vs. Human-Generated Medical TemplatesSemantic proximity Read

An auditable, retrieval-grounded agent architecture improves medical accuracy and conversational safety in language modelsSemantic proximity Read

AM-RAF: An Agentic, Retrieval-Augmented, Uncertainty-Gated Framework for Trustworthy Clinical Decision Support in Resource-Constrained Health SystemsSemantic proximity Read

A Multi-Agent clinical pre-consultation system for structuring noisy patient reported information into clinical reports and AI-ready dataSemantic proximity Read

Augmenting Large Language Models and Retrieval-Augmented Generation with an Evidence-Based Medicine-Enabled Agent SystemSemantic proximity Read

Multilingual Evaluation of a Large Language Model-Based Primary Care ChatbotSemantic proximity Read

How easily can AI chatbots spread misinformation in audiology and otolaryngology?Semantic proximity Read