Article

Systematic benchmarking demonstrates large language models have not reached the diagnostic accuracy of traditional rare-disease decision support tools

2024-07-22

Abstract excerpt

Large language models (LLMs) show promise in supporting differential diagnosis, but their performance is challenging to evaluate due to the unstructured nature of their responses and their accuracy compared to existing diagnostic tools is not well characterized. To assess the current capabilities of LLMs to diagnose genetic diseases, we benchmarked these models on 5,213 case reports using the Phenopacket Schema, t...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: f881f52b-0499-53ff-8e6b-f35f69e3a40c
DOI: 10.1101/2024.07.22.24310816

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Systematic benchmarking demonstrates large language models have not reached the diagnostic accuracy of traditional rare-disease decision support toolsDOI 10.1101/2024.07.22.24310816Read

Diagnostic Accuracy of a Custom Large Language Model on Rare Pediatric Disease Case Reports.Semantic proximity Read

Consistent Performance of GPT-4o in Rare Disease Diagnosis Across Nine Languages and 4967 CasesSemantic proximity Read

Extraction of Human Phenotype Ontology (HPO) Concepts from Clinical Notes Utilizing Large Language Models (LLM) with Model Context Protocol (MCP)Semantic proximity Read

Genosolver: Rare Disease Diagnosis through Holistic Integration of Unstructured Clinical Narratives Using Large Language and Reasoning ModelsSemantic proximity Read

Large language models facilitate the generation of electronic health record phenotyping algorithms.Semantic proximity Read

Large Language Models Facilitate the Generation of Electronic Health Record Phenotyping AlgorithmsSemantic proximity Read

On the limitations of large language models in clinical diagnosisSemantic proximity Read

Diagnostic Accuracy of Large Language Models for Rare Diseases: A Systematic Review and Meta-AnalysisSemantic proximity Read

ChatGPT vs DeepSeek: A Comparative Study of Diagnostic Accuracy and Clinical Reasoning in Rare and Complex DiseasesSemantic proximity Read

Applying natural language processing and large language models to clinical notes for phenotyping and diagnosing rare diseases: a systematic review.Semantic proximity Read

An agentic system for rare disease diagnosis with traceable reasoning.Semantic proximity Read

A hybrid framework with large language models for rare disease phenotyping.Semantic proximity Read