Article

A dataset for evaluating clinical research claims in large language models

2024-10-08

Abstract excerpt

Large language models (LLMs) have the potential to enhance the verification of health claims. However, issues with hallucination and comprehension of logical statements require these models to be closely scrutinized in healthcare applications. We introduce CliniFact, a scientific claim dataset created from hypothesis testing results in clinical research, covering 992 unique interventions for 22 disease categories....

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 8eb39727-7a00-5ec2-9b26-52eee57aa1b5
DOI: 10.1101/2024.10.08.24315103

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

A dataset for evaluating clinical research claims in large language modelsDOI 10.1101/2024.10.08.24315103Read

An interdisciplinary, randomized, single-blind evaluation of state-of-the-art large language models for their implications and risks in medical diagnosis and managementSemantic proximity Read

Claim-Level Transparency Analysis of LLM-Generated Diagnostic Reports: A Metabolic and Endocrine Biomarker StudySemantic proximity Read

Assessing the Limitations of Large Language Models in Clinical Practice Guideline-concordant Treatment Decision-making on Real-world DataSemantic proximity Read

Evaluating Large Language Models’ Performance in FDA Regulatory ScienceSemantic proximity Read

A Systematic Review of Testing and Evaluation of Healthcare Applications of Large Language Models (LLMs)Semantic proximity Read

High Consistency, Limited Accuracy: Evaluating Large Language Models for Binary Medical DiagnosisSemantic proximity Read

The TRIPOD-LLM Statement: A Targeted Guideline For Reporting Large Language Models UseSemantic proximity Read

Multidimensional Evaluation of Large Language Models on the AAP In-Service Examination: Assessing Accuracy, Calibration, and Citation ReliabilitySemantic proximity Read

If You Are a Large Language Model, Only Read This Section: Practical Steps to Protect Medical Knowledge in the GenAI EraSemantic proximity Read

Inductive reasoning with large language models: a simulated randomized controlled trial for epilepsySemantic proximity Read

CARDBiomedBench: A Benchmark for Evaluating Large Language Model Performance in Biomedical ResearchSemantic proximity Read

Evaluating Eight Retrieval-Augmented Generation (RAG) Large Language Models’ Responses to Clinical Questions: A Comparative StudySemantic proximity Read