Article

Auditing frontier general-purpose large language models in biomedical tasks: reasoning gains, extraction limits, and benchmark reliability

2026-02-18

Abstract excerpt

<title>Abstract</title> <p>As large language models approach clinical deployment, their deployment-relevant reliability and the validity of the benchmarks used to assess it remain insufficiently examined. Here, we present a unified, reproducible, and human-centric audit of frontier general-purpose language models using representative biomedical text-mining tasks and nine biomedical question-answering benchmarks s...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 4d61cb3a-ece4-53c3-a6e8-d6aadd85a514
DOI: 10.21203/rs.3.rs-8605899/v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Auditing frontier general-purpose large language models in biomedical tasks: reasoning gains, extraction limits, and benchmark reliabilityDOI 10.21203/rs.3.rs-8605899/v1Read

CLEVER: Clinical Large Language Model Evaluationby Expert ReviewSemantic proximity Read

Systematic Review of Large Language Models for Patient Care: Current Applications and ChallengesSemantic proximity Read

Evaluation of Large Language Model Performance on the Biomedical Language Understanding and Reasoning Benchmark: Comparative StudySemantic proximity Read

Entity-centric evaluation of large language model responses for medical question-answering tasksSemantic proximity Read

A Systematic Review of Testing and Evaluation of Healthcare Applications of Large Language Models (LLMs)Semantic proximity Read

Understanding Clinical Reasoning Variability in Medical Large Language Models: A Mechanistic Interpretability StudySemantic proximity Read

How well it works: Benchmarking performance of GPT models on medical natural language processing tasksSemantic proximity Read

Performance Characteristics of Reasoning Large Language Models for Evidence Extraction from Clinical Genomics LiteratureSemantic proximity Read

CARDBiomedBench: A Benchmark for Evaluating Large Language Model Performance in Biomedical ResearchSemantic proximity Read

A Systematic Exploration of LLM Behavior for EHR PhenotypingSemantic proximity Read

The Economics of Accuracy for Medical Reasoning with Large Language ModelsSemantic proximity Read

Investigating Expectations and Needs of Medical Professionals Regarding the Use of Large Language Models: A Study at German University ClinicsSemantic proximity Read