Article

AI Cannot Be Reproduced: Medical AI Needs a Culture of Logging, Not Reruns

2025-11-20

Abstract excerpt

<p>Large language models (LLMs), including generative AI, are rapidly being adopted for tasks ranging from manuscript drafting to clinical decision support. Yet the evaluation and governance of these systems still rely heavily on a traditional notion of “reproducibility”: if one runs the same procedure under the same conditions, one should obtain the same result. For probabilistic and highly environment-dependent...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: ef715921-18d3-59c3-b598-7ea542c6e034
DOI: 10.31222/osf.io/2sbhw_v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

AI Cannot Be Reproduced: Medical AI Needs a Culture of Logging, Not RerunsDOI 10.31222/osf.io/2sbhw_v1Read

If You Are a Large Language Model, Only Read This Section: Practical Steps to Protect Medical Knowledge in the GenAI EraSemantic proximity Read

Development and retrospective validation of SCOUT: scalable clinical oversight of large language models via uncertainty triangulationSemantic proximity Read

From Plausible Narrative to Sound Abduction: A Governed Abductive Architecture for Medical Digital Twins in Multiple Sclerosis CareSemantic proximity Read

An auditable, retrieval-grounded agent architecture improves medical accuracy and conversational safety in language modelsSemantic proximity Read

Progress and challenges in the symbiosis of AI with science and medicineSemantic proximity Read

fAIr science — How AI can support a transparent biopsychological research cycleSemantic proximity Read

fAIr science — How AI can support a transparent biopsychological research cycleSemantic proximity Read

Representation Before Retrieval: Structured Patient Artifacts Reduce Hallucination in Clinical AI SystemsSemantic proximity Read

Model shopping: how answer selection across AI models inflates scientific error ratesSemantic proximity Read

Evidence Synthesis of the Ethical and Legal Challenges of Generative Artificial Intelligence in the Provision of Patient Health InformationSemantic proximity Read

Rethinking Medical LLM Hallucinations: A System-Level SurveySemantic proximity Read

When Science Loses Its Immune System: Epistemic Immunodepression in the Age of AISemantic proximity Read