Article

The Challenge Dataset – simple evaluation for safe, transparent healthcare AI deployment

2022-12-16

Abstract excerpt

In this paper, we demonstrate the use of a “Challenge Dataset”: a small, site-specific, manually curated dataset – enriched with uncommon, risk-exposing, and clinically important edge cases – that can facilitate pre-deployment evaluation and identification of clinically relevant AI performance deficits. The five major steps of the Challenge Dataset process are described in detail, including defining use cases, edg...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 9e0b8fdb-412c-58ee-81f2-4a4e3357f0f4
DOI: 10.1101/2022.12.15.22280619

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

The Challenge Dataset – simple evaluation for safe, transparent healthcare AI deploymentDOI 10.1101/2022.12.15.22280619Read

The NLP-to-Expert Gap in Chest X-ray AISemantic proximity Read

Conformal Triage for Medical Imaging AI DeploymentSemantic proximity Read

Cethraian-Shift is a reproducible medical-imaging research project evaluating the cross-dataset reliability, calibration, label-policy sensitivity, and Grad-CAM++ stability of multi-label chest X-ray classifiers across NIH ChestX-ray14 and VinDr-CXRSemantic proximity Read

Developing Deep Learning Models for Classifying Medical Imaging DataSemantic proximity Read

Artificial Intelligence in Diagnostic Imaging: Enhancing Patient Care Through Advanced Algorithms and Data IntegrationSemantic proximity Read

MHub.ai: A Standardized Platform for Reproducible AI Research in Medical ImagingSemantic proximity Read

Beyond the Black Box: Avenues for Transparency in Regulating Radiological AI/ML-enabled SaMD via the FDA 510(k) PathwaySemantic proximity Read

The clinician-AI interface: intended use and explainability in FDA-cleared AI devices for medical image interpretationSemantic proximity Read

Diagnostic Accuracy of an Offline CNN Framework Utilizing Multi-View Chest X-Rays for Screening 14 Co-Occurring Communicable and Non-Communicable DiseasesSemantic proximity Read

AI-Assisted Pneumonia Detection, Localisation and Report Generation from Chest X-raysSemantic proximity Read

AI-generated data contamination erodes pathological variability and diagnostic reliabilitySemantic proximity Read

IHRAS: Automated Medical Report Generation from Chest X-rays via Classification, Segmentation, and LLMsSemantic proximity Read