Article

Can AI Match Human Experts? Evaluating LLM-Generated Feedback on Resident Scholarly Projects

2026-03-04

Abstract excerpt

<h4>Background</h4> Delivering timely, high-quality feedback on resident scholarly projects is labour-intensive, especially in large programmes. We developed an AI-assisted evaluation system, powered by the open-weight LLaMA-3.1 large-language model (LLM), to generate formative feedback on Family Medicine residents’ scholarly projects and compared its performance with expert human evaluators. <h4>Methods</h4> We...

Topics

Open a Topic to create a Post that cites this publication.

Uncategorized literature

Identifiers and source

Literature Corpus work: 6f5887e3-81a1-5973-9735-471ec9637511
DOI: 10.64898/2026.03.04.26346878

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Can AI Match Human Experts? Evaluating LLM-Generated Feedback on Resident Scholarly ProjectsDOI 10.64898/2026.03.04.26346878Read

Natural Language Processing and Generative AI in the Automated Scoring and Feedback of Reflective Writing in Medical Education: A Validity and Fairness AnalysisSemantic proximity Read

Augmenting thesis supervision with generative AI feedback: Evaluating the quality and perceived usefulness GenAI feedbackSemantic proximity Read

From Output Errors to Workflow Harm: A Practitioner-Audit Method for LLM-Mediated ResearchSemantic proximity Read

Evaluating Generative AI as an Educational Tool for Radiology Resident Report DraftingSemantic proximity Read

A Performance-Based Rubric for Generative AI Use in Medical Students’ Research Tasks: Development and Initial Psychometric EvaluationSemantic proximity Read

What Do Clinicians Edit in Ambient AI-Drafted Clinical Documentation? A Qualitative Content AnalysisSemantic proximity Read

Clinicians’ Rationale for Editing Ambient AI–Drafted Clinical Notes: Persistent Challenges and Implications for ImprovementSemantic proximity Read

Physician Evaluations of Large Language Model-Generated Responses to Medical Questions by Region and Years in Practice: A preliminary studySemantic proximity Read

Quality, consistency, and clinical safety of AI-generated versus clinician-written clinical notes: a multi-country paired simulation studySemantic proximity Read

The use of a large language models to create plain language summaries of evidence reviews in healthcare: a feasibility studySemantic proximity Read

Using ChatGPT-4 to Check Assignments and Provide Feedback; A Mixed Methods StudySemantic proximity Read

Can ChatGPT write better scientific titles? A comparative evaluation of human-written and AI-generated titlesSemantic proximity Read