Article

Know When to Trust: Making AI Scoring More Reliable for Educational Assessment

2026-02-28

Abstract excerpt

<p>The rapid rise of large language models (LLMs) has created new opportunities for educational measurement. This paper introduces and evaluates three improvements to LLM-based automated scoring tools: model self-confidence, weighted probabilistic scoring, and ensemble modeling. The study utilizes data from over 20,000 responses to the Alternative Uses Task, most popular divergent thinking task, from more than 2,0...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 7ca2a76f-020a-5144-89d5-d1bba230abf7
DOI: 10.31234/osf.io/26qf3_v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Know When to Trust: Making AI Scoring More Reliable for Educational AssessmentDOI 10.31234/osf.io/26qf3_v1Read

Can ChatGPT Replace the Teacher in Assessment? A Review of Research on the Use of Large Language Models in Grading and Providing FeedbackSemantic proximity Read

ClassInsight: Developing an Agentic AIEd Tool for Responsive Teaching and LearningSemantic proximity Read

Can AI replace Teachers? Comparing AI and Human Teachers’ ratings and feedback on students’ essaysSemantic proximity Read

Are LLMs better expert judges? Rethinking content validity assessment in the age of AISemantic proximity Read

Are LLMs better expert judges? Rethinking content validity assessment in the age of AISemantic proximity Read

Leveraging Large Language Models for Assessments Grading and Personalised Feedback Generation: A Case of Software EngineeringSemantic proximity Read

Natural Language Processing and Generative AI in the Automated Scoring and Feedback of Reflective Writing in Medical Education: A Validity and Fairness AnalysisSemantic proximity Read

Field-testing multiple-choice questions with AI examinees: English grammar itemsSemantic proximity Read

Passing the Turing Test: Fine-tuned AI feedback is less detectable than human or prompt-engineered feedbackSemantic proximity Read

AI is Misled by GenAI: Stylistic Bias in Automated Assessment of Creativity in Large Language ModelsSemantic proximity Read

Using ChatGPT-4 to Check Assignments and Provide Feedback; A Mixed Methods StudySemantic proximity Read

Beyond Automated Scoring: An Evidence-Informed Human–GenAI Complementarity Framework for Formative Writing AssessmentSemantic proximity Read