Article

Evaluating Open-Source LLMs for Automated Essay Scoring: The Critical Role of Prompt Design

2025-11-19

Abstract excerpt

This paper evaluates the Automated Essay Scoring (AES) performance of five open-source Large Language Models (LLMs)—LLaMA 3.2 3B, DeepSeek-R1 7B, Mistral 8×7B, Qwen2 7B, and Qwen2.5 7B—on the PERSUADE 2.0 dataset. We assess each model under three distinct prompting strategies: (1) rubric-aligned prompting, which embeds detailed, human-readable definitions of each scoring dimension; (2) instruction-based prompting,...

Topics

Open a Topic to create a Post that cites this publication.

Uncategorized literature

Identifiers and source

Literature Corpus work: 5c6e80bb-ddae-5d2f-9228-7d14f9f0668a
DOI: 10.20944/preprints202511.1429.v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Evaluating Open-Source LLMs for Automated Essay Scoring: The Critical Role of Prompt DesignDOI 10.20944/preprints202511.1429.v1Read

Explainable AI for Education: Enhancing Essay Scoring via Rubric-Aligned Chain-of-Thought PromptingSemantic proximity Read

MultiLLM – Self Reflect Iterative Prompt Methodology based Automated Essay Scoring SystemSemantic proximity Read

STL-LoRA: A Parameter-Efficient Single-Task Framework for Multi-Trait Arabic Automated Essay ScoringSemantic proximity Read

Can AI replace Teachers? Comparing AI and Human Teachers’ ratings and feedback on students’ essaysSemantic proximity Read

Passing the Turing Test: Fine-tuned AI feedback is less detectable than human or prompt-engineered feedbackSemantic proximity Read

DAFE: LLM-Based Evaluation Through Dynamic Arbitration for Free-Form Question-AnsweringSemantic proximity Read

Fine-Tuning PHI-3 for Multiple-Choice Question Answering: Methodology, Results, and ChallengesSemantic proximity Read

Large Language Model–Generated Tutoring Responses Receive Higher Ratings but Are Penalized by Source-Based Evaluation BiasSemantic proximity Read

Large Language Model–Generated Tutoring Responses Receive Higher Ratings but Are Penalized by Source-Based Evaluation BiasSemantic proximity Read

Can ChatGPT Replace the Teacher in Assessment? A Review of Research on the Use of Large Language Models in Grading and Providing FeedbackSemantic proximity Read

The AI Paradox in L2 Writing: Why Helpful Feedback Creates Unhelpful Dependency in Higher EducationSemantic proximity Read

Risks of Using Large Language Models in Grading: LLMs and Humans Prefer LLM-Generated Writing Over Human’s but LLMs Show a Stronger Systematic BiasSemantic proximity Read