Article

Explainable Automatic Evaluation: Interpretable Decision-Making in Large Language Model Autoraters

2026-05-05

Abstract excerpt

The deployment of Large Language Models (LLMs) as automatic evaluators—termed "LLM autoraters"—has emerged as a scalable alternative to human judgment for assessing natural language generation quality. However, the opacity of LLM decision-making processes fundamentally undermines trust, debuggability, and regulatory compliance in high-stakes evaluation contexts. This study introduces a framework for explainable au...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: d7dce200-0f38-5025-a42f-37034b358b76
DOI: 10.14293/pr2199.003509.v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Explainable Automatic Evaluation: Interpretable Decision-Making in Large Language Model AutoratersDOI 10.14293/pr2199.003509.v1Read

Exploring Explainability in Large Language ModelsSemantic proximity Read

Explainable AI for Education: Enhancing Essay Scoring via Rubric-Aligned Chain-of-Thought PromptingSemantic proximity Read

Context-Grounded Factuality Enhancement in LLM Responses via Multi-Stage Critique and RefinementSemantic proximity Read

Post Hoc Explanations of Language Models Can Improve Language ModelsSemantic proximity Read

Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models: A Case Study on ChatGPTSemantic proximity Read

From Illusion to Insight: A Taxonomic Survey of Hallucination Mitigation Techniques in LLMsSemantic proximity Read

From Linguistic Rights to Explainable Learning Analytics: A Cross-jurisdictional Policy Framework for Generative AI Feedback in Professional Interpreter EducationSemantic proximity Read

DAFE: LLM-Based Evaluation Through Dynamic Arbitration for Free-Form Question-AnsweringSemantic proximity Read

The Unreliable Judges: Assessing Reproducibility and Self-Preference Bias of LLMs as Free-Text EvaluatorsSemantic proximity Read

RLHF-Aligned Open LLMs: A Comparative SurveySemantic proximity Read

Limits of Self-Correction in LLMs: An Information-Theoretic Analysis of Correlated ErrorsSemantic proximity Read

Addressee Framing Systematically Reduces LLM Structural Pathology: A Controlled Empirical Study of Directed Information FlowSemantic proximity Read