Article

Context-Aware LLM Evaluators for Content Moderation Judgments in a Low-Resource Setting

2026-08-26

Abstract excerpt

<title>Abstract</title> <p>Large Language Models (LLMs) are increasingly used as automated evaluators in domains where expert annotation is scarce and expensive. Moderation of German-language newspaper forums is such a domain: reference annotations require professional moderators and reach only moderate inter-annotator agreement.We ask how reliably LLM evaluators reproduce these expert judgments, and how far that...

Topics

Open a Topic to create a Post that cites this publication.

Uncategorized literature

Identifiers and source

Literature Corpus work: 2d783e26-7fbc-55ec-84a4-4c2da29da3b6
DOI: 10.21203/rs.3.rs-10576121/v1

Open publication

Related research

Semantic proximity does not establish scientific evidence. Graph snapshot: 2026-09-29.

Click a neighbor to travelStep 1 · 12 closest

Context-Aware LLM Evaluators for Content Moderation Judgments in a Low-Resource SettingDOI 10.21203/rs.3.rs-10576121/v1Read

Explainable Automatic Evaluation: Interpretable Decision-Making in Large Language Model AutoratersSemantic proximity Read

Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models: A Case Study on ChatGPTSemantic proximity Read

Anchoring Bias in Large Language Models: An Experimental StudySemantic proximity Read

Using LLMs for Pre-Annotation of Emotional Manipulation Techniques in a Low-Resource Language Corpus: Are We There Yet?Semantic proximity Read

Limits of Self-Correction in LLMs: An Information-Theoretic Analysis of Correlated ErrorsSemantic proximity Read

The Unreliable Judges: Assessing Reproducibility and Self-Preference Bias of LLMs as Free-Text EvaluatorsSemantic proximity Read

Addressee Framing Systematically Reduces LLM Structural Pathology: A Controlled Empirical Study of Directed Information FlowSemantic proximity Read

Mapping how LLMs debate societal issues when shadowing human personality traits, sociodemographics and social media behaviorSemantic proximity Read

Foresight Is Not Enough: Sentence-Level Future Signals, Self-Loop Hard Negatives, and the Calibration Gap in Small Language ModelsSemantic proximity Read

Benchmarking the Personalization Capabilities of Large Language ModelsSemantic proximity Read

Benchmarking the Personalization Capabilities of Large Language ModelsSemantic proximity Read

Post Hoc Explanations of Language Models Can Improve Language ModelsSemantic proximity Read