Article

A Multi-Model, Multi-Domain Benchmark of Large Language Model Agreement with Humans and with Each Other in Sentiment Classification

2026-08-18

Abstract excerpt

<title>Abstract</title> <p>Large language models (LLMs) are widely used to classify sentiment, yet cross-provider comparisons on the same data are rare and inter-model agreement is largely unmeasured. We evaluate five LLMs from four providers (Claude Opus 4.7, GPT-4o, GPT-5.5, Llama 3.1 8B, and Gemini 2.5 Pro), with VADER (a rule-based lexicon) and RoBERTa (a supervised transformer) as a baseline, on five human-a...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 04c54625-49a3-526e-8cbc-a7c400b551a6
DOI: 10.21203/rs.3.rs-10415268/v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

A Multi-Model, Multi-Domain Benchmark of Large Language Model Agreement with Humans and with Each Other in Sentiment ClassificationDOI 10.21203/rs.3.rs-10415268/v1Read

How Well Do Large Language Models Agree with Humans — and Each Other? A Multi-Model, Multi-Domain Sentiment Benchmark StudySemantic proximity Read

Validity of Large Language Models for Sentiment Analysis: Evidence of performance comparable to human codersSemantic proximity Read

Bridging Finance and AI: A Comprehensive Survey of Large Language Models in Financial ApplicationsSemantic proximity Read

Enhancing Multi-label Emotion Prediction through Rule-based Voting with LLM and BERT VariantsSemantic proximity Read

Upscaling A Smaller LLM to More Parameters via Manual Regressive DistillationSemantic proximity Read

On the Development of ToxicBias-Reasoning for Responsible Multicultural Bias Detection and ExplanationSemantic proximity Read

Benchmarking of Commercial Large Language Models: ChatGPT, Mistral, and LlamaSemantic proximity Read

Using Large Language Models for sentiment analysis of health-related social media data: empirical evaluation and practical tipsSemantic proximity Read

Advancing Sentiment Analysis in Gujarati: Performance Enhancement through a Hybrid Annotation FrameworkSemantic proximity Read

Open-source solution for evaluation and benchmarking of large language models for public healthSemantic proximity Read

Boosting Domain Knowledge Understanding of LLMs through Fine-Tuning with a Novel KNN AlgorithmSemantic proximity Read

Negative Preference Reduction in Large Language Model Unlearning: An Experimental ApproachSemantic proximity Read