Article

How Well Do Large Language Models Agree with Humans — and Each Other? A Multi-Model, Multi-Domain Sentiment Benchmark Study

2026-06-23

Abstract excerpt

<title>Abstract</title> <p>Large language models (LLMs) are now widely used to classify sentiment. Yet we rarely compare models from different providers on the same data, and we know little about how often they agree with each other. We evaluate five LLMs from four providers (Claude Opus 4.7, GPT-4o, GPT-5.5, Llama 3.1 8B, and Gemini 2.5 Pro) and add VADER, a rule based lexicon, as a baseline. The test set has fi...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 51d74941-aabb-5521-8c4f-a44a7fef32eb
DOI: 10.21203/rs.3.rs-9960161/v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

How Well Do Large Language Models Agree with Humans — and Each Other? A Multi-Model, Multi-Domain Sentiment Benchmark StudyDOI 10.21203/rs.3.rs-9960161/v1Read

A Multi-Model, Multi-Domain Benchmark of Large Language Model Agreement with Humans and with Each Other in Sentiment ClassificationSemantic proximity Read

Validity of Large Language Models for Sentiment Analysis: Evidence of performance comparable to human codersSemantic proximity Read

Bridging Finance and AI: A Comprehensive Survey of Large Language Models in Financial ApplicationsSemantic proximity Read

Enhancing Multi-label Emotion Prediction through Rule-based Voting with LLM and BERT VariantsSemantic proximity Read

Upscaling A Smaller LLM to More Parameters via Manual Regressive DistillationSemantic proximity Read

Do Large Language Models see the world with rose-colored glasses? Prompting ChatGPT-4o to associate colors with emotions in Spanish and MandarinSemantic proximity Read

Advancing Sentiment Analysis in Gujarati: Performance Enhancement through a Hybrid Annotation FrameworkSemantic proximity Read

Using Large Language Models for sentiment analysis of health-related social media data: empirical evaluation and practical tipsSemantic proximity Read

Contrasting Linguistic Patterns in Human and LLM-Generated News TextSemantic proximity Read

Benchmarking of Commercial Large Language Models: ChatGPT, Mistral, and LlamaSemantic proximity Read

Reliability of LLM-based Citation Sentiment Analysis under Structural Pressure: A Holistic Pressure Index Approach for Scientometric InferenceSemantic proximity Read

On the Development of ToxicBias-Reasoning for Responsible Multicultural Bias Detection and ExplanationSemantic proximity Read