Article

Benchmarking Large Language Model Rationality Using Measurement Axioms

2026-07-01

Abstract excerpt

<title>Abstract</title> <p>While Large language models (LLMs) are increasingly deployed as decision-makers, current evaluation practice emphasizes benchmark accuracy rather than the structural properties that make decisions coherent and interpretable. We introduce a measurement-theoretic framework for evaluating AI decision-making using formal axioms derived from utility theory, focusing on transitivity of prefer...

Topics

Open a Topic to create a Post that cites this publication.

Uncategorized literature

Identifiers and source

Literature Corpus work: 952df75c-fc87-5564-a8e4-9af4103f79f9
DOI: 10.21203/rs.3.rs-10119008/v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Benchmarking Large Language Model Rationality Using Measurement AxiomsDOI 10.21203/rs.3.rs-10119008/v1Read

Are LLMs better expert judges? Rethinking content validity assessment in the age of AISemantic proximity Read

Are LLMs better expert judges? Rethinking content validity assessment in the age of AISemantic proximity Read

Model shopping: how answer selection across AI models inflates scientific error ratesSemantic proximity Read

Nonlinear Transformation of Probabilities by Large Language ModelsSemantic proximity Read

Worth the Weight: Modern LLMs Demonstrate Accurate Metacognitive Knowledge of Decision Weights in Multi-Attribute ChoiceSemantic proximity Read

Artificial Intelligence and Metacognitive Knowledge: LLMs Can Report Decision Weights as Faithfully as Humans in Multi-Attribute ChoiceSemantic proximity Read

Studying and improving reasoning in humans and machinesSemantic proximity Read

The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-MakingSemantic proximity Read

Artificial intelligence in psychology researchSemantic proximity Read

Machine Psychometrics for Artificial AgentsSemantic proximity Read

A Moral Turing Test to assess How Subjective Belief and Objective Source Affect Detection and Agreement with LLM JudgmentsSemantic proximity Read

Epistemological Fault Lines Between Human and Artificial IntelligenceSemantic proximity Read