Article

Large Language Models and the Wisdom of Small Crowds

2023-12-04

Abstract excerpt

<p>Recent advances in Large Language Models (LLMs) have raised the question of replacing human subjects with LLM-generated data. While some believe that LLMs capture the “wisdom of the crowd”—due to their vast training data—empirical evidence for this hypothesis remains scarce. We present a novel methodological framework to test this: the “number needed to beat” (NNB), which measures how many humans are needed for...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 14dc9574-a438-5af7-94e2-4829c6afc2b9
DOI: 10.31234/osf.io/a48zq

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Large Language Models and the Wisdom of Small CrowdsDOI 10.31234/osf.io/a48zqRead

Which Humans?Semantic proximity Read

Can Large Language Models Help Augment English Psycholinguistic Datasets?Semantic proximity Read

Synthetic Participants Generated by Large Language Models: A Systematic Literature ReviewSemantic proximity Read

Perils and Opportunities in Using Large Language Models in Psychological ResearchSemantic proximity Read

LLMs in the Lab: Can AI Predict What Real Participants Do?Semantic proximity Read

Evaluating Large Language Models in Psychological Research: A Guide for ReviewersSemantic proximity Read

The Unreliable Judges: Assessing Reproducibility and Self-Preference Bias of LLMs as Free-Text EvaluatorsSemantic proximity Read

Large language models as linguistic simulators and cognitive models in human researchSemantic proximity Read

Are LLMs better expert judges? Rethinking content validity assessment in the age of AISemantic proximity Read

Are LLMs better expert judges? Rethinking content validity assessment in the age of AISemantic proximity Read

A Survey of Large Language Models: Evolution, Architectures, Adaptation, Benchmarking, Applications, Challenges, and Societal ImplicationsSemantic proximity Read

LLMs and AI: Understanding Its Reach and ImpactSemantic proximity Read