Back to search

Article

BLM-Hum1800: A Human-Validated Benchmark for Linguistic Competence in English and French

2026-08-21

Abstract excerpt

<p>Evaluating the fine-grained linguistic competence of Large Language Models (LLMs) requires diagnosticbenchmarks anchored by controlled syntactic paradigms and human behavioral baselines. Here, wepresent a bilingual dataset comprising 1,800 human-validated linguistic problem instances across Englishand French. Based on the Blackbird Language Matrices (BLM) framework, the dataset targets three coregrammatical phe...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work
f7612927-cbef-5ad3-a3f0-4675a0f6e745
DOI
10.31234/osf.io/r3egu_v1
Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest
Interactive article relationship graphSelect a related publication card to move it into the centre and load its closest explainable connections. Solid lines are source-backed structured connections. Dashed lines are semantic discovery signals and are not scientific evidence.
BLM-Hum1800: A Human-Validated Benchmark for Linguistic Competence in English and FrenchDOI 10.31234/osf.io/r3egu_v1
Can Large Language Models Help Augment English Psycholinguistic Datasets?Semantic proximity
BLiMP-NL: A corpus of Dutch minimal pairs and acceptability judgements for language model evaluationSemantic proximity
BLiMP-NL: A corpus of Dutch minimal pairs and acceptability judgements for language model evaluationSemantic proximity
BLiMP-NL: A corpus of Dutch minimal pairs and acceptability judgements for language model evaluationSemantic proximity
Standardized Assessment of LLM English ProficiencySemantic proximity
A Survey of Large Language Models: Evolution, Architectures, Adaptation, Benchmarking, Applications, Challenges, and Societal ImplicationsSemantic proximity
SuperGLUE: The AI raceSemantic proximity
Large Language Models and the Wisdom of Small CrowdsSemantic proximity
Transformation of Large Language Models: A State of ArtSemantic proximity
Semantic Contextual Embedding for Domain-Specific Task Transfer in Large Language ModelsSemantic proximity
StimMAS: A Multi-Agent Framework for Automated Linguistic Stimulus Construction in Psychological ResearchSemantic proximity
StimMAS: A Multi-Agent Framework for Automated Linguistic Stimulus Construction in Psychological ResearchSemantic proximity
Select a neighboring publication to make it the new centre.