Back to search

Article

BLiMP-NL: A corpus of Dutch minimal pairs and acceptability judgements for language model evaluation

2024-04-15

Abstract excerpt

<p>We present a corpus of 8400 Dutch sentence pairs, intended for the grammatical evaluation of language models. Each pair consists of a grammatical sentence and a minimally different ungrammatical sentence. The corpus covers 84 paradigms, classified into 22 syntactic phenomena. Nine sentences of each paradigm are rated for acceptability by at least 30 participants each, and for the same 9 sentences reading times...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work
9292e90e-c154-5dec-acbc-af2f3254ad40
DOI
10.31234/osf.io/mhjbx
Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest
Interactive article relationship graphSelect a related publication card to move it into the centre and load its closest explainable connections. Solid lines are source-backed structured connections. Dashed lines are semantic discovery signals and are not scientific evidence.
BLiMP-NL: A corpus of Dutch minimal pairs and acceptability judgements for language model evaluationDOI 10.31234/osf.io/mhjbx
BLiMP-NL: A corpus of Dutch minimal pairs and acceptability judgements for language model evaluationSemantic proximity
BLiMP-NL: A corpus of Dutch minimal pairs and acceptability judgements for language model evaluationSemantic proximity
BLM-Hum1800: A Human-Validated Benchmark for Linguistic Competence in English and FrenchSemantic proximity
Can Large Language Models Help Augment English Psycholinguistic Datasets?Semantic proximity
Enhancing Text Quality with Human-Machine Collaboration: A Refinement ApproachSemantic proximity
Neural Language Models’ sensitivity to long-distance dependencies and their island constraints is not universal: a comparison between English and Dutch humans and modelsSemantic proximity
Standardized Assessment of LLM English ProficiencySemantic proximity
DAFE: LLM-Based Evaluation Through Dynamic Arbitration for Free-Form Question-AnsweringSemantic proximity
An Extensive Analysis Between Different Language Models: GPT-3, BERT and MACAWSemantic proximity
When Transformer models are more compositional than humans: The case of the depth charge illusionSemantic proximity
A Discriminative Lexicon Approach to Word Comprehension, Production and Processing: Maltese PluralsSemantic proximity
AI-generated estimates of Dutch words and expressionsSemantic proximity
Select a neighboring publication to make it the new centre.