Article

Success and Failure of Compositional Generalization in Distributional Models of Language

2024-10-07

Abstract excerpt

<p>Are distributional learning mechanisms capable of complex linguistic inferencesrequiring compositional generalization? This question has become contentious withthe development of large language models, which mimic human language abilities inmany ways, but which struggle with compositional generalization. We investigateda set of qualitatively different distributional models (word co-occurrence models,graphical m...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 5149c222-03c8-595b-a0b2-a54e9f783983
DOI: 10.31234/osf.io/ps84q

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Success and Failure of Compositional Generalization in Distributional Models of LanguageDOI 10.31234/osf.io/ps84qRead

Exploring Textual Semantics in Large Language ModelsSemantic proximity Read

Structural Generalization of Modification in Adult Learners of an Artificial LanguageSemantic proximity Read

Core vocabulary reveals differences between human word prediction and large language modelsSemantic proximity Read

The roles of neural networks in language acquisitionSemantic proximity Read

Emergent Numeric Bias in Large Language Models: An Empirical Study on the Anomalous Recurrence of the Number 27 Across Independent SessionsSemantic proximity Read

Divergent Semantic Integration (DSI): Extracting Creativity from Narratives with Distributional Semantic ModelingSemantic proximity Read

Semantic Calibration of Contextual Embedding in Large Language Model Generative Pathways Using Contextual-Distillation MechanismsSemantic proximity Read

Core vocabulary reveals differences between human word prediction and large language modelsSemantic proximity Read

Inductive Algebraic Resonance Memory for Attention-Free Language ModellingSemantic proximity Read

When Transformer models are more compositional than humans: The case of the depth charge illusionSemantic proximity Read

Artificial Language LearningSemantic proximity Read

Engineering A Large Language Model From ScratchSemantic proximity Read