Article

Attention-Head Binding as a Term-Conditioned Mechanistic Marker of Accessibility Concept Emergence in Language Models

2026-02-11

Abstract excerpt

Assessing when language models develop specific capabilities remains challenging, as behavioral evaluations are expensive and internal representations are opaque. We introduce attention-head binding (EB *), a lightweight mechanistic metric that tracks how attention heads bind multi-token technical terms, such as accessibility concepts ("screen reader," "alt text"), into coherent units during training. Using Pythia...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: f5468c83-8966-5a63-88a9-70969802ccba
DOI: 10.22541/au.177067444.42828315/v2

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Attention-Head Binding as a Term-Conditioned Mechanistic Marker of Accessibility Concept Emergence in Language ModelsDOI 10.22541/au.177067444.42828315/v2Read

Emergent Numeric Bias in Large Language Models: An Empirical Study on the Anomalous Recurrence of the Number 27 Across Independent SessionsSemantic proximity Read

Transformation of Large Language Models: A State of ArtSemantic proximity Read

SORT-AI: A Structural Safety and Reliability Framework for Advanced AI Systems with Retrieval-Augmented Generation as a Diagnostic TestbedSemantic proximity Read

Basics of Large Language Models - transformers to LLMsSemantic proximity Read

Inductive Algebraic Resonance Memory for Attention-Free Language ModellingSemantic proximity Read

Unraveling the Mysteries of AI ChatbotsSemantic proximity Read

Semantic Calibration of Contextual Embedding in Large Language Model Generative Pathways Using Contextual-Distillation MechanismsSemantic proximity Read

Constraint Poisoning and Parametric Memory Override: A Failure Taxonomy and Intervention Architecture for LLM-Assisted Philological Editing of Historical TextsSemantic proximity Read

Exploring Explainability in Large Language ModelsSemantic proximity Read

Explainable Automatic Evaluation: Interpretable Decision-Making in Large Language Model AutoratersSemantic proximity Read

Beyond Words: The Evolution of Large Language Models in Context-AwareSemantic proximity Read

BLM-Hum1800: A Human-Validated Benchmark for Linguistic Competence in English and FrenchSemantic proximity Read