Article

Inductive Algebraic Resonance Memory for Attention-Free Language Modelling

2026-07-16

Abstract excerpt

Self-attention couples every pair of positions in a sequence, incurring a cost that grows quadratically with se-quence length and a key–value state that grows without bound during generation. Here we introduce the inductive algebraicresonance memory (IARM), an attention-free sequence-mixing mechanism that replaces pairwise attention with two ingre-dients: a softmax-gated mixture of learned low-rank operators that...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: f7ad5d00-88a1-5cc2-80f0-88835cd109a5
DOI: 10.20944/preprints202607.1228.v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Inductive Algebraic Resonance Memory for Attention-Free Language ModellingDOI 10.20944/preprints202607.1228.v1Read

Engineering A Large Language Model From ScratchSemantic proximity Read

Beyond Words: The Evolution of Large Language Models in Context-AwareSemantic proximity Read

Semantic Calibration of Contextual Embedding in Large Language Model Generative Pathways Using Contextual-Distillation MechanismsSemantic proximity Read

Findings from Sparse Autoencoders for DNA Sequence Models: Motif Detectors, Reading-Frame Features, and the Scarcity of Regulatory LogicSemantic proximity Read

A Comparative Survey of Large Language Models: Foundation, Instruction-Tuned, and Multimodal VariantsSemantic proximity Read

Dynamic Token Clustering: A Novel Architectural Enhancement for Large Language ModelsSemantic proximity Read

Attention-Head Binding as a Term-Conditioned Mechanistic Marker of Accessibility Concept Emergence in Language ModelsSemantic proximity Read

Neural Re-Contextualization for Dynamic Semantic Control in Large Language ModelsSemantic proximity Read

RLHF-Aligned Open LLMs: A Comparative SurveySemantic proximity Read

Basics of Large Language Models - transformers to LLMsSemantic proximity Read

Upscaling A Smaller LLM to More Parameters via Manual Regressive DistillationSemantic proximity Read

Emergent Numeric Bias in Large Language Models: An Empirical Study on the Anomalous Recurrence of the Number 27 Across Independent SessionsSemantic proximity Read