Article

Statistical Order in Representation Learning: Sufficiency, Architectural Blindness, and Generalization Bounds via Maximum Entropy

2026-04-01

Abstract excerpt

<title>Abstract</title> <p>We establish three formal results about the role of statistical order in representation learning. (i) Sufficiency: under a maximum-entropy (Gibbs) data model, a representation that preserves all K-order multipoint statistics is a sufficient statistic for the model’s natural parameters, and is equivalent to saturating the mutual information I(Z; Φ K (X)) = H(Φ K (X)). (ii) Architectural...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 69162583-4417-5b3b-b161-79db5650e493
DOI: 10.21203/rs.3.rs-9273811/v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Statistical Order in Representation Learning: Sufficiency, Architectural Blindness, and Generalization Bounds via Maximum EntropyDOI 10.21203/rs.3.rs-9273811/v1Read

Shared Representations of Stability in Humans, Supervised & Unsupervised Deep Neural NetworksSemantic proximity Read

The representational geometry of out‐of‐distribution generalization in primary visual cortex and artificial neural networksSemantic proximity Read

High-performing neural network models of visual cortex benefit from high latent dimensionalitySemantic proximity Read

Scaffolded representation learning in deep networksSemantic proximity Read

Too Symmetric to See: Diagnosing Over-Invariance in Finite-Symmetry Geometric LearningSemantic proximity Read

Understanding transformation tolerant visual object representations in the human brain and convolutional neural networksSemantic proximity Read

Modeling second-order boundary perception: A machine learning approachSemantic proximity Read

AI-discovered tuning laws explain neuronal population code geometrySemantic proximity Read

What Makes Neural Networks Trainable? Invexity as a Structural Design Principle in AISemantic proximity Read

The Combinatorial Capacity and Robustness of Hierarchical Concept Coding in the Human Medial Temporal LobeSemantic proximity Read

Sparse coding models predict a spectral bias in the development of primary visual cortex (V1) receptive fieldsSemantic proximity Read

Why Compression Creates Intelligence: The Architecture of Experience in Large ModelsSemantic proximity Read