Article

Task-Conditional Accuracy–Support Trade-offs of Sparse Attention Normalizers in Compact Classifiers

2026-08-18

Abstract excerpt

<title>Abstract</title> <p>Purpose: Sparse alternatives to softmax can reduce attention support, but their value in compact classifiers may depend on the task and implementation. This study tests whether fixed and adaptive sparse normalizers improve predictive quality under a shared compact-classifier protocol. <h4>Methods:</h4> Dense softmax, sparsemax, entmax-1.5, three fixed-ratio top-k softmax settings, and h...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: aa953712-375f-5ad8-8992-965d1f393458
DOI: 10.21203/rs.3.rs-10720877/v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Task-Conditional Accuracy–Support Trade-offs of Sparse Attention Normalizers in Compact ClassifiersDOI 10.21203/rs.3.rs-10720877/v1Read

Symbiotic Convolution Block (SCB): An Implicit Feature Recalibration without Attention-Mechanism for Efficient CNNs in Image ClassificationSemantic proximity Read

Backpropagation-Free Continual Learning: A Unified Energy-Based Framework, Reproducible Benchmark, and Open ChallengesSemantic proximity Read

Parameter Efficient Fine-tuning of Transformer-based Masked Autoencoder Enhances Resource Constrained Neuroimage AnalysisSemantic proximity Read

BOPA-Fast: Bipolar Outer Product Attention for Efficient and Interpretable Multimodal FusionSemantic proximity Read

Scaffolded representation learning in deep networksSemantic proximity Read

Parameter-efficient Prompt Tuning of VisionFoundation Model With Adaptive Focal Loss forInterpretable MCI ScreeningSemantic proximity Read

What Makes Neural Networks Trainable? Invexity as a Structural Design Principle in AISemantic proximity Read

Scalable and Interpretable Mixture of Experts Models in Machine Learning: Foundations, Applications, and ChallengesSemantic proximity Read

Probing machine-learning classifiers using noise, bubbles, and reverse correlationSemantic proximity Read

A survey on attention mechanisms for medical applications: are we moving towards better algorithms?Semantic proximity Read

Memory Consolidation with Orthogonal Gradients for avoiding Catastrophic ForgettingSemantic proximity Read

KAN-EEG: Towards Replacing Backbone-MLP for an Effective Seizure Detection SystemSemantic proximity Read