Article

Findings from Sparse Autoencoders for DNA Sequence Models: Motif Detectors, Reading-Frame Features, and the Scarcity of Regulatory Logic

2026-07-22

Abstract excerpt

<title>Abstract</title> <p>Sparse autoencoders (SAEs) have become the standard tool for decomposing the internal activations of language models into human-interpretable features, but their behaviour on DNA sequence models— transformers and long-convolution models pretrained on genomes—remains largely uncharted. Rather than train an exhaustive suite, we present a deliberately narrow study: we fit top-𝐾 SAEs to th...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: b174326f-692d-56ec-8c52-5a03ab1e09e7
DOI: 10.21203/rs.3.rs-10434163/v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Findings from Sparse Autoencoders for DNA Sequence Models: Motif Detectors, Reading-Frame Features, and the Scarcity of Regulatory LogicDOI 10.21203/rs.3.rs-10434163/v1Read

NucEL: Single-Nucleotide ELECTRA-Style Genomic Pre-training for Efficient and Interpretable RepresentationsSemantic proximity Read

MINA: linear probes reveal coding-sequence family signal in frozen DNA encoders up to protein compositionSemantic proximity Read

DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genomeSemantic proximity Read

Genomic Touchstone: Benchmarking Genomic Language Models in the Context of the Central DogmaSemantic proximity Read

Regulatory Logic in Larger Single-Cell Foundation Models: A Sparse Autoencoder Comparison of scFoundation and GeneformerSemantic proximity Read

Fine-Tuning Transformers For Genomic TasksSemantic proximity Read

Decoding Gene Enhancers with DNA Language Models: A Survey of Methods, Limitations, and ProspectsSemantic proximity Read

TabVI: Leveraging Lightweight Transformer Architectures to Learn Biologically Meaningful Cellular RepresentationsSemantic proximity Read

CellFM: a large-scale foundation model pre-trained on transcriptomics of 100 million human cellsSemantic proximity Read

Sparse autoencoders reveal organized biological knowledge but minimal regulatory logic in single-cell foundation models: a comparative atlas of Geneformer and scGPTSemantic proximity Read

Uncovering the Mechanistic Landscape of Regulatory DNA with Deep LearningSemantic proximity Read