Article

InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders

2024-11-15

Abstract excerpt

<h4> A bstract </h4> Protein language models (PLMs) have demonstrated remarkable success in protein modeling and design, yet their internal mechanisms for predicting structure and function remain poorly understood. Here we present a systematic approach to extract and analyze interpretable features from PLMs using sparse autoencoders (SAEs). By training SAEs on embeddings from the PLM ESM-2, we identify thousand...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: acdc1333-a5f7-5147-a344-243a7ca88267
DOI: 10.1101/2024.11.14.623630

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse AutoencodersDOI 10.1101/2024.11.14.623630Read

Paying Attention to Attention: High Attention Sites as Indicators of Protein Family and Function in Language ModelsSemantic proximity Read

Mixture of Experts Enable Efficient and Effective Protein Understanding and DesignSemantic proximity Read

Interpreting Protein Language Models: high attention sites predict functional regionsSemantic proximity Read

Improving Variant Effect Prediction by Steering Sparse Mechanistic Features in Protein Language ModelsSemantic proximity Read

Contrasting Sequence with Structure: Pre-training Graph Representations with PLMsSemantic proximity Read

Findings from Sparse Autoencoders for DNA Sequence Models: Motif Detectors, Reading-Frame Features, and the Scarcity of Regulatory LogicSemantic proximity Read

Multi-Modal Large Language Model Enables Protein Function PredictionSemantic proximity Read

BioReason-Pro: Advancing Protein Function Prediction with Multimodal Biological ReasoningSemantic proximity Read

Fine-Tuning Protein Language Models Unlocks the Potential of Underrepresented Viral ProteomesSemantic proximity Read

Light Attention Predicts Protein Location from the Language of LifeSemantic proximity Read

P 3: A Framework for Predicting Protein-Protein Interactions Using Large Language ModelsSemantic proximity Read

Multi-Modal Large Language Model Enables Protein Function PredictionSemantic proximity Read