Article

Paying Attention to Attention: High Attention Sites as Indicators of Protein Family and Function in Language Models

2024-12-17

Abstract excerpt

Protein Language Models (PLMs) use transformer architectures to capture patterns within protein sequences, providing a powerful computational representation of the protein sequence [1]. Through large-scale training on protein sequence data, PLMs generate vector representations that encapsulate the biochemical and structural properties of proteins [2]. At the core of PLMs is the attention mechanism, which facilitat...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 62e2044e-ceaa-5d97-a9c0-40849bf6700e
DOI: 10.1101/2024.12.13.628435

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Paying Attention to Attention: High Attention Sites as Indicators of Protein Family and Function in Language ModelsDOI 10.1101/2024.12.13.628435Read

Interpreting Protein Language Models: high attention sites predict functional regionsSemantic proximity Read

InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse AutoencodersSemantic proximity Read

Pool PaRTI: A PageRank-Based Pooling Method for Identifying Critical Residues and Enhancing Protein Sequence RepresentationsSemantic proximity Read

Fast and accurate protein function prediction from sequence through pretrained language model and homology-based label diffusionSemantic proximity Read

Applying Protein Language Models Using Limited Dataset. Sequence-Based Hot Spot Prediction in Protein Interactions Using AutoGluonSemantic proximity Read

Mixture of Experts Enable Efficient and Effective Protein Understanding and DesignSemantic proximity Read

Light Attention Predicts Protein Location from the Language of LifeSemantic proximity Read

Post-Translational Modification Prediction via Prompt-Based Fine-Tuning of a GPT-2 ModelSemantic proximity Read

ProtSyntax: a protein large language model for decoding post-translational modification syntax and functionSemantic proximity Read

ProtAttn-QuadNet: An attention-based deep learning framework for protein–protein interaction prediction using ProtBERT embeddingsSemantic proximity Read

P 3: A Framework for Predicting Protein-Protein Interactions Using Large Language ModelsSemantic proximity Read

ProtSyntax: a protein large language model for decoding post-translational modification syntax and functionSemantic proximity Read