Article

Fine-Tuning Transformers For Genomic Tasks

2022-02-10

Abstract excerpt

Transformers are a type of neural network architecture that has been successfully used to achieve state-of-the-art performance in numerous natural language processing tasks. However, what about DNA, the language life written in the four-letter alphabet? In this paper, we review the current state of Transformers usage in genomics and molecular biology in general, introduce a collection of benchmark datasets for the...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 241c45bb-132b-52f6-943f-4e3cff22a286
DOI: 10.1101/2022.02.07.479412

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Fine-Tuning Transformers For Genomic TasksDOI 10.1101/2022.02.07.479412Read

NucEL: Single-Nucleotide ELECTRA-Style Genomic Pre-training for Efficient and Interpretable RepresentationsSemantic proximity Read

DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genomeSemantic proximity Read

Genomic Touchstone: Benchmarking Genomic Language Models in the Context of the Central DogmaSemantic proximity Read

Benchmarking DNA Foundation Models for Genomic and Genetic TasksSemantic proximity Read

Findings from Sparse Autoencoders for DNA Sequence Models: Motif Detectors, Reading-Frame Features, and the Scarcity of Regulatory LogicSemantic proximity Read

Annotating the genome at single-nucleotide resolution with DNA foundation modelsSemantic proximity Read

Distinguishing word identity and sequence context in DNA language modelsSemantic proximity Read

LAMBDA: A Prophage Detection Benchmark for Genomic Language ModelsSemantic proximity Read

BEACON: Benchmark for Comprehensive RNA Tasks and Language ModelsSemantic proximity Read

The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human GenomicsSemantic proximity Read

The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human GenomicsSemantic proximity Read

MINA: linear probes reveal coding-sequence family signal in frozen DNA encoders up to protein compositionSemantic proximity Read