Article

DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome

2020-09-19

Abstract excerpt

<h4>ABSTRACT</h4> Deciphering the language of non-coding DNA is one of the fundamental problems in genome research. Gene regulatory code is highly complex due to the existence of polysemy and distant semantic relationship, which previous informatics methods often fail to capture especially in data-scarce scenarios. To address this challenge, we developed a novel pre-trained bidirectional encoder representation, n...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 9e385009-6d73-5578-8ae6-c2a8a559c3fb
DOI: 10.1101/2020.09.17.301879

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genomeDOI 10.1101/2020.09.17.301879Read

Genomic Touchstone: Benchmarking Genomic Language Models in the Context of the Central DogmaSemantic proximity Read

The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human GenomicsSemantic proximity Read

Fine-Tuning Transformers For Genomic TasksSemantic proximity Read

NucEL: Single-Nucleotide ELECTRA-Style Genomic Pre-training for Efficient and Interpretable RepresentationsSemantic proximity Read

Findings from Sparse Autoencoders for DNA Sequence Models: Motif Detectors, Reading-Frame Features, and the Scarcity of Regulatory LogicSemantic proximity Read

The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human GenomicsSemantic proximity Read

Deciphering 3’ UTR mediated gene regulation using interpretable deep representation learningSemantic proximity Read

The human genome’s vocabulary as proposed by the DNA language model GROVERSemantic proximity Read

Benchmarking DNA Foundation Models for Genomic and Genetic TasksSemantic proximity Read

Distinguishing word identity and sequence context in DNA language modelsSemantic proximity Read

GENA-Web - GENomic Annotations Web Inference using DNA language modelsSemantic proximity Read

Benchmarking DNA Foundation Models for zero-shot variant effect prediction: the role of context, training, and architectureSemantic proximity Read