Article

<i>genomicBERT</i> : A Light-weight Foundation Model for Genome Analysis using Unigram Tokenization and Specialized DNA Vocabulary

2023-06-01

Abstract excerpt

The genome, which serves as the inherent language directing the blueprint of life, offers significant analysis prospects by combining Natural Language Processing (NLP) and machine learning (ML). Integrating biological sequences with other digital healthcare information has potential to transform data-driven diagnostics. Large language models (LLMs) can be harnessed to decode the genomic language. This endeavor enc...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 43a30cd0-4b9b-5f3b-a04c-db86b261fbd4
DOI: 10.1101/2023.05.31.542682

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

<i>genomicBERT</i> : A Light-weight Foundation Model for Genome Analysis using Unigram Tokenization and Specialized DNA VocabularyDOI 10.1101/2023.05.31.542682Read

The human genome’s vocabulary as proposed by the DNA language model GROVERSemantic proximity Read

Distinguishing word identity and sequence context in DNA language modelsSemantic proximity Read

Genomic Touchstone: Benchmarking Genomic Language Models in the Context of the Central DogmaSemantic proximity Read

Genomic Language Models (gLMs) decode bacterial genomes for improved gene prediction and translation initiation site identificationSemantic proximity Read

NucEL: Single-Nucleotide ELECTRA-Style Genomic Pre-training for Efficient and Interpretable RepresentationsSemantic proximity Read

Understanding Pathways in Bioinformatics, Genomics, and Health ApplicationsSemantic proximity Read

BioToken and BioFM – Biologically-Informed Tokenization Enables Accurate and Efficient Genomic Foundation ModelsSemantic proximity Read

ProkBERT Family: Genomic Language Models for Microbiome ApplicationsSemantic proximity Read

DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genomeSemantic proximity Read

LncRNA-BERT: An RNA Language Model for Classifying Coding and Long Non-Coding RNASemantic proximity Read

Benchmarking DNA Foundation Models for Genomic and Genetic TasksSemantic proximity Read

Tokenization to Transfer: Do Genomic Foundation Models Learn Good Representations?Semantic proximity Read