Article

The human genome’s vocabulary as proposed by the DNA language model GROVER

2023-07-19

Abstract excerpt

Large Language Models (LLMs) on natural language have achieved a level of performance that allows the generation of coherent and syntactically correct text. DNA sequence of genomes follows rules similar to natural language, but a distinguishing factor is the absence of a concept analogous to words. We established byte-pair tokenization on the human genome and trained a foundation language model called GROVER (“ G...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 31b0b483-bea1-5e56-849b-4e8cbc3f4d16
DOI: 10.1101/2023.07.19.549677

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

The human genome’s vocabulary as proposed by the DNA language model GROVERDOI 10.1101/2023.07.19.549677Read

Distinguishing word identity and sequence context in DNA language modelsSemantic proximity Read

<i>genomicBERT</i> : A Light-weight Foundation Model for Genome Analysis using Unigram Tokenization and Specialized DNA VocabularySemantic proximity Read

NucEL: Single-Nucleotide ELECTRA-Style Genomic Pre-training for Efficient and Interpretable RepresentationsSemantic proximity Read

DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genomeSemantic proximity Read

Benchmarking DNA Foundation Models for Genomic and Genetic TasksSemantic proximity Read

Genomic Touchstone: Benchmarking Genomic Language Models in the Context of the Central DogmaSemantic proximity Read

Human Genome Book:Words,Sentences and ParagraphsSemantic proximity Read

Human Genome Book: Words, Sentences and ParagraphsSemantic proximity Read

Tokenization to Transfer: Do Genomic Foundation Models Learn Good Representations?Semantic proximity Read

Understanding Pathways in Bioinformatics, Genomics, and Health ApplicationsSemantic proximity Read

Genolator: A Multimodal Large Language Model Fusing Natural Language, Genomic, and Structural Tokens for Protein Function InterpretationSemantic proximity Read

BioToken and BioFM – Biologically-Informed Tokenization Enables Accurate and Efficient Genomic Foundation ModelsSemantic proximity Read