Article

Engineering A Large Language Model From Scratch

2024-01-31

Abstract excerpt

The proliferation of deep learning in natural language processing (NLP) has led to the development and release of innovative technologies capable of understanding and generating human language with remarkable proficiency. Atinuke, a Transformer-based neural network, optimises performance across various language tasks by utilising a unique configuration. The architecture interweaves layers for processing sequential...

Topics

Open a Topic to create a Post that cites this publication.

Uncategorized literature

Identifiers and source

Literature Corpus work: 40bed039-0da4-5f9a-9c10-88133cfa5baa
DOI: 10.20944/preprints202401.2120.v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Engineering A Large Language Model From ScratchDOI 10.20944/preprints202401.2120.v1Read

Unified Efficient Fine-Tuning Techniques for Open-Source Large Language ModelsSemantic proximity Read

A Comparative Survey of Large Language Models: Foundation, Instruction-Tuned, and Multimodal VariantsSemantic proximity Read

SuperGLUE: The AI raceSemantic proximity Read

HyperLLM: The Next Generation of Large Language Models with Multimodal CapabilitiesSemantic proximity Read

Inductive Algebraic Resonance Memory for Attention-Free Language ModellingSemantic proximity Read

Bridging Finance and AI: A Comprehensive Survey of Large Language Models in Financial ApplicationsSemantic proximity Read

Upscaling A Smaller LLM to More Parameters via Manual Regressive DistillationSemantic proximity Read

Enhancements to Large Language Models: Introducing Dynamic Syntactic Insertion for Improved Model Robustness and GeneralizationSemantic proximity Read

Dynamic Token Clustering: A Novel Architectural Enhancement for Large Language ModelsSemantic proximity Read

Optimizing Large Language Model Scaling with Micro Batch Pipeline and Inference ParallelismSemantic proximity Read

Me-LLaMA: Foundation Large Language Models for Medical ApplicationsSemantic proximity Read

A Survey of Large Language Models: Evolution, Architectures, Adaptation, Benchmarking, Applications, Challenges, and Societal ImplicationsSemantic proximity Read