Article

Unified Efficient Fine-Tuning Techniques for Open-Source Large Language Models

2024-07-02

Abstract excerpt

<title>Abstract</title> <p>Natural language processing has been revolutionized through the development of powerful large language models capable of understanding and generating human-like text, yet fine-tuning these models for specific tasks remains a computationally demanding process. Our novel approach to efficient fine-tuning, utilizing techniques such as adapter layers, Low-Rank Adaptation (LoRA), and layer-w...

Topics

Open a Topic to create a Post that cites this publication.

Uncategorized literature

Identifiers and source

Literature Corpus work: 212c0656-73ba-5adf-85f5-a896db34b696
DOI: 10.21203/rs.3.rs-4660140/v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Unified Efficient Fine-Tuning Techniques for Open-Source Large Language ModelsDOI 10.21203/rs.3.rs-4660140/v1Read

Layer-wise Adaptive Sparse Fine-Tuning: Boosting LLM Efficiency through Dynamic Ranks and SparsitySemantic proximity Read

Parameter-Efficient Fine-Tuning in Large Models: A Survey of MethodologiesSemantic proximity Read

Upscaling A Smaller LLM to More Parameters via Manual Regressive DistillationSemantic proximity Read

Engineering A Large Language Model From ScratchSemantic proximity Read

Optimizing Large Language Model Scaling with Micro Batch Pipeline and Inference ParallelismSemantic proximity Read

Dynamic Contextual Interpolation for More Efficient Model Generalization in Large Language ModelsSemantic proximity Read

A Survey of Large Language Models: Evolution, Architectures, Adaptation, Benchmarking, Applications, Challenges, and Societal ImplicationsSemantic proximity Read

A Comparative Survey of Large Language Models: Foundation, Instruction-Tuned, and Multimodal VariantsSemantic proximity Read

Large Language Models Robustness Against PerturbationSemantic proximity Read

SuperGLUE: The AI raceSemantic proximity Read

Bridging Finance and AI: A Comprehensive Survey of Large Language Models in Financial ApplicationsSemantic proximity Read

Me-LLaMA: Foundation Large Language Models for Medical ApplicationsSemantic proximity Read