Article

Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies

2024-11-26

Abstract excerpt

<title>Abstract</title> <p>The large models, as predicted by scaling law forecasts, have made groundbreaking progress in many fields, particularly in natural language generation tasks, where they have approached or even surpassed human levels. However, the unprecedented scale of their parameters brings significant computational and storage costs. These large models require substantial computational resources and...

Topics

Open a Topic to create a Post that cites this publication.

Genomics and Rare Diseases

Identifiers and source

Literature Corpus work: f1489f21-616f-599a-b398-a7e25ead7145
DOI: 10.21203/rs.3.rs-5393239/v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Parameter-Efficient Fine-Tuning in Large Models: A Survey of MethodologiesDOI 10.21203/rs.3.rs-5393239/v1Read

Unified Efficient Fine-Tuning Techniques for Open-Source Large Language ModelsSemantic proximity Read

Layer-wise Adaptive Sparse Fine-Tuning: Boosting LLM Efficiency through Dynamic Ranks and SparsitySemantic proximity Read

A Survey of Large Language Models: Evolution, Architectures, Adaptation, Benchmarking, Applications, Challenges, and Societal ImplicationsSemantic proximity Read

Upscaling A Smaller LLM to More Parameters via Manual Regressive DistillationSemantic proximity Read

Optimizing Large Language Model Scaling with Micro Batch Pipeline and Inference ParallelismSemantic proximity Read

Dynamic Contextual Interpolation for More Efficient Model Generalization in Large Language ModelsSemantic proximity Read

Boost Large Language Model Performance through Self-Training with Reward Guided Tree SearchSemantic proximity Read

Adapting Large Language Models for Low-Resource Regulated Domains: A Fine-Tuning and Retrieval-Augmented Generation Approach to Insurance Information Delivery in KenyaSemantic proximity Read

A Comparative Survey of Large Language Models: Foundation, Instruction-Tuned, and Multimodal VariantsSemantic proximity Read

Dynamic Probabilistic Contextual Realignment in Transformer-Based Large Language Models for Semantic ConsistencySemantic proximity Read

Large Language Models Robustness Against PerturbationSemantic proximity Read

SimLoRA for Enhanced Domain Specific LLM Fine Tuning Using Attention Weighted Low Rank Adaptation and RLHFSemantic proximity Read