Article

SimLoRA for Enhanced Domain Specific LLM Fine Tuning Using Attention Weighted Low Rank Adaptation and RLHF

2026-05-20

Abstract excerpt

<title>Abstract</title> <p>The digital transformation of public services has created an urgent demand for intelligent question-answering (QA) systems that can deliver both accurate and context-aware responses. While Low-Rank Adaptation (LoRA) enables efficient fine-tuning of large language models (LLMs) for domain-specific tasks, it struggles to capture sparse yet critical personalized information—such as residen...

Identifiers and source

Literature Corpus work: e5311be9-81b5-5e2a-8a7f-1fa833bf90e8
DOI: 10.21203/rs.3.rs-9492576/v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

SimLoRA for Enhanced Domain Specific LLM Fine Tuning Using Attention Weighted Low Rank Adaptation and RLHFDOI 10.21203/rs.3.rs-9492576/v1Read

Adaptive Contextual Preference Alignment for Large Language ModelsSemantic proximity Read

Unified Efficient Fine-Tuning Techniques for Open-Source Large Language ModelsSemantic proximity Read

Fine-grained Debiasing for Large Language Modelsvia Bias Intensity and Probability DecouplingSemantic proximity Read

Adapting Large Language Models for Low-Resource Regulated Domains: A Fine-Tuning and Retrieval-Augmented Generation Approach to Insurance Information Delivery in KenyaSemantic proximity Read

Benchmarking language models for postgraduate medical education on the Italian national specialty examinationSemantic proximity Read

Doc Bot: The Medical LLM Fine-tuned on LLaMA 3 8B Using LoRA and Insights from the Medical FieldSemantic proximity Read

An Enhanced Lightweight Clinical Decision Support System via Refined Fine-Tuning and Intelligent Retrieval-Augmented GenerationSemantic proximity Read

Medical Large Language Models: Methods and ApplicationsSemantic proximity Read

Neural Re-Contextualization for Dynamic Semantic Control in Large Language ModelsSemantic proximity Read

Assessing and optimizing large language models on spondyloarthritis multi-choice question answering (SpAMCQA): study protocol for a bilingual evaluation benchmarkSemantic proximity Read

Basics of Large Language Models - transformers to LLMsSemantic proximity Read

MenstLLaMA: A Specialized Large Language Model for Menstrual Health Education in IndiaSemantic proximity Read