Article

Noise-Robust Preference Alignment for Large Language Models via Confidence Estimation and Adaptive Optimization

2025-11-19

Abstract excerpt

Preference alignment is essential for aligning language models with human intentions, yet synthetic preference data often contains noise that hinders generalization. To address this issue, we introduce a noise-robust alignment framework that enhances model resilience to imperfect training data. The approach integrates a Preference Confidence Estimation module, which assigns reliability scores to preference samples...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 3ecb63d3-522a-5ede-87f2-c82b61ce8fd8
DOI: 10.20944/preprints202511.1435.v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Noise-Robust Preference Alignment for Large Language Models via Confidence Estimation and Adaptive OptimizationDOI 10.20944/preprints202511.1435.v1Read

Adaptive Contextual Preference Alignment for Large Language ModelsSemantic proximity Read

Optimizing Human-Controlled Preference Alignment in Large Language Models via Dense Token Masking: A Methodological ApproachSemantic proximity Read

Basics of Large Language Models - transformers to LLMsSemantic proximity Read

Enhancing Contextual Understanding in Large Language Models with Dynamic Dependency Structures: A Methodological ApproachSemantic proximity Read

Approximated Variational Bayesian Inverse Reinforcement Learning for Large Language Model AlignmentSemantic proximity Read

Discovering Latent Sycophantic Patterns in LLMs via Distributional Anomaly DetectionSemantic proximity Read

RLHF-Aligned Open LLMs: A Comparative SurveySemantic proximity Read

Self-Pluralising Culture Alignment for Large Language ModelsSemantic proximity Read

Negative Preference Reduction in Large Language Model Unlearning: An Experimental ApproachSemantic proximity Read

Fine-grained Debiasing for Large Language Modelsvia Bias Intensity and Probability DecouplingSemantic proximity Read

Anchoring Bias in Large Language Models: An Experimental StudySemantic proximity Read