Article

Foresight Is Not Enough: Sentence-Level Future Signals, Self-Loop Hard Negatives, and the Calibration Gap in Small Language Models

2026-07-14

Abstract excerpt

<title>Abstract</title> <p>A language model can learn a representation that appears to anticipate future text, but such a representation may still fail to control generation behavior. We study this representation-to-behavior gap in ForesightLM-v2, a DistilGPT-2-scale language model trained with a sentence-level future-prediction head and self-loop hard negatives. Across three WikiText-103 seeds, the held-out hard...

Topics

Open a Topic to create a Post that cites this publication.

Uncategorized literature

Identifiers and source

Literature Corpus work: b70fef49-8059-5aa0-9344-af8d8f7a2197
DOI: 10.21203/rs.3.rs-10331341/v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Foresight Is Not Enough: Sentence-Level Future Signals, Self-Loop Hard Negatives, and the Calibration Gap in Small Language ModelsDOI 10.21203/rs.3.rs-10331341/v1Read

Boost Large Language Model Performance through Self-Training with Reward Guided Tree SearchSemantic proximity Read

Dynamic Probabilistic Contextual Realignment in Transformer-Based Large Language Models for Semantic ConsistencySemantic proximity Read

Semantic Calibration of Contextual Embedding in Large Language Model Generative Pathways Using Contextual-Distillation MechanismsSemantic proximity Read

Limits of Self-Correction in LLMs: An Information-Theoretic Analysis of Correlated ErrorsSemantic proximity Read

Reasoning in Large Language Models: From Chain-of-Thought to Massively Decomposed Agentic ProcessesSemantic proximity Read

Anchoring Bias in Large Language Models: An Experimental StudySemantic proximity Read

Large Language Models Robustness Against PerturbationSemantic proximity Read

Emergent Numeric Bias in Large Language Models: An Empirical Study on the Anomalous Recurrence of the Number 27 Across Independent SessionsSemantic proximity Read

Basics of Large Language Models - transformers to LLMsSemantic proximity Read

Post Hoc Explanations of Language Models Can Improve Language ModelsSemantic proximity Read

Reverse Prompting: A Memory-Efficient Paradigm for LLM Agents Through Structural RegenerationSemantic proximity Read

Upscaling A Smaller LLM to More Parameters via Manual Regressive DistillationSemantic proximity Read