supervised fine-tuning ↔ reinforcement learning

Relations (1)

related 2.00 — strongly supporting 3 facts

Reinforcement learning and supervised fine-tuning are related as distinct optimization regimes for model training, where SFT is often used for initialization [1] and exhibits different generalization properties [2] and optimization dynamics [3] compared to reinforcement learning.

Facts (3)

Sources

A Survey on the Theory and Mechanism of Large Language Models arxiv.org arXiv 2 facts

claimZhu et al. (2025c) proved that Reinforcement Learning updates occur in low-curvature subspaces orthogonal to the principal components updated by Supervised Fine-Tuning (SFT), suggesting that Reinforcement Learning operates in a distinct optimization regime that fine-tunes behavior without significantly altering primary feature representations.

claimChu et al. (2025) provided empirical evidence that Supervised Fine-Tuning (SFT) tends to memorize training data, leading to poor performance on out-of-distribution (OOD) tasks, whereas Reinforcement Learning (RL) demonstrates superior generalization capabilities.

Detecting hallucinations with LLM-as-a-judge: Prompt ... - Datadog datadoghq.com Aritra Biswas, Noé Vernier · Datadog 1 fact

claimPrompts are used to augment labeled data with reasoning chains for supervised fine-tuning (SFT) or in SFT initialization steps before reinforcement learning (RL).