teaching_llm_applications

Week 13 — Reinforcement Learning from Human Feedback (RLHF): A Deep Dive

Lecture Overview

Week 7 introduced RLHF as one component of the fine-tuning pipeline. This week we go considerably deeper: deriving the mathematics from first principles, understanding the reward model in detail, working through the PPO update rule as applied to language models, examining failure modes, and surveying the frontier of RLHF alternatives. By the end of this week students should be able to implement every stage of RLHF from scratch on a toy problem.


1. Why RLHF?

Supervised fine-tuning (SFT) teaches a model to imitate demonstrations. But human preferences are not always easily captured by demonstrations alone:

RLHF reframes the problem: instead of asking humans to write ideal completions, ask them to compare two completions and say which they prefer. This comparison signal is cheaper to collect and more reliable — humans are better at ranking than generating.


2. The RLHF Pipeline: Three Stages

┌─────────────────────────────────────────────────────────────────┐
│  Stage 1: Supervised Fine-tuning (SFT)                          │
│  Pre-trained LLM  →  Fine-tune on curated demonstrations        │
│  Result: π_SFT  (a good starting policy)                        │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│  Stage 2: Reward Model Training                                 │
│  Collect pairwise comparisons (y_w ≻ y_l | x)                  │
│  Train RM to assign scalar reward r(x, y)                       │
│  Result: r_φ  (a proxy for human preference)                    │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│  Stage 3: RL Optimisation (PPO)                                 │
│  Optimise π_θ to maximise E[r_φ(x,y)] - β·KL(π_θ || π_SFT)    │
│  Result: π_RLHF  (the final aligned model)                      │
└─────────────────────────────────────────────────────────────────┘

3. Stage 1: Supervised Fine-tuning

The SFT model is the initialisation for RL. It is trained on a dataset of high-quality (prompt, response) pairs:

L_SFT = -∑_t log π_θ(y_t | x, y_{<t})

The SFT model serves two roles:

  1. It is the starting policy for PPO — already a reasonable model that produces sensible completions.
  2. It is the reference policy π_ref used in the KL penalty during RL to prevent the model from drifting too far.

4. Stage 2: The Reward Model

4.1 Preference Data Collection

Human annotators are shown a prompt x and two completions y_w (winner / preferred) and y_l (loser / dispreferred). They choose which they prefer. This produces a dataset:

D = {(x^(i), y_w^(i), y_l^(i))}_{i=1}^{N}

Annotators may be given a rubric covering:

4.2 The Bradley-Terry Model

We model human preference as arising from an underlying latent quality score r*(x, y):

P(y_w ≻ y_l | x) = σ(r*(x, y_w) - r*(x, y_l))

where σ is the sigmoid function. This is the Bradley-Terry model of paired comparisons.

4.3 Training the Reward Model

We initialise the reward model from the SFT model and add a linear head that maps the final hidden state to a scalar:

r_φ(x, y) = W^T h_{final}(x, y)

The reward model is trained to minimise the negative log-likelihood of the observed preferences:

L_RM = -E_{(x,y_w,y_l) ~ D} [ log σ(r_φ(x, y_w) - r_φ(x, y_l)) ]

This is equivalent to binary cross-entropy on the preference pair, where the label is always “y_w is better.”

4.4 Reward Model Properties


5. Stage 3: Reinforcement Learning with PPO

5.1 The RL Objective

We treat the language model as a policy π_θ that maps a prompt x to a completion y. The goal is to maximise:

J(θ) = E_{x ~ D, y ~ π_θ(·|x)} [ r_φ(x, y) ] - β · KL(π_θ(·|x) || π_ref(·|x))

The two terms:

The KL penalty can be absorbed into a per-token reward:

r̃(x, y) = r_φ(x, y) - β · ∑_t log [π_θ(y_t | x, y_{<t}) / π_ref(y_t | x, y_{<t})]

5.2 Proximal Policy Optimisation (PPO)

PPO (Schulman et al., 2017) is the standard RL algorithm used in RLHF. It is an on-policy actor-critic method.

Key components:

Actor (the LLM being trained): π_θ — generates completions.

Critic (value function): V_ψ(x, y_{<t}) — estimates the expected future reward from state (x, y_{<t}).

Advantage estimate: how much better is action y_t than the average?

A_t = r̃_t + γ V_ψ(s_{t+1}) - V_ψ(s_t)

In practice for LLMs, with γ = 1 and using GAE (Generalised Advantage Estimation):

A_t^{GAE} = ∑_{k=0}^{T-t} (γλ)^k δ_{t+k}    where δ_t = r̃_t + V_ψ(s_{t+1}) - V_ψ(s_t)

Clipped surrogate objective:

L_PPO = E_t [ min(ρ_t A_t,  clip(ρ_t, 1-ε, 1+ε) A_t) ]
where ρ_t = π_θ(y_t x, y_{<t}) / π_old(y_t x, y_{<t}) is the importance sampling ratio.

The clip prevents excessively large updates when π_θ deviates far from π_old.

5.3 The PPO-RLHF Training Loop

for each iteration:
    1. Sample prompts x ~ D
    2. Generate completions y ~ π_θ(·|x)           [rollout]
    3. Score completions: r = r_φ(x, y)             [reward]
    4. Compute KL penalty against π_ref             [regularise]
    5. Compute advantages using critic V_ψ          [estimate]
    6. Update π_θ with clipped PPO objective        [actor update]
    7. Update V_ψ with MSE loss on returns          [critic update]

This is computationally expensive: at each step we need four model forward passes — the actor, critic, reference policy, and reward model.

5.4 Practical Tricks


6. Reward Hacking

Reward hacking (Amodei et al., 2016; Skalse et al., 2022) occurs when the policy finds behaviours that achieve high reward model scores without actually satisfying the underlying human preference.

6.1 Examples

Behaviour Why it scores highly Why it is bad
Very long responses RMs often reward thoroughness Wastes tokens; buries the answer
Excessive hedging RMs reward safety Becomes unhelpful (“I can’t be sure but…”)
Sycophantic agreement RMs reward positively-toned responses Confirms user misconceptions
Repetition Padding to seem comprehensive Reduces information density
Confident incorrect answers Confidence may be rewarded Hallucination

6.2 Measuring Reward Hacking

Monitor the KL divergence from π_ref throughout training. Rapid KL growth without corresponding human evaluation improvement signals reward hacking.

The reward-KL frontier: as β decreases (weaker KL penalty), reward increases but so does the gap between RM score and true human preference. Optimal β is task-dependent.

6.3 Mitigations


7. Variants and Alternatives to PPO-RLHF

7.1 Direct Preference Optimisation (DPO)

Rafailov et al. (2023) show that the RLHF objective with a KL constraint has a closed-form optimal policy:

π*(y|x) = π_ref(y|x) exp(r*(x,y)/β) / Z(x)

Substituting back and rearranging, the reward can be expressed in terms of the policy:

r*(x,y) = β log [π*(y|x) / π_ref(y|x)] + β log Z(x)

Plugging this into the Bradley-Terry preference model and optimising directly over π_θ gives the DPO loss:

L_DPO(θ) = -E_{(x,y_w,y_l)} [ log σ( β log[π_θ(y_w|x)/π_ref(y_w|x)] - β log[π_θ(y_l|x)/π_ref(y_l|x)] ) ]

DPO advantages:

DPO limitations:

7.2 REINFORCE Leave One Out (RLOO)

A simpler RL algorithm than PPO for language models: for each prompt, sample K completions and use the mean reward of the other K-1 as a baseline for each:

∇J ≈ ∑_k (r_k - mean_{j≠k} r_j) ∇ log π_θ(y_k | x)

Lower variance than vanilla REINFORCE; no separate value network required.

7.3 GRPO (Group Relative Policy Optimisation)

DeepSeek-R1 (2025): samples a group of completions per prompt, normalises rewards within the group, and uses the normalised advantage for the PPO-style update. No critic needed; trains efficiently on mathematical reasoning tasks.

7.4 Iterative DPO / Online DPO

Generate new completions with the current policy, score them with the RM, construct new preference pairs, and run DPO. Bridges the gap between offline DPO and online PPO — better coverage of the current policy’s distribution.

7.5 Rejection Sampling Fine-tuning (RFT)

Generate K completions per prompt, keep only those with reward above a threshold, and fine-tune on them with SFT. Simple and effective; used by LLaMA-2 (Touvron et al., 2023) as a first pass before PPO.

7.6 Comparison Summary

Method RM needed RL loop Stability Data efficiency
SFT No No High Moderate
Rejection sampling Yes No High Low
DPO No (π_ref needed) No High Moderate
RLOO Yes Yes Medium Moderate
PPO-RLHF Yes Yes Low High
Iterative DPO Yes Partial Medium High
GRPO Yes Yes Medium High

8. Constitutional AI

Bai et al. (2022, Anthropic) replace human preference labels with AI-generated critiques.

Two-stage process:

Stage 1 — Supervised Learning from AI Feedback (SL-CAI):

  1. Prompt the model to generate a potentially harmful response.
  2. Ask the model to critique the response according to a constitution (a list of principles).
  3. Ask the model to revise the response to better satisfy the principles.
  4. Fine-tune on the revised responses.

Stage 2 — Reinforcement Learning from AI Feedback (RLAIF):

  1. Generate pairs of responses.
  2. Ask the model (or a separate model) to score which is more constitutional.
  3. Train a reward model on these AI-generated preference labels.
  4. Run PPO with this RM.

Benefits: scalable (no human raters needed for RM training), more consistent than crowd-sourced annotation, allows explicit statement of values.


9. Process Reward Models (PRMs)

Standard reward models score the final output. For multi-step reasoning tasks (mathematics, code), this is a weak signal — the model may reach the right answer via wrong reasoning, or wrong answer via mostly correct reasoning.

Process reward models assign a reward to each reasoning step:

r_total = ∑_t r_t(x, y_{≤t})

Benefits:

Drawbacks:

OpenAI’s Let’s Verify Step by Step (Lightman et al., 2023) shows PRMs outperform outcome reward models on MATH.


10. Annotator Agreement and Reward Model Quality

Human annotation is noisy:

Implications:


11. Evaluation of RLHF-Trained Models

11.1 Win Rate

Compare the RLHF model against the SFT baseline using human evaluation or LLM-as-judge:

Win Rate = (# prompts where RLHF is preferred) / (# total prompts)

A win rate > 50% indicates improvement.

11.2 MT-Bench and AlpacaEval

11.3 Reward Model Score vs Human Preference

Plot RM score vs human preference rating on a held-out set. The correlation should be high early in training; degradation in correlation signals reward hacking.


12. Practical This Week

See practicals/week13_practical.py:


13. Further Reading


Discussion Questions

  1. Derive the DPO loss from the RLHF objective with a KL constraint. What key assumption allows the reward model to be eliminated?
  2. Explain reward hacking with two concrete examples from deployed systems. Why does the KL penalty not fully prevent it?
  3. Compare PPO-RLHF and DPO in terms of data requirements, training stability, and computational cost. In what scenarios would you choose each?
  4. A team wants to apply RLHF to a clinical decision support system. What are the risks of using crowd-sourced annotators to generate preference data? Propose an annotation protocol that mitigates these risks.
  5. What is a process reward model and why might it outperform an outcome reward model for mathematical reasoning tasks?