Post-training 0/0

Post-training

Policy gradients

Improving a policy directly from sampled returns: REINFORCE, baselines, actor-critic, generalised advantage estimation and PPO’s clipped objective. PPO and its variants are the RL algorithms used to post-train language models.

0 / 5 solved

Questions

0/5 solved
Question Tags Difficulty Success
1. Score Function Sleight of Hand
reinforcepolicy-gradient +1
Medium —
2. The Baseline That Bites Back
baselineadvantage-function +1
Medium —
3. Farming The Shaping Reward
reinforcement-learningreward-shaping +2
Hard —
4. Lambda's Balancing Act
actor-criticgae +1
Medium —
5. The Clip That Keeps You Honest
ppoclipped-objective +1
Hard —