White RoomNEW

Score Function Sleight of Hand

REINFORCE turns θE[R]\nabla_\theta E[R] into E[Rθlogπθ(as)]E[R \cdot \nabla_\theta \log \pi_\theta(a|s)] by rewriting θπ(as)\nabla_\theta \pi(a|s) as π(as)θlogπ(as)\pi(a|s) \cdot \nabla_\theta \log \pi(a|s). What is this identity called, and why is it essential to the algorithm?