One Episode, Two Updates

Tabular prediction with α=0.1\alpha = 0.1, γ=1\gamma = 1, and every state initialised to V=0.5V = 0.5. A single episode is observed:

A→ r=0 B→ r=0 C→ r=1 terminalA \xrightarrow{,r=0,} B \xrightarrow{,r=0,} C \xrightarrow{,r=1,} \text{terminal}

Apply constant-α\alpha Monte Carlo, V(St)←V(St)+α[Gt−V(St)]V(S_t) \leftarrow V(S_t) + \alpha[G_t - V(S_t)], and TD(0), V(St)←V(St)+α[Rt+1+γV(St+1)−V(St)]V(S_t) \leftarrow V(S_t) + \alpha[R_{t+1} + \gamma V(S_{t+1}) - V(S_t)]. What is V(A)V(A) after this one episode under each method?

A

Monte Carlo 0.50, TD(0) 0.55

B

Monte Carlo 0.55, TD(0) 0.50

C

Monte Carlo 0.55, TD(0) 0.55

D

Monte Carlo 1.00, TD(0) 0.50