Averaging Q Into V

State $s$ has three actions with these action values under the current policy $\pi$ , and $\pi$ chooses among them stochastically:

Compute $v_\pi(s)$ , then give the value of $s$ under the policy that acts greedily with respect to these same $q_\pi$ values.

$v_\pi(s) = 3.69$ , because the action values must be discounted by $\gamma = 0.9$ before averaging, and greedification raises it to 7.2.

$v_\pi(s) = 4.1$ , and the greedy policy's value at $s$ is at least 8.

$v_\pi(s) = 8$ , since the value of a state is the value of its best action, and 8 stays the value after greedification.

$v_\pi(s) = 5.0$ , the average of the three action values, and greedification raises it to 8.