Expected Reward Under A Stochastic Policy

In state ss an agent has three actions. The immediate reward is deterministic given the action, and the current policy is stochastic:

action π\pi probability in ss immediate reward
a1a_1 0.5 2
a2a_2 0.3 10
a3a_3 0.2 -4

What is the expected immediate reward when the agent is in ss and follows π\pi?

A

It cannot be computed from this table: π\pi gives probabilities over actions, so the transition probabilities p(s′∣s,a)p(s' \mid s, a) are still needed before any expectation over rewards can be taken.

B

3.2

C

10

D

≈2.67\approx 2.67