One Sample Is Not An Expectation

A team has a simulator that returns a sampled $(r, s')$ for any $(s, a)$, but no access to $p(s' \mid s, a)$. They keep their value-iteration loop and swap the expectation for one sampled successor per action:

for s in states:
    targets = []
    for a in actions:
        r, s_next = simulator.step(s, a)      # one sampled transition
        targets.append(r + gamma * v[s_next])
    v[s] = max(targets)

The environment is stochastic. What actually goes wrong, and what does the model-free fix change?