Four Times The Data Or A Quarter More

You are learning axis-aligned rectangles in the plane, a class with VC dimension $d = 4$ , in the agnostic setting. The fundamental theorem of statistical learning gives sample complexity $m_{\mathcal{H}}(\epsilon, \delta) = \Theta\left(\frac{d + \log(1/\delta)}{\epsilon^2}\right)$ with $\log$ natural, and at $\epsilon = 0.10$ , $\delta = 0.05$ the constants work out to about 8,000 labeled examples.

You can label one more batch. Halving $\epsilon$ to 0.05, or cutting $\delta$ fivefold to 0.01: what does each ask for?

Halving $\epsilon$ asks for about 32,000, cutting $\delta$ for about 40,000, five times the confidence costing five times the data

Halving $\epsilon$ asks for about 16,000, cutting $\delta$ for about 40,000, so confidence is the expensive axis

Halving $\epsilon$ asks for about 32,000, cutting $\delta$ for about 9,800, since $\delta$ enters only through $\log(1/\delta)$

Both ask for about 32,000: accuracy and confidence enter the bound the same way, so either costs a factor of four