The Eight Times Rule

LightGBM's Gradient-based One-Side Sampling is training on 100,000 rows with a=0.2a = 0.2 and b=0.1b = 0.1. Each iteration keeps the 20,000 rows with the largest ∣gi∣|g_i|, then randomly draws 10,000 more from the remaining 80,000. When the variance gain of a candidate split is computed, the gradients of the sampled small-gradient rows are multiplied by a constant.

What is that constant, and what breaks without it?

A

No constant is needed: the small-gradient rows are drawn uniformly, so the estimated gain is already unbiased and any factor just rescales the learning rate

B

1−ab=8\frac{1-a}{b} = 8: the 10,000 sampled rows stand in for all 80,000 small-gradient rows, and without it their side of every split sum is understated eightfold

C

1b=10\frac{1}{b} = 10: each sampled row represents the rows not drawn alongside it, so the inverse sampling rate is the right weight and the 1−a1-a factor double-counts

D

ab=2\frac{a}{b} = 2: the two sampling rates are put on a common scale, and omitting it only slows convergence