Eight Cores Forty Trees

You train two 40-tree ensembles on the same tabular data on an 8-core machine: a random forest and a gradient-boosted tree model with 40 boosting rounds. Individual tree fitting takes about the same time in both.

The forest finishes far sooner. Which explanation is correct?

A

Neither parallelizes across trees; both must hold the full dataset in memory, so the difference is cache behaviour

B

Each forest tree is fit on its own bootstrap sample and depends on no other tree, so all 40 fit at once; boosting round mm needs the residuals of Fm−1F_{m-1}, so its trees are strictly sequential and only the split search inside one tree parallelizes

C

Both parallelize across trees; the forest is faster only because bootstrap samples are smaller than the full training set

D

Boosting parallelizes across trees and the forest cannot, because its bootstrap samples overlap and would race on shared rows