Question 1119

Diminishing Returns

A spam page repeats the word "kubernetes" 200 times and outranks a genuinely useful tutorial that mentions it 8 times. Switching the similarity from classic TF-IDF to BM25 largely fixes the ordering. Why?

A

BM25 needs a dense vector form of the document, so raw term counts no longer drive the score

B

BM25 saturates term frequency through k1, so the 50th occurrence adds far less than the 2nd

C

BM25 drops the inverse document frequency term, so common technical words stop inflating a score

D

BM25 scores only the first occurrence of each query term in a document and ignores all later ones