The Nameless Chunk

A 10-K filing is split into 400-token chunks. A user asks: "By what percent did Acme's revenue grow in Q2 2025?" The chunk holding the answer is never retrieved in the top 20, by either the dense retriever or BM25.

--- chunk 12 (three pages earlier) ---
Acme Corporation reported total revenue of $312.4 million for the
three months ended June 30, 2025 ...

--- chunk 47 (holds the answer) ---
The company's revenue grew by 3% over the previous quarter, and
gross margin improved 40 basis points to 61.2%. Management attributes
the improvement to the mix shift toward subscription contracts.

Which change most reduces the top-20 failure rate for queries shaped like this one?

Switch the similarity metric from cosine to dot product and normalize the embeddings before indexing.

Before indexing, prepend to each chunk a short generated sentence situating it in the document, and index that combined text in both the embedding and the BM25 index.

Add a cross-encoder reranker over the top 150 candidates, so the chunk that answers the question is scored directly against the question instead of by vector distance.

Raise the chunk size to 2,000 tokens so chunk 12 and chunk 47 land in the same chunk and share one vector.