2 / 6 AI Product Systems

The Chunk Says Company

A RAG assistant over a corpus of filings splits documents into roughly 800 token chunks and indexes them with both an embedding model and BM25. One chunk reads, in full:

The company's revenue grew by 3% over the previous quarter.

A user asks: how much did ACME's revenue grow in Q2 2023? The chunk is not in the top 20 for either the vector search or the keyword search.

Which change fixes this specific failure, and why?

A

Move to a larger embedding model with a longer context window and re-embed the corpus.

B

Prepend one or two generated sentences of document context to each chunk before indexing, so the company and the quarter are in the text both indexes see.

C

Raise top-K from 20 to 200 so the chunk is somewhere in the context the generator reads.

D

Increase the chunk size to 4000 tokens so the surrounding paragraphs come along in the same embedding.