1 / 6 RAG Architecture & Retrieval

Cut Down the Middle

You index a handbook by slicing it into fixed 512-token chunks with no overlap. Users report that definitions spanning a chunk boundary come back half-complete: the retrieved chunk has the term but the sentence that defines it landed in the next chunk. Which change to the chunking scheme addresses this directly?

A

Switch to an embedding model with fewer output dimensions so indexing and search both get cheaper

B

Overlap adjacent chunks by a fixed number of tokens so boundary-spanning passages appear whole inside at least one chunk

C

Halve the chunk size to 256 tokens so each chunk covers a single idea and its embedding is less diluted by neighbouring text

D

Deduplicate chunks by content hash before indexing so near-identical passages do not crowd each other out of the result list