Twenty Two Thousand Tokens

An internal assistant answers HR questions from the employee handbook. There is exactly one handbook, currently 40 pages, about 22,000 tokens. Previous editions from 2023 and 2024 are also in the index, because someone loaded the whole SharePoint folder.

The complaint: asked about PTO carryover, the assistant cites the 2023 rule, which was replaced. The 2023 chunk is a strong lexical and semantic match for the query. Traffic is roughly 300 questions a day.

Which approach actually fixes this?

Keep everything and add a recency boost to the retrieval score so newer documents outrank older ones.

Delete the superseded editions from the index, then stop retrieving: put the current handbook in the system prompt behind a cache breakpoint.

Keep retrieval, add a cross-encoder reranker over the candidates, and tune chunk boundaries so each policy section stays intact in one chunk.

Fine-tune a small model on the handbook so the current policy is in the weights and no retrieval is needed.