Ask My Docs

Design a retrieval-augmented assistant over a large enterprise's internal documents. An employee asks a question in natural language and gets an answer grounded in documents they are allowed to see, with citations they can click.

Functional requirements

Ingest documents from wikis, ticket systems, and file shares. Answer a question with a streamed response and inline citations. Never surface content the asker lacks permission to read. Reflect an edited or newly restricted document quickly. Let a user follow up in the same conversation.

Non-functional requirements

A permission change must take effect within 60 seconds; this is a security requirement, not a freshness nicety. A content edit should be answerable within 5 minutes. p95 time to first token under 3 seconds. The answer must be traceable to the exact passages that produced it.

Scale

50 million documents averaging 8KB, growing by 400,000 per day. 120,000 employees, 90,000 questions per day with a sharp weekday-morning peak of 12 questions per second. A document averages 6 chunks, so roughly 300 million vectors. Permission groups per document average 14, and a group can hold 20,000 people.

Out of scope

The chat UI, model fine-tuning, and connector authentication to each source system.