Lesson 5 of 10
Cost and latency for AI features
Price a task by input, cache-write, cache-read and output tokens, then cut cost and latency via prompt caching, streaming, smaller models and routing.
In this lesson
- Budget the request
- Count the full task
- How prompt caching works
- Reasoning and batch pricing
- A latency equation
- Adjust the budget
- A cost ledger
- What stays true
- Costs and trade-offs
- Common mistakes
- Related topics and practice
Ultra Premium
Unlock this lesson
Ultra Premium opens the 10 premium lessons in AI systems, along with the rest of WhiteBox's premium lessons and problems.
Not ready yet? Read the free lesson: Backpressure