Inference
Frontier inference
Recent inference research: FlashAttention, prefix sharing in a radix tree, multi-head latent attention, 4-bit floating-point formats and tree drafting. Each targets a cost from the earlier inference topics: memory traffic, cache size or decoding steps.
Questions
Question Difficulty Success