Inference 0/0

Inference

Frontier inference

Recent inference research: FlashAttention, prefix sharing in a radix tree, multi-head latent attention, 4-bit floating-point formats and tree drafting. Each targets a cost from the earlier inference topics: memory traffic, cache size or decoding steps.

0 / 8 solved