Frontier & Recent Research

Recent inference and architecture research: FlashAttention, DeepSeek (MLA, aux-loss-free MoE, FP8, multi-token prediction), speculative decoding, PagedAttention, linear/state-space models, and newer attention/softmax variants.

0 / 8 solved

Question Tags Difficulty Success

1. What FlashAttention Does Instead of Storing S and P

flash-attentionbackward-pass +3

Hard —

2. The Critic Is Half the Memory

grporlhf +3

Hard —

3. Counting Nodes in a Draft Tree

speculative-decodingtree-attention +3

Hard —

4. KV Cache Paging

pagedattentionvllm +1

Hard —

5. What a Radix Tree Saves on a Shared Prompt

prefix-cachingradix-attention +3

Hard —

6. Latent KV, Decoupled RoPE

mlakv-cache-compression +1

Insane —

7. Counting Bits in NVFP4 and MXFP4

nvfp4mxfp4 +3

Hard —

8. Prefill/Decode Split

chunked-prefilldisaggregated-serving +1

Insane —

Learning Resources

FlashAttention (Dao et al.)

DeepSeek-V3 technical report

How to Scale Your Model (the scaling book)

Up next

8 of 8 questions here are still unsolved.

Next in Advanced ML Learning Theory & Phenomena