KV Bytes Per Token: MHA, GQA, MLA

DeepSeek-V2's Table 1 gives per-token KV cache in elements: MHA $2 n_h d_h l$ , GQA $2 n_g d_h l$ , MLA $(d_c + d_h^R) l$ . The released model has $l = 60$ layers, $n_h = 128$ heads, $d_h = 128$ , $d_c = 4 d_h = 512$ , $d_h^R = d_h / 2 = 64$ , all cached in fp16.

What does one token cost under MLA on this shape, and how does that compare with 8-group GQA?

About 8.8 MB per token, since each of the 128 heads caches its own 576-element latent.

About 69 KB per token, but only with RoPE disabled, since rotary keys cannot be compressed into the latent at all.

About 69 KB per token, roughly 3.6x less than GQA-8's 246 KB and about 57x less than MHA's 3.9 MB. The paper describes the result as matching GQA with only 2.25 groups.

About 138 KB per token, since the latent must be cached once for keys and once for values.

About 246 KB per token, the same as GQA-8: MLA shifts the cost from memory to compute without shrinking the bytes stored.