Question 925

Counting Bits in NVFP4 and MXFP4

NVFP4 stores one 4-bit value plus one FP8 (E4M3) scale per 16 values, with a second-level FP32 per-tensor scale. OCP MXFP4 stores one 4-bit value plus one E8M0 power-of-two scale per 32 values.

For a 671B-parameter model's weights, what do the two formats cost, and why does NVIDIA ship the larger one?

A

MXFP4 is the more accurate format, because averaging rounding error over 32-value blocks cancels more of it than 16-value blocks do.

B

About 1,006 GB for NVFP4, since every value carries its own FP8 scale alongside its 4-bit mantissa.

C

About 377 GB for NVFP4 at 4.5 bits per value against about 356 GB for MXFP4 at 4.25. The extra quarter bit buys a 16-value block with an E4M3 scale instead of a 32-value power-of-two scale, so each block tracks its local dynamic range more closely.

D

About 336 GB for both, a flat 4 bits per value: the scale factors live in a separate side table that does not count toward weight memory.

E

About 377 GB for NVFP4, and the accuracy comes from the FP32 per-tensor scale; block size stops mattering once a global scale exists.