3 / 9 Quantization & Low Precision

One Ruler vs Many

A weight matrix has output channels whose value magnitudes differ by roughly 10x. What's the main accuracy benefit of per-channel quantization scales over a single per-tensor scale here?

A

Per-channel scales remove the need for any calibration data since each channel scales itself

B

Per-channel scaling converts the tensor back to full floating point, avoiding quantization error

C

Each channel gets its own scale factor, so low-magnitude channels aren't crushed toward zero by a scale calibrated for the tensor's high-magnitude channels

D

Per-channel scales eliminate the need for a zero-point offset entirely

E

Per-channel scaling shrinks the model's memory footprint by avoiding storage of scale factors