Inference 0/0

Inference

Quantization

Storing weights, activations and the KV cache in fewer bits: bf16, FP8, INT4 weights, and methods such as GPTQ, AWQ and SmoothQuant. Fewer bytes per weight means faster memory-bound decoding and a larger model per GPU.

0 / 9 solved

Questions

0/9 solved
Question Tags Difficulty Success
1. Same Range, Blurrier Picture
bf16numeric-formats
Medium —
2. Half a Byte, Two Flavors
fp8numeric-formats
Medium —
3. One Ruler vs Many
quantizationcalibration
Medium —
4. Salient Channel Rescue
awqptq +1
Medium —
5. Skinny Weights, Fat Activations
quantizationllm-inference
Hard —
6. Key vs Value Granularity
kv-cache quantizationinference +1
Hard —
7. Second-Order Cleanup
gptqptq +1
Hard —
8. Block-Scaled FP8
fp8quantization +1
Hard —
9. Smooth Before You Round
quantizationgptq +1
Insane —