Inference Performance

The roofline model, memory- vs compute-bound, arithmetic intensity, static vs continuous batching.

0 / 9 solved

Question Tags Difficulty Success

inferencelatency +3

Medium —

rooflineridge-point +3

Hard —

rooflinememory-bound +1

Medium —

decodebandwidth +3

Hard —

batchingoperational-intensity +1

Hard —

batchingthroughput

Hard —

disaggregated-servingprefill-decode +1

Hard —

continuous-batchingtail-latency +1

Insane —

continuous-batchinggoodput +1

Insane —

9 of 9 questions here are still unsolved.