4 / 11 Kernels (CUDA / Triton / Pallas)

Computing Occupancy, Then Ignoring It

A kernel compiles to 128 registers per thread and is launched with 256-thread blocks on an H100 (compute capability 9.0: 64K 32-bit registers per SM, at most 64 resident warps and 2,048 resident threads per SM).

The profiler also reports the kernel is already sustaining 92% of peak HBM bandwidth. What is the occupancy, and would raising it help?

A

50%: two blocks of 256 threads is 512 of the 1,024 threads an SM can host.

B

12.5%: 128 registers x 256 threads already exceeds the SM's register file, so only one block can be resident.

C

25%: at 32,768 registers per block only 2 blocks fit in the 64K-register file, so 16 of a possible 64 warps are resident. Raising it will not help, since the kernel is already near the bandwidth roof and occupancy buys latency hiding, not bandwidth.

D

25%, and capping registers at 64 per thread would double the resident warps and roughly double throughput.

E

100%: registers are capped per thread at 255, so register usage limits what a thread can do, not how many warps are resident.