GPU and ML systems 0/0

GPU and ML systems

Kernels

Writing GPU kernels in CUDA, Triton and Pallas: launch geometry, tiling, fusion, occupancy and the online softmax behind FlashAttention. A fused, tiled kernel avoids round trips to slow memory, which is where FlashAttention’s speedup comes from.

0 / 11 solved

Questions

0/11 solved
Question Tags Difficulty Success
1. Counting Threads in a Launch
cudakernel-launch +2
Easy 67%
2. What pid Indexes in a Triton Kernel
tritonkernels +3
Medium —
3. Async Copy Fence
pallastpu-dma
Hard —
4. Computing Occupancy, Then Ignoring It
occupancyregisters +3
Hard —
5. Softmax on the Fly
flashattentionkernels +1
Hard —
6. One Trip, No Detours
kernel-fusiongpu-memory +1
Hard —
7. Which Tile Am I
pallasblockspec
Hard —
8. Split-Q, Not Split-K
flash-attentiongpu-kernels +1
Hard —
9. Rescale The Running Max
flashattentiononline-softmax
Insane —
10. Producers, Consumers, and Rotated Outliers
flash-attentionhopper +2
Insane —
11. Pallas Skips the Refetch
pallastpu +2
Insane —