GPU and ML systems
Kernels
Writing GPU kernels in CUDA, Triton and Pallas: launch geometry, tiling, fusion, occupancy and the online softmax behind FlashAttention. A fused, tiled kernel avoids round trips to slow memory, which is where FlashAttention’s speedup comes from.
Questions
Question Difficulty Success