GPU and ML systems 0/0

GPU and ML systems

GPU and TPU architecture

How GPUs and TPUs run work: streaming multiprocessors and warps, the memory hierarchy, tensor cores and systolic arrays, and the roofline that compares compute with memory bandwidth. Kernel and serving performance follow from these limits.

0 / 9 solved

Reading

Papers and essays1

  1. Roofline: An Insightful Visual Performance Model for Multicore Architectures(opens in a new tab)

    Williams, Waterman and Patterson 2009 Communications of the ACM 45 min

    The roofline model: plot attainable performance against operational intensity (operations per byte moved from memory), so each kernel is capped either by peak compute or by memory bandwidth.

Videos3

LLM Inference and GPU Performance Various speakers 2 videos

Jane Street Tech Talks: Machine Learning Jane Street 1 video

Links1