How Much Traffic Tiling Removes

NVIDIA's best-practices guide analyzes $C = AB$ for $N \times N$ matrices computed in $w \times w$ tiles. In the unoptimized kernel, A is read from global memory once for each of the $N/w$ column tiles; in the shared-memory version each tile is read once and reused from on-chip memory.

For $N = 4096$ , $w = 32$ , fp32 (4 bytes per element), how much global memory traffic for reading A and B does the tiling remove?

No change in bytes moved: shared memory is a cache, so the same data still crosses the HBM bus, it just arrives faster the second time.

Traffic goes up, because each tile now has to be written into shared memory and read back out again.

From about 17.2 GB down to about 134 MB, a factor of $N/w = 128$ , because each staged tile is reused by all $w \times w = 1{,}024$ output elements the block computes.

From about 17.2 GB to about 134 MB, a factor of 32, since the tile is $w = 32$ elements wide.

From about 17.2 GB to about 8.6 GB, a factor of 2, since only A is reused and B still has to be streamed once per output tile.