7 / 9 GPU / TPU Architecture

Stride Bank Collision

On a GPU with 32 shared-memory banks, a warp's 32 threads each read a 4-byte word at a fixed stride of 32 words. What happens, and why?

A

Tensor cores automatically reroute the conflicting accesses across neighboring warps at no cost

B

Bank conflicts apply only to texture memory reads, so shared-memory throughput is unaffected here

C

All 32 accesses land in the same bank, so they serialize into up to 32 sequential transactions instead of one

D

No conflict occurs because 32 threads matching 32 banks always guarantees a full-throughput broadcast

E

The compiler always inserts padding for stride-32 patterns, guaranteeing conflict-free access at compile time