Why the Extra Dimension

TPU v4 and v5p pods wire each chip to 6 ICI neighbors in a 3D torus, while v5e and v6e wire each chip to only 4 neighbors in a 2D torus. What is the main systems reason v4/v5p pay for the extra wiring dimension?

The extra ICI dimension lets v4/v5p run collectives directly in fp8 over the interconnect, avoiding a cast on the sending chip

A 3D torus cuts the network diameter (worst-case hop count) from roughly $O(\sqrt{N})$ to $O(\sqrt[3]{N})$ chips, shrinking worst-case latency and raising bisection bandwidth for the large- $N$ all-reduce/all-gather traffic that big-pod training generates

3D-torus chips carry more HBM per chip, so they need the extra links to keep their memory controllers fed

2D torus was adopted on v4/v5p for power efficiency, while 3D torus is used on the larger v5e/v6e pods