The Partial Sum Problem
A weight matrix is sharded along its contraction (inner) dimension across 4 GPUs, so each GPU computes a partial sum contributing to every element of the output. Which collective operation is required to produce the correct final output on all GPUs?
Sign in to answer questions and track your progress
Sign In