Bubble Trouble
A GPipe-style pipeline splits a model across p=8 stages and processes a batch as m=32 microbatches, assuming forward and backward pass time per stage are equal and no interleaving is used. Using the standard bubble time fraction approximation (p-1)/m, what fraction of the pipeline's total execution time is wasted on bubbles?
Sign in to answer questions and track your progress
Sign In