The Bias Shift

A linear layer neuron computes $z = w \cdot x + b$ before its activation. Geometrically, what does adding $b$ do to the hyperplane defined by $w \cdot x = 0$ ?

Turns the linear function into a nonlinear one in $x$

Shifts the hyperplane away from the origin without changing its orientation

Rotates the hyperplane about the origin

Rescales the input vector $x$ before the dot product