Too Cool, Too Chill

You're tuning k for a k-NN classifier. Setting k=1 versus k=n (the whole training set) has very different effects on the decision boundary. What's the tradeoff?

Small k reduces variance because it relies on fewer noisy points

Small k gives a jagged, low-bias but high-variance boundary that can overfit noise; large k gives a smoother, high-bias but low-variance boundary that can underfit

Small k always underfits and large k always overfits, regardless of the data

k has no effect on bias or variance, only on training time