The interpolation-threshold explanation for the double-descent peak applies only when the x-axis is model capacity; when test risk is plotted against training epochs instead, a different mechanism must be responsible for the rise-then-fall pattern.

False. The same interpolation-threshold logic holds either way: whether the x-axis is capacity or epochs, risk peaks right as the fit first nears zero training error, then falls as the effective solution space widens and smoother fits get selected. The mechanism is about crossing near-interpolation, not about which axis measures it.

Double descent risk-curve explanation Interview Q&A

Classical bias-variance view

Single U-shaped curverisk falls as capacity grows, then rises again once the model starts overfitting.
Predicts a strict tradeoff — more capacity past the sweet spot can only hurt.

Double descent view

Risk rises to a peak at the interpolation threshold, then descends again as capacity keeps growing.
The second descent is invisible to bias-variance intuition, which stops modeling risk once training error hits zero.

The unstable peak

At the interpolation threshold, the model has just enough parameters to drive training error to zero — but only one (or a few) such fit exists, and it must contort itself to hit every training point exactly. Small changes in the training sample swing this fit wildly, so its variance explodes and test risk spikes. This is capacity-starved interpolation, not classical overfitting.

Past the threshold: risk falls again

1
More capacity, more solutionsOnce capacity exceeds what’s needed to interpolate, many different zero-training-error fits become available instead of just one.
2
Implicit regularization picks smoothlyGradient-based training among these many fits tends to select smoother, lower-norm interpolants rather than jagged, unstable ones.
3
Variance drops, risk descendsSmoother interpolants generalize better, so test risk falls again even though training error stays at zero.

Setup

Train one large model for more epochs, plotting test risk against training time instead of model size.

Expected

Classical intuition expects risk to keep falling smoothly, or rise steadily once training ‘overfits.’

What breaks

Risk instead peaks right when training loss first nears zero, then falls again with more epochs.

Lesson

The same interpolation-threshold mechanism applies to training time, not just capacity — crossing near-zero training error is what matters, whatever the x-axis measures.

Double descent risk-curve explanation

The unstable peak

Past the threshold: risk falls again

Sources

A focused session on your AI Research Scientist interview

More Generalization Theory & Phenomena questions