Wrong the same way.Or wrong every time.

Two thousand models, each trained on its own 30 noisy points. Drag the line.

A straight linebias² 0.196 · variance 0.0062

A degree-12 curvebias² <0.0002 · variance 0.120

There is asine wave.You never see it.

Under the data sits a true curve. What reaches you is only points, each one the curve plus a random error.

30 points.Fit a cubic.

This is one training set and the least-squares cubic through it. A different 30 points would give a different curve.

Again.1 times.

Each new set bends the cubic a little differently. Stack them all and the bundle shows two things: where the fits land, and how far apart they land.

Tight.And wrong.

A straight line cannot bend, so all 2,000 of them land in almost the same place. They agree with each other and disagree with the sine.

Bias.

The hatched gap between the average line and the truth is the bias; squared and averaged over x, it is the big number. More data would not close the gap.

Variance.

How far the 2,000 lines scatter around their own average: the dashed lines mark one standard deviation either side. Square that spread, average it over x, and you have the variance. A line can barely move, so it is the small number.

Every lineis a point.

In Episode 1 one training set made one bowl, and its bottom was the best line: one point, slope across, intercept up. Each of these 2,000 sets has its own bowl. Here are their 2,000 bottoms.

The cloud isthe variance.

How far the cloud spreads is how much the line depends on which 30 points it got. Its centre is the best line any data could give, and that line still misses the sine.

Smaller cloud.Same gap.

Give every set more points, 30 up to 300. The variance falls from 0.0062 to 0.00061. The centre stays put, and so does the gap: bias² 0.196 at 300 points. Add x² and x³ as extra inputs and the line can bend: still Episode 1’s least squares, with more knobs.

Right on average.Wrong every time.

Degree 12 can bend anywhere, so it bends toward each set's noise. The average of 2,000 fits sits on the sine (bias² <0.0002). No single fit looks like it.

2,000 answersat one x.

Slide across the plot. Scroll and the probe sweeps. At one x the 2,000 predictions pile up into a bell; the dashed lines mark one standard deviation either side of their average. Square that spread, average it over every x, and you have the variance: 0.120.

Turn the dial up.

Episode 4 had a dial too: one neighbour memorised the town, 239 smoothed it over. Here the dial is the degree, 1 to 14, refit on the same 2,000 training sets. Hatching marks the bias; the dashed lines mark the spread.

New points.The real test.

Every training set also gets 128 new points it never learns from, drawn from the same sine and noise: the rings. Test error is the squared miss on them, averaged over 256,000 such points. Even the sine would miss them by the noise: the floor of every bar.Each training set also gets 128 new points it never learns from: the rings. Test error is the squared miss on them.

Bias falls.

A straight line misses the sine, and so does a parabola: watch the hatched gap. At degree 3 the average fit has found the curve and the gap all but closes.

The bottom of the U.

Degree 3 gives the lowest test error, 0.107. Everything added after it makes the model worse on data it has not seen.

Then variance takes over.

Each extra bend chases noise, and the ends go first, where points hold the curve from one side only. Each extra bend chases noise, the ends first. By degree 14 the variance is 39 times what it was at the bottom.

Training errornever warns you.

Error on the model's own training points, the dotted line, keeps falling:The dotted line, training error, keeps falling: 0.0825 at the sweet spot, 0.0456 at degree 14. Error on new data climbs to 0.563. The bold curve is one of the 2,000 fits: it threads its own dots and misses the new rings.

The sweet spotdegree 3 · test error 0.107

Your dialdegree · test error

Move the sweet spot.

It starts at degree 3. Can you push it to 5? Less noise and more points let a model afford more bend. It only ever lands on odd degrees: a sine is odd about its middle.

Wrong a little,both ways.

The best model sits between the two worlds: a little bias, a little variance. More data moves it toward more bend. Training error can’t find it; only data the model has never seen can. How do you set that test honestly? Next: The Honest Exam.

Computed in this page: 28,000 polynomial fits (refit in 234 ms) and 48,000 straight-line fits for the cloud. Each set’s x are spread evenly, one in each of 30 equal slices; random x would make the high degrees far wilder. Measured directly on 256,000 fresh test points, test error agrees with noise + bias² + variance within 1.0 standard errors at every degree. A bias² printed as “<…” is smaller than 2,000 datasets can measure.