C04 · Lesson 17 of 22 · 15 min · 4 hands-on tasks

Overfitting: Testing Models on New Data

A software vendor shows Ingrid a demand model that fits Halden's history almost perfectly. Kofi is suspicious. Build ever-wigglier curves, watch the error on the data they learned from fall to almost nothing while the error on new data explodes, and learn the habits that keep models honest: hold-out tests, cross-validation and checking for leaks.