Overfitting

A phenomenon that occurs in ML when a model is trained too well on training data and thus performs poorly on new, unseen data.

This typically happens when a model:

• Is trained using too many features
• Has inbalance in the data
• Has too high a complexity compared to the dataset

Overfitting results in poor generalisation to new data, by memorising the noise in the training data rather than learning the correct pattern.

This typically happens if the model complexity is too high compared to the dataset complexity, where model complexity refers to the polynomial degree we use.