Regularisation

Helps the model avoid overfitting

By adding a penalty term called the regularisation term to the loss function, it discourages models from assigning too much weight to any one feature by making a certain parameter $a_k$ too prominent.

The parameter $\lambda$ is manually decided and used to weigh its importance against Mean Square Error.

It uses Lp norms of weights to determine which regularisation e.g. L1 regularisation uses L1 norm of weights.

Examples of regularisation terms

• Ridge Regressor
• Lasso Regressor
• ElasticNet