Lasso Regressor

A polynomial regressor which includes an additional regularisation term $λ R (a, b)$ which consists of the absolute values of the trainable parameters

λ R (a, b) = \frac{λ}{2 N} (\sum_{k}^{M} | a_{k} | + | b |)

Lasso stands for the Least Absolute Shrinkage and Selection Operator.

Lasso Regressor adds a penalty term proportional to the absolute value of weights.

This allows weights to become exactly zero, effectively performing feature selection, where we can identify which features are most important for our model.

This allows for sparse models, which refer to models where some features have weights of exactly zero and are completely eliminated.

On the other hand, non-sparse models mean all features still contribute to the prediction.

They are represented by the penalty term: $α \sum | w_{j} |$