🔧 What is Regularization?

Regularization is a way to prevent overfitting in machine learning models.

• Overfitting happens when the model learns too much from the training data, including noise.
• Regularization adds a penalty to the model’s loss function to make it simpler and more general.

Imagine trying to draw a smooth curve through points. Without regularization, you might end up drawing a super wiggly line that fits every point exactly. With regularization, you're encouraged to draw a simpler, smoother line.

🧮 Types of Regularization: L1 and L2

1. L1 Regularization (Lasso)

• Adds the sum of the absolute values of the coefficients to the loss.

• Formula:

  Loss=Original Loss+λ∑∣wi∣Loss = Original\ Loss + \lambda \sum |w_i|

• Tends to make some coefficients exactly zero → helps with feature selection.

✅ Use L1 When:

• You think only a few features are important.
• You want to automatically ignore irrelevant features.

📦 Example:

Imagine a dataset with 100 features, but only 10 really matter. L1 can shrink the other 90 coefficients to 0, simplifying the model.

2. L2 Regularization (Ridge)

• Adds the sum of the squares of the coefficients to the loss.

• Formula:

  Loss=Original Loss+λ∑wi2Loss = Original\ Loss + \lambda \sum w_i^2