Solving More Complex Boundaries (Polynomial Regression)

1. Limitation of Linear Regression

The standard linear model:

y=mx+cy = mx + c

is a straight line, and cannot fit curves like parabolas, circles, or other nonlinear shapes.

2. Expanding Features to Fit Curves

We can model curves by adding nonlinear transformations of existing features to our dataset.

Example:

Add a new feature x2=x2x_2 = x^2

Now our model becomes:

y=m1x+m2x2+cy = m_1 x + m_2 x^2 + c

This is a quadratic model, which can fit parabolas.

This is still linear regression, because the model is linear in the parameters m1,m2,cm_1, m_2, c — even though it's nonlinear in terms of xx.

You can continue this process to model higher-degree polynomials:

y=m1x+m2x2+m3x3+⋯+mkxk+cy = m_1 x + m_2 x^2 + m_3 x^3 + \dots + m_k x^k + c

This approach is known as Polynomial Regression.

3. Overfitting

Adding more features (especially higher-degree polynomial terms) can improve the model's ability to fit complex patterns — but comes with a cost: