Why Linear Regression Is Not So Linear

When we think of linear regression models, we imagine straight lines. Curves are associated with non linear models. Here's a catch - linear models can also fit the curves. Let's look at some linear equations to understand better.

A linear regression model has specific components.

• There's a constant

• There's a parameter or coefficient multiplied by an independent variable or feature

The linear equation is the sum of the constant β0 and product of coefficient β1 and feature X. Hence, the basic form of linear equation is:

And multiple coefficient-features can be added together to get a multilinear equation:

In statistics, this equation is linear in coefficients β1, β2 ...βi. By raising the power of a feature Xi, the equation can be made to fit the curve while retaining its linear characteristic. For example, if you square a feature, the model will follow a curve.

Though the feature is squared the model is still linear in the coefficients i.e. it still follows the basic linear equation Y = a + bx. Linear models can also contain log terms and inverse terms to follow different kinds of curves and yet continue to be linear in the parameters or coefficients like shown in the graph below:

So, why do we use Linear model to fit a curve instead of non-linear models? Linear regression is easier to use, simpler to interpret, and we can get more statistical information that help to evaluate the model. But linear models are restricted in the shapes of the curves that it can fit and sometimes it just can’t fit the specific curve in the data.

Nonlinear regression can fit various types of curves but it requires more effort to find the best fit and it's difficult to understand the influence of features due to high complexity.

If you are interested in more regression analysis concepts, look out for this space or share your comments below.