8 Types of Regression Models

Machine Learning Modeling Methods

What is Regression?

Regression is a predictive modelling, used to predict numeric value, that analyzes the relation between the target or dependent variable and independent variable in a dataset. In statistical modeling, regression analysis is a set of statistical processes for estimating the relationships between a dependent variable and one or more independent variables. The primary technique to solve the regression problems in machine learning using data modelling. It involves determining the best fit line, which is a line that passes through all the data points where the distance of the line from each data point is reduced.

There are 6 Types of Regression:

  1. Linear Regression

Linear regression is one of the most basic types of regression in machine learning. A Singular Linear Regression model consists of a predictor variable/or a feature and a dependent variable related linearly to each other. If there are multiple predictor variables or features, it is called Multiple Linear Regression model. In this regression model, there is homoscedasticity which means that the error does not vary by the size of independent variables. The error does not increase substantially if the variables get larger or smaller.

2. Logistic Regression

Logistic regression is a type of regression model, which gets used when the dependent variable is discrete, and predicts a binary variable. In statistics, the this model is used to show the probability of a certain class or event existing. Examples: 0 or 1, true or false, yes or no, failed or pass, dead or alive, etc. This means the target variable can have only two values.

3. Polynomial Regression

Polynomial Regression is another model in machine learning which is almost the same as Multiple Linear Regression. In Polynomial Regression, the relationship between independent and dependent variables, that is X(predictor variables) and y(target variable), is denoted by the n-th degree.

4. Ridge Regression

Ridge Regression model in machine learning is usually used when there is a high correlation between the X(predictor variables). It is a regression method where the model is less susceptible to overfitting.This is because, in the case of multicollinearity, the least square estimates give unbiased values. When the collinearity is very high, there can be high bias. Therefore, a bias matrix is introduced in the equation through this model.

5. Lasso Regression

Lasso Regression is a machine learning model that performs regularization along with feature selection or X(predictor variables). Lasso stands for Least Absolute Shrinkage Selector Operator. It only uses the required features and the rest are made to 0. It assigns a penalty that minimizes the coefficient value to gets it closer to zero. This helps in avoiding overfitting in the model.

6. ElasticNet Regression

ElasticNet Regression combines characteristics of both Lasso and Ridge models. This model reduces the impact of different features while not eliminating all features. It combines feature elimination from Lasso and feature coefficient reduction from the Ridge model to improve the model’s prediction.

7. k-NN

k-NN Regression is a machine learning model that approximates the association between independent variables and the continuous outcome by averaging the observations in the same k(neighborhood). The size of the k can be set or adjusted using cross-validation to select the size that minimizes the mean-squared error.