ML World: Learn Machine Learning! Part 2

What is Regression Analysis?

Regression analysis is implemented when you want to predict a continuous dependent variable (Target) from several independent variables (Features).

In simple words, regression analysis is predicting something.

What are the different types of Regression Analysis?

Different types of Regression Analysis methods are as follows:

1. Simple Linear Regression
2. Multiple Linear Regression
3. Polynomial Regression
4. Support Vector Regression
5. Decision Tree Regression
6. Random Forest regression
7. Logistic Regression

Woah! That is a lot to deal with at the moment. Let us get started with the simplest Machine Learning model: Simple Linear Regression.

What is Simple Linear Regression?

Simple Linear Regression/ Univariate Regression in ML is a Supervised Learning algorithm.

Basic Linear Regression permits us to sum up and watch the connection between two ceaseless quantitative factors x and y, which is spoken to by a straight line.

In basic words, Simple Linear Regression is the indication of the condition of straight line having coordinates(x,y) given as:

In the above equation,

x: x is the independent variable or feature. It is the input.

y: y is a dependent variable or target variable which our model predicted/estimated.

m: m is the slope. It determines the angle at all times.

c: c is an intercept. It is constant that decides the value of x when y=0.

Similarly, The equation of Simple Linear Regression is as follows:

• y is the predicted value of the dependent variable/target variable (y) for any given value of the independent variable/feature (x).
• B0 is the intercept, the predicted value of y when the x=0.
• B1 is the regression coefficient: the amount of change in y as x tends to increase
• x is the independent variable/feature ( the variable we expect is influencing y or the controlling variable).
• e is the error function or the difference in the actual and predicted values.

Simple Linear regression finds the line of the best fit line through your data by searching for the regression coefficient (B1) that minimizes the total error (e) of the model.

How does Simple Linear Regression find the best fit line?

To understand this, let us consider an example.

Suppose we have a data set of grades for some students and need to predict the grades of students based on the time they spent studying.

Now, in this case :

y = Grades of students.

x = Time spent Studying.

Let us consider a pattern line which speaks to the conceivable best fit line. In the above dataset, yellow stars speak to the evaluations of understudies subject to the time they spent considering.

Presently let us draw a vertical line from these data focuses onto the best fit line. Hence, you can see where the dataset point is and where it ought to be put by the best fit line.

The yellow star shows the real worth while the green dab speaks to the worth saw by the Simple Linear Regression model.

Presently, to get this best fitting line, you take this distinction between the real and assessments (y-y’) and square it and ascertain its whole. When you have the total, you have to locate the base out of them.

What a basic linear regression does is it draws loads and heaps of these lines and checks the entirety of the squares each and every time and records it and afterward it finds the base aggregate of squares.

Also, that line is known as the best fit line, and this technique is known as the normal least squares strategy.

What are the Assumptions of Simple Linear Regression?

For a simple linear regression model in practice, the model should conform to the assumptions of linear regression.

There are four assumptions associated with a linear regression model:

1. Linearity: The first assumption of simple linear regression is that the two variables (target and feature) should have a Linear relationship.
2. Homoscedasticity: The variance of residual is the same for any value of x.
3. Independence: Observations are independent of each other.
4. Normality: For any fixed value of x, y is normally distributed. In other words, it suggests that the linear combination of the random variables should have a normal distribution.

How to Implement a Simple Linear Regression model?

To execute the easiest ML model, first, you have to handle the data. I recommend you look at our Data handling arrangement to acclimate yourself with certain ideas.

To begin assembling a Simple Linear Regression Model, we first need a dataset and a difficult explanation. Let us consider a basic dataset having a reliant variable ‘Pay’ and autonomous variable ‘Long periods of involvement’.

The datasets in real-world applications are endless, but to keep things simple and wrap our heads around the concept, we should start with a much smaller dataset.

Here we have a dataset of a few employees’ salaries corresponding to their years of experience.

The first thing we need to do is import all the necessary libraries. Here we need to import the LinearRegression class from sklearn.linear_model library.

When we are through with bringing in libraries, we have to import our dataset.

After successfully bringing in the dataset and isolating it into highlights and targets, we have to part the dataset into train and test sets.

Presently, we have to make an object of the class LinearRegression() and train the model on the preparation set. After our model is prepared, we have to test it on a test set.

We can then visualize the performance of our model on the training set and testing set to get a better intuition of its efficiency.

When to use Simple Linear Regression?

Simple Linear regression helps you in understanding and quantifying the relationship between your numerical data.

You can use simple linear regression when you want to know:

1. How strong the relationship is between two variables (for example the relationship between area and house prices).
2. The value of the dependent variable at a certain value of the independent variable (For example the price of a house when the area is given).

Learning Parameter for Simple Linear Regression: Cost Function

The objective of any ML model is to gain from data. To make expectations and gain as a matter of fact. It is like the similarity of a baby learning life exercises and creating himself as indicated by his/her experience,

Additionally, the cost function causes the student to learn and address his mix-ups and create from it to evade future mistakes.

In ML, cost functions are utilized to gauge how severely models are performing.

A Cost function is a proportion of how erroneously the model acts regarding its capacity to foresee the connection among X and y.

Function to minimize Cost Function: Gradient Descent

Now that you know that our model functions accurately once the cost function is minimized, we need to think of some parameters to do this task. Thus Gradient Descent comes into play.

Gradient descent is an efficient optimization algorithm that attempts to find a local or global minimum of a function.

What are Local and Global Minima?

Local Minima :

Functions can have an elliptical, hill or valley like structure: a place from where they reach a minimum or maximum value.

It may not be the minimum or maximum for the whole function, but locally it is.

Global Minima :

The minimum over the entire function is called an “Absolute” or “Global” minimum.

There is only one global minimum, but there can be more than one local minimum.

Well, I hope these definitions gave you a little insight. Now we can try and understand the concept of gradient descent much more easily.

Gradient descent enables a model to learn the direction that the model should take to reduce the difference between actual and predicted values.

Gradient Descent in the simple linear regression refers to how the model parameters b0 and b1 should be adjusted or corrected to further minimize the cost function.

As the model iterates, it gradually converges towards a minimum where further adjustments made in the parameters produce little or zero changes in the loss.

This point is also known as convergence. At this point the weights of the model are optimum and the cost function is minimized.

This iteration depends on the learning rate of the model (α).

The learning rate determines how big the step would be on each iteration.

• If α is very small, it would take a long time to converge and become computationally expensive.
• If α is large, it may fail to converge and surpass the minimum.

What are the limitations of simple linear regression?

Some limitations of simple linear regression are as follows:

1. So far, we’ve only been able to examine the relationship between two variables. Simple linear regression allows us to predict a target variable only when it is controlled by one feature.
2. It oversimplifies a real-world problem by assuming a linear relationship between variables.
3. Simple Linear Regression is sensitive to outliers. Outliers are the data points that lie outside the scope of the regression line in this case. Thus, accuracy is harmed in the presence of outliers.

Congratulations! I hope you were able to compute your first ever ML model: Linear Regression successfully. There are many concepts to wrap your head around, but with a little dedication, you will surpass this phase.