If we don’t set the threshold value then it may take forever to reach the exact zero value. We fix a threshold of a very small value (example: 0.0001) as global minima. We will keep repeating this step until we reach the minimum value (we call it global minima). Then we will subtract the result of the derivative from the initial weight multiplying with a learning rate (α). To achieve this we should take the first-order derivative of the loss function for the weights (m and c). Now as our moto is to minimize the loss function, we have to reach the bottom of the curve. If we plot the loss function for the weight (in our equation weights are m and c), it will be a parabolic curve. If we look at the formula for the loss function, it’s the ‘mean square error’ means the error is represented in second-order terms. Let’s discuss how gradient descent works (although I will not dig into detail as this is not the focus of this article). To minimize the loss function, we use a technique called gradient descent. To achieve the best-fitted line, we have to minimize the value of the loss function. In the case of Linear Regression, we calculate this error (residual) by using the MSE method (mean squared error) and we name it as loss function: Now, as we have our calculated output value (let’s represent it as ŷ), we can verify whether our prediction is accurate or not. Step 3Īs Logistic Regression is a supervised Machine Learning algorithm, we already know the value of actual Y (dependent variable). Now, to derive the best-fitted line, first, we assign random values to m and c and calculate the corresponding value of Y for a given x.
This is an equation of a straight line where m is the slope of the line and c is the intercept. Thus, by using Linear Regression we can form the following equation (equation for the best-fitted line): Let’s assume that we have a dataset where x is the independent variable and Y is a function of x ( Y=f(x)). I am going to discuss this topic in detail below.Īs the name suggested, the idea behind performing Linear Regression is that we should come up with a linear equation that describes the relationship between dependent and independent variables. On the other hand, Logistic Regression is another supervised Machine Learning algorithm that helps fundamentally in binary classification (separating discreet values).Īlthough the usage of Linear Regression and Logistic Regression algorithm is completely different, mathematically we can observe that with an additional step we can convert Linear Regression into Logistic Regression. In simple words, it finds the best fitting line/plane that describes two or more variables. Linear Regression assumes that there is a linear relationship present between dependent and independent variables. Linear Regression is a commonly used supervised Machine Learning algorithm that predicts continuous values. This article was published as a part of the Data Science Blogathon.