Approach 1 – Closed-form Solution

Closed-form solution to gradient – set gradient = 0 and solve for and

Simple Linear Regression Closed Form Solution

There are some things to note;

• is in terms of so we want to calculate first.
• There are only really 4 summations we need to calculate, which we can them combine to get our values;
  • is the sum of y values
  • is the sum of x values
  • this is the sum of x*y
  • this is the sum of x squared
• is the mean of our observations. In our house example, the average house price of the houses in the training set.
• is the slope times the mean of our features. In our house example, the mean of the features is the average square footage of the houses in the training set.

So the slope is given by numerator/denominator;

numerator = (sum of XY) - (1/N)((sum of X) * (sum of Y))

denominator = (sum of X^2) - (1/N)((sum of X) (sum of X))

We can divide both the numerator and denominator by N (essentially multiplying the ratio by 1, which does not change anything) to reformulate in terms of means (sums/N);

numerator = (mean of X Y) - (mean of X)(mean of Y)

denominator = (mean of X^2) - (mean of X)*(mean of X)

Using the computed slope, the intercept can be calculated

intercept = (mean of Y) - slope * (mean of X)