Estimating the function from the data

First we must decide the kind of function; is it constant, like , it is linear like , is it quadratic like , or is it some higher order polynomial.

Once we have chosen the kind of function, then we estimate a specific fit for that function to the data. So if we have data and we decide we want to use a quadratic function, then we would estimate a specific quadratic function that best fits the data; this is the estimated fit, (f-hat of x).

y is the actual observed output value taken from the training data.

x is a feature of the data obtained by extracting it from the training data (and associated with the actual value y).

is the predicted output value after applying our regression model to x.

Given y and , the quality metric is used to calculate the error between them.