Simple Regression: Linear Regression with a Single Input

The value x is the input.

The value y is the output.

We assume they are related by some simple linear function, y = f(x).

The function is used to predict a value of y given a value for x.

Since it is a prediction, we say that y ~ mx + b and that there is some error.

So our regression model is:

is the observed value

is the predicted value

is the error – the difference between the predicted and observed values.

The error is treated as a ‘random quantity’. More precisely, the Expected Value (E) of the error is zero;

It is saying the weighted average of all (values that the error could take times the probability that it will take the value). Basically, it is saying there is no systematic error.

It also means that the error is just as likely to be positive, as it is to be negative. In other words, the observed value is just as likely to be above predicted value as it is to be below it.