Influence of High Leverage points

A high leverage point is point along the x-axis that is an outlier. This may or may not affect the fit. It has the potential, but only influences the fit if it is outside the trend of the other data.

An influential observation is one where the removal of the point significantly changes the fit. So a high leverage point may be an influential observation and we can test that by taking it out of the dataset and recalculating.

It is important when fitting data, to look at the data (make visual plots) to see if there are influential observations that you may want to remove to get a better fit for predictive purposes.

Asymmetric Cost Functions

What if the cost of over-estimating if different than the cost of under-estimating? In this case we can use something other that RSS(w) to calculate the cost of a fit.