COMPARISONS BETWEEN ROBUST REGRESSION APPROACHES IN THE PRESENCE OF OUTLIERS AND HIGH LEVERAGE POINTS

Abstract

The study aimed to compare a few robust approaches in linear regression in the presence of outlier and high leverage points. Ordinary least square (OLS) estimation of parameters is the most basic approach practiced widely in regression analysis. However, some fundamental assumptions must be fulfilled to provide good parameter estimates for the OLS estimation. The error term in the regression model must be identically and independently comes from a Normal distribution. The failure to fulfill the assumptions will result in a poor estimation of parameters. The violation of assumptions may occur due to the presence of unusual observations (which is known as outliers or high leverage points. Even in the case of only one single extreme value appearing in the set of data, the result of the OLS estimation will be affected. The parameter estimates may become bias and unreliable if the data contains outlier or high leverage point. In order to solve the consequences due to unusual observations, robust regression is suggested to help in reducing the effect of unusual observation to the result of estimation. There are four types of robust regression estimations practiced in this paper: M estimation, LTS estimation, S estimation, and MM estimation, respectively. Comparisons of the result among different types of robust estimator and the classical least square estimator have been carried out. M estimation works well when the data is only contaminated in response variable. But in the case of presence of high leverage point, M estimation cannot perform well.