Heteroskedasticity and diagnostics
The classical assumption required for the OLS estimator to be efficient states that the variance of the error term has to be constant and the same for all observations. This is referred to as a homoskedastic error term. When that assumption is violated and the variance is different for different observations we refer to this as heteroskedasticity. This chapter will discuss the consequences of violating the homoskedasticity assumption, how to detect any deviations from the assumption and how to solve the problem when present.
Consequences of using OLS
The classical assumptions made on the error terms are that they are uncorrelated, with mean zero and constant varianceoU2 . In technical terms this means that
Assumption (9.1) is in use to make the OLS estimators unbiased and consistent. Assumptions (9.2) and (9.3) are important for the OLS estimator to be efficient. Hence, if (9.2) is ignored and the residual variance is heteroskedastic we can no longer claim that our estimator is the best estimator among linear unbiased estimators. This means that it is possible to find another linear unbiased estimator that is more efficient.
Heteroskedasticity implies that
The OLS estimators of the population parameters are still unbiased and consistent. The usual standard errors of the estimated parameters are biased and inconsistent.
It is important to understand that the violation of (9.2) makes the standard errors of the OLS estimators and the covariances among them biased and inconsistent. Therefore tests of hypothesis are no longer valid, since the standard errors are wrong. To see this, consider the variance of the estimator for the slope coefficient of the simple regression model:
The expression given by (9.4) represents the correct variance that should be used. Unfortunately it involves the unknown population variance of the error term which is different for different observations.
Since the error term is heteroskedastic, each observation will have a different error variance. The expression will therefore deviate from the variance estimated under homoskedasticity, that is:
As can be seen in (9.5) the variance of the OLS estimator is different from the expected value of the sample variance of the estimator that works under the assumption of a constant variance.
An important use of the regression equation is that of making predictions and forecasts of the future. Since the OLS estimators are unbiased and consistent, so will the forecasts. However, since the estimators are inefficient, the uncertainty of the forecasts will increase, and the confidence interval of the forecast will be biased and inconsistent.
Since we know that heteroskedasticity invalidate test results it is very important to investigate whether our empirical model is homoskedastic. Fortunately there are a number of test, graphical as well as statistical that one can apply in order to receive an answer to the question. Below the most commonly used test will be discussed.
A natural starting point in detecting possible deviations from homoskedasticity is to plot the data. Since we are interested in the behavior of the error term and its variation, two obvious scatter plots are given in Figure 9.1 that comes from a simple linear regression model. In Figure 9.1a the dependent variable is plotted against its explanatory variable X. Here we can see a clear pattern of heteroskedastidity which is driven by the explanatory variable. That is, the larger the value of x, the larger is the variance of the error term.
As an alternative to Figure 9.1a we can also plot the estimated residuals directly against X, as is done in Figure 9.1b. In the simple regression case with just one explanatory variable these two graphs are always very similar. However, when using a multiple regression models the picture might be different, since the residual is a linear combination of all variables included in the model. Since it is the partial effect of x on the residual that is of primary interest, it is advised that the residual should be plotted against all involved variables separately. If it is possible to find a systematic pattern that give indications of differences of the variances over the observations, one should be concerned. Graphical methods are useful, but sometimes it is difficult to say if heteroskedasticy is present and found harmful. It is therefore necessary to use statistical test. The graphical method is therefore merely a first step in the analysis that can give a good picture of the nature of the heteroscedasticity we might have, which will be helpful later on when correcting for it.
Figure 9.1 Scatter plots to detect heteroskedasticity
We are interested in the rate of the return to education and estimate the coefficients of the following human capital model:
We use a sample of 1483 individuals with information on hourly wages in logarithmic form (lnY), years of schooling (ED), and years of work experience (year). Both explanatory variables are also squared to control for any non linear relation between the dependent variable and the two explanatory variables. Using OLS we received the following results with f-values within parenthesis:
In Y should be interpreted as the predicted value of lnY. We observe that all coefficients are significantly different from zero. The squared terms have very small coefficients, even though their f-values are sufficiently large to make them significant. Observe that the coefficient for the square of year is different from zero. Since the value is very small and we only report the first three decimals it appears to be zero. Its f-value shows that the standard error is even smaller.
We suspect that our residual might be heteroskedastic and we would like to investigate this by looking at a graph between the residual and the two explanatory variables. Sometimes, to enlarge the possible differences in variance among the residuals it is useful to square the estimated residual. If we do that we receive the graphs given in Figure 9.2.
Figure 9.2 Graphical analysis of error variance
In Figure 9.2a we see the squared error term against the number of years of schooling, and a pattern can be identified. The error variance seems to be larger for lower years of schooling than for more years of schooling. This is of course just an indication that we need to investigate further using formal statistical test.
In Figure 9.1b the picture is less obvious. If ignoring the top outlier that makes it look likes the variance is higher around 35 year of work experience, it is difficult to say if there is some heteroskedasticity to worry about. Since it is an unclear case, we still need to investigate the issue further, holding in mind that hypothesis testing is meaningless if the standard errors of the parameter estimates are wrong. Below we will go through the basic steps of the three most popular tests discussed in the textbook literature.