HYPOTHESIS TESTING IN LINEAR REGRESSION

Because the parameters for the linear regression model have been estimated by maximum likelihood, we can use the asymptotic results about MLEs to tell us the distribution of the parameter estimates. We know that the estimates will be asymptotically Gaussian with mean equal to the true value of the parameter and the variance related to the second derivative of the likelihood evaluated at the maximum. Once again, this can all be computed analytically so that the distribution of the parameters is known to be N (bi,(shfns_x)). Combined with the convenient null hypothesis that b₁ = 0, we can compare N(b,(shins_x)) to 0 to put a P-value on any regression model. Furthermore, we can formally compare whether the slopes of two different regression models are different. In practice, the variance of the MLEs converges asymptotically, so it works well only when the number of datapoints is large (e.g., Figure 7.2) Of course, there are also approximations for small samples sizes that are implemented in any standard statistics software.

FIGURE 7.2 Convergence of the MLEs to the Gaussian distributions predicted by statistical theory. In each case, 30 regressions were calculated on randomly generated data where the true slope was equal to 3. In the left panel, regressions were fit based on 20 datapoints, while in the right panel, 100 datapoints were used.