Homework 5

CHAPTER 4, PROBLEMS 4-6

Before proceeding with this week's questions, think about where the last questions related to this data set left off. The case statistics indicated that observation #17 (1982) is influential. The residual plot was not "all clear," with or without observation #17. Furthermore, even though the F statistic was significant (with and without #17), the t statistics for both regression coefficients (YEAR and AREA) were not significant. Remonving an outlier from the regression did not solve the problems we found, so further exploration is warranted. (FYI: The answers below to questions 4-6 are with #17 INCLUDED.)

4) Test for autocorrelation using the Durbin-Watson statistic.
Null hypothesis: Residuals are positively autocorrelated.
The DW statistic is 1.087. With 17 observations and K-1=2 df, the lower and upper critical values are 1.02 and 1.54, respectively. Therefore, the DW statistic is in the inconclusive range - we do not know whether the null hypothesis should be rejected. (You should also calculate the DW statistic by hand using the formula on page 118. You should get the same answer by hand as you get using SAS.)
Since the DW test is inconclusive, further investigation is warranted. A regression of the residuals against YEAR provides a nice plot to examine patterns in the residuals. There is CLEARLY a pattern. A straight downward-sloping line is followed by a curvilinear pattern with decreasing, then increasing, and again decreasing residuals. Not good!! This indicates possible autocorrelation.
5) You have already regressed production on YEAR alone and AREA alone. The R-squared did not change much in either case, but whereas in the multiple regression neither coefficient was significant, both are significant in the bivariate regressions. Furthermore, the standard errors for both coefficients are smaller when you regress production on YEAR alone or AREA alone in comparison to YEAR and AREA together. The residual plots still look funky. Nevertheless, we are hot on the trail of multicollinearity.
a) Therefore, we shall look at the matrix of correlations (from the multiple regression). This will tell us how much the coefficients are correlated with each other. The correlation between AREA and YEAR is -0.9522. This is high and indicates multicollinearity.
b) Another way to check for multicollinearity is to regress YEAR on AREA (or AREA on YEAR) and look at the R-squared. It is 0.9067 (in both cases). This is very high and indicates multicollinearuty. The square root of the R-squared is the same as the correlation coefficient.
6) What have we found? First, the DW statistic was inconclusive, but an examination of the residuals vs. YEAR indicated a potential problem. The textbook indicated several ways to deal with autocorrelation (p. 123). One of them might be appropriate.
As for multicollinearity, there is no doubt: it is there. Again, the book has offered several suggestions on dealing with multicollinearity (p. 136). In this case, our best options is probably to throw out a variable, probably YEAR.

CHAPTER 5, PROBLEMS 5-9

5) A regression of C on LATITUDE gives a mere 0.0266 for the R-squared, and the F statistic is not significant (p=0.4165). The residual plot shows possible curvilinearity: the residuals are positive, then negative, then positive again. We shall explore further!
6) A regression of C on LATITUDE and LATITUDE SQUARED gives an R-squared of 0.4205. The F stat is now significant (p=0.0014), as are the t stats for both coefficients. The regression curve is a parabola, as we would expect with a second-order polynomial regression. If you plot the predicted values against LATITUDE and the actual values against LATITUDE, you see that the regression line fits the data moderately well.
7) Equation 5.11 in the textbook (p. 152) gives us either a minimum or maximum point. In the case of D, the parabola faced down, so the formula gave us a maximum point. With C, the parabola faces up, so the formula gives us a minimum. With b1 = -0.1472 and b2 = 0.0015, the latitude with the smallest C value is 49.1 (Xm = 49.1). Trappes, with a latitude of 48.8, is the closest observation to the minimum.
8) A scatterplot of LATITUDE SQ vs LATITUDE shows an approximately linear relationship. To determine the degree of correlation, regress LATITUDE SQ on LATITUDE. The R-squared is 0.9944, so the correlation is 0.9972 (the square root). This is very high and indicates multicolinearity.
Let's look at the effects of multicollinearity in the linear and polynomial regressions for both D and C. For D, the standard error on LATITUDE is actually smaller in the polynomial regression as compared to the linear regression (0.0006 vs. 0.006). Furthermore, LATITUDE is not significant in the linear regression (p=0.3843), but it is significant in the polynomial regression (p=0.0001). This situation does not match the typical pattern of multicollinearity! What's going on? It looks like the problem of curvilinearity is overshadowing multicollinearity. The careful regression practitioner looks at the tolerances and correlation matrix to uncover the lurking multicollinearity.
How about C? The situation is slighty different. The standard error on LATITUDE is smaller in the linear regression than in the polynomial regression (0.0034 vs. 0.0358). Nevertheless, LATITUDE is not significant in the linear regression (p=0.4165), but it is significant in the polynomial regression (p=0.0004).
Looking at all of the evidence, we conclude that multicollinearity exists between the X variables.
9a) Centering reduces multicollinearity between powers of X. A regression of CENLAT SQUARED on CENLAT shows no collinearity. (R-squared is a mere 0.0319.) True, there is curvilinearity, but this does not concern us. As long as the X variables do not have a LINEAR relationship, regression can proceed smoothly.
9b) Let's look at the regression of D on CENLAT and CENLAT SQUARED. First notice that the coefficient on the squared term (i.e., SQUARED) is the same (-0.003). This is as the book predicts; it is a mathematical marvel to be pondered by mathematicians, but not by us. The R-squared is exactly the same (0.5452). The standard error on the squared term remains the same, but the standard error is smaller for CENLAT than for LATITUDE (0.0042 vs. 0.0558). Interestingly, the p value is larger for CENLAT than for LATITUDE (p=0.0395 vs. p=0.0001). However, the tolerances and correlation matrix have improved by using the centered variables, and we are pleased that we have eliminated multicollinearity.
Now on to C. Again, many numbers stay the same when using centered variables: the coefficient on the squared terms, the R-squared, the standared errors (and p values) on the squared terms - all stay the same! As with D, the standard error on CENLAT is smaller than on LATITUDE (0.0027 vs. 0.0358).
Similar to D, the p value on CENLAT is higher than on LATITUDE, only this time it is so high we do not reject the null hypothesis that B1=0! (The p value is 0.0923 as compared to 0.0004 in the non-centered regression). What is going on?! Well, first let's realize that we have banished multicollinearity; our tolerances are quite acceptable. Next, let's look at a scatterplot of predicted C vs. CENLAT (from the regression using CENLAT and CENLAT squared). Do you see how the parabola reaches its minimum point at just a little over zero? The fact that the coefficient on CENLAT is not significant means that it is possible that in real life the parabola touches ground at zero itself, and not above zero as we see in the scatterplot. If you have any remembrances of trigonometry, you will understand why this is so. Otherwise, worry not, but rather be content that we have solved the multicollinearity problem for both D and C.