1. GLS and BLUP: Consider the linear model \(y=X\beta + \epsilon\), \(E[\epsilon]=0\), \(V[\epsilon]=\Sigma\). Let \(\delta= \hat\beta - \beta\), where \(\hat\beta\) is the OLS (not GLS) estimator, so \(\hat \beta = \beta + \delta\). Note that if we could make a good “prediction” of \(\delta\), we might be able to subtract it from \(\hat\beta\) to get a better estimator of \(\beta\).
    1. Find the covariance of \(\delta\) with \(y\) and with \(N^\top y\), where \(N\) is an orthonormal basis for the left null space of \(X\).
    2. Find the BLP \(\tilde\delta\) of \(\delta\) based on \(N^\top y\), and the BLUP \(\check \delta\) of \(\delta\) based on \(y\).
    3. Let \(\tilde \beta = \hat\beta - \tilde \delta\), and \(\check \beta = \hat\beta - \check \delta\). Under what conditions on \(X\) and \(\Sigma\) are these estimators equal, and equal to the GLS estimator?
  2. Random plot: A field experiment is performed to assess the differences among \(p\) levels of a certain agricultural factor. Suppose \(q\) plots of land have each been divided into \(p\) subplots, to which the \(p\) levels of the experimental factor have been randomly assigned. Consider the additive model \(y_{i,j} = a_i + b_j + \epsilon_{i,j}\) where \(y_{i,j}\) is the yield from factor level \(j\) in plot \(i\).
    1. Using sum-to-zero side conditions for the \(a_i\)’s, write out the OLS estimates of the \(a_i\)’s and \(b_j\)’s.
    2. Now consider the random effects model where the vector of plot effects \(a=(a_1,\ldots, a_q)\) has zero mean and variance \(\tau^2 I\). Find the BLUE of the \(b_j\)’s and the BLUPS of the \(a_i\)’s.
    3. Describe the covariance matrix of the (vectorized version) of the \(y_{i,j}\)’s under the random effects model. Which observations are correlated, and how are they correlated?
  3. Fixed versus random: Suppose \(y=W\alpha+ X \beta + \epsilon\) where \(E[\epsilon] = 0\) and \(V[\epsilon] = \sigma^2 I\). Let \(\hat\beta\) be the OLS estimate of \(\beta\) from this model, that is, \(\hat\beta\) is the “\(\beta\) part” of \((\hat\alpha_{OLS},\hat\beta_{OLS})\). Let \(\tilde \beta\) be the GLS estimate based on the assumption that \(E[\alpha] =0\), \(V[\alpha] = \Phi\) and \(E[ \alpha \epsilon^\top ] =0\).
    1. Compute the bias, variance and mean squared error of \(\hat\beta\) and \(\tilde \beta\) as a function of \(\alpha\) and \(\beta\).
    2. Describe values of \(\alpha\) for which \(\hat \beta\) beats \(\check\beta\) in terms of MSE, and values for which it is worse.