1. Improper Bayes regression: Consider inference for the normal regression model \(y\sim N_n(X\beta, \sigma^2 I)\) using the prior distribution with density proportional to \(p(\beta)\times p(\sigma^2)\) where \(p(\beta)\) is the density of the \(N(0,I \sigma^2/\lambda)\) distribution and \(p(\sigma^2) = 1/\sigma^2\).
    1. Find the conditional distribution of \(\beta\) given \(y\) and \(\sigma^2\).
    2. Find the conditional densities \(p(\sigma^2|y)\) and \(p(\beta|y)\).
  2. Confidence ellipses: Consider again the analysis of the exercise data from the previous homework.
    1. Compute and plot a two-dimensional 95% frequentist confidence ellipse for the effects of age and exmin. You may want to do this by first “projecting out” the intercept (see Homework 2 Problem 4).
    2. After projecting out the intercept, use the prior distribution from Problem 1 to compute the posterior mean of the effects of age and exmin for a reasonable range of \(\lambda\) values, and plot the “trajectory” of the posterior mean as \(\lambda\) ranges from very large to very small.
    3. Via numerical approximation, simulation or otherwise compute the 95% posterior confidence region for the effects of age and exmin, for a few \(\lambda\)-values along the trajectory in part b. Plot these on top of the trajectory of the posterior mean, and compare to the frequentist ellipse from part a.
  3. One-factor ANOVA: Consider the “treatment means” parametrization \(E[y_{i,j}] = \theta_j\) and the “treatment effects” parametrization \(E[y_{i,j}] = \mu + \tau_j\) for the one-factor ANOVA model, and recall the model matrices \(Z\) and \(T\) from the notes.
    1. Show that \(C(Z) = C(T)\).
    2. Compute the orthogonal projection matrices \(P_Z\) and \(P_T\) using basic approaches: For \(P_Z\) you can use the formula \(P_Z = Z (Z^\top Z)^{-1} Z^\top\). For \(P_T\), choose an OLS solution \(\hat \gamma\) (e.g. set to zero or sum to zero), and use the fact that \(P_T y = T \hat\gamma\) to identify \(P_T\). Show that \(P_Z=P_T\).
  4. Two-factor ANOVA: Consider the model \(E[y_{i,j}] = \mu+ a_i +b_j\) for \(i=1,\ldots,q\), \(j=1,\ldots,p\).
    1. Let \(y\) be the vectorization of the \(q\times p\) matrix with \(y_{i,j}\) as its \((i,j)\)th element. Find matrices \(X_0\), \(X_1\) and \(X_2\) so that the model may be written \(y = X_1 \mu + X_2 a + X_3 b +e\), where \(a=(a_1,\ldots,a_q)^\top\) and \(b=(b_1,\ldots, b_p)^\top\).
    2. Using linear algebra, calculus or otherwise, find the projection of \(y\) onto the model space, i.e., find the fitted value of \(y_{i,j}\) for each \((i,j)\).