- Improper Bayes regression: Consider inference for the normal
regression model \(y\sim N_n(X\beta, \sigma^2
I)\) using the prior distribution with density proportional to
\(p(\beta)\times p(\sigma^2)\) where
\(p(\beta)\) is the density of the
\(N(0,I \sigma^2/\lambda)\)
distribution and \(p(\sigma^2) =
1/\sigma^2\).
- Find the conditional distribution of \(\beta\) given \(y\) and \(\sigma^2\).
- Find the conditional densities \(p(\sigma^2|y)\) and \(p(\beta|y)\).
- Confidence ellipses: Consider again the analysis of the exercise
data from the previous homework.
- Compute and plot a two-dimensional 95% frequentist confidence
ellipse for the effects of age and exmin. You may want to do this by
first “projecting out” the intercept (see Homework 2 Problem 4).
- After projecting out the intercept, use the prior distribution from
Problem 1 to compute the posterior mean of the effects of age and exmin
for a reasonable range of \(\lambda\)
values, and plot the “trajectory” of the posterior mean as \(\lambda\) ranges from very large to very
small.
- Via numerical approximation, simulation or otherwise compute the 95%
posterior confidence region for the effects of age and exmin, for a few
\(\lambda\)-values along the trajectory
in part b. Plot these on top of the trajectory of the posterior mean,
and compare to the frequentist ellipse from part a.
- One-factor ANOVA: Consider the “treatment means” parametrization
\(E[y_{i,j}] = \theta_j\) and the
“treatment effects” parametrization \(E[y_{i,j}] = \mu + \tau_j\) for the
one-factor ANOVA model, and recall the model matrices \(Z\) and \(T\) from the notes.
- Show that \(C(Z) = C(T)\).
- Compute the orthogonal projection matrices \(P_Z\) and \(P_T\) using basic approaches: For \(P_Z\) you can use the formula \(P_Z = Z (Z^\top Z)^{-1} Z^\top\). For \(P_T\), choose an OLS solution \(\hat \gamma\) (e.g. set to zero or sum to
zero), and use the fact that \(P_T y = T
\hat\gamma\) to identify \(P_T\). Show that \(P_Z=P_T\).
- Two-factor ANOVA: Consider the model \(E[y_{i,j}] = \mu+ a_i +b_j\) for \(i=1,\ldots,q\), \(j=1,\ldots,p\).
- Let \(y\) be the vectorization of
the \(q\times p\) matrix with \(y_{i,j}\) as its \((i,j)\)th element. Find matrices \(X_0\), \(X_1\) and \(X_2\) so that the model may be written
\(y = X_1 \mu + X_2 a + X_3 b +e\),
where \(a=(a_1,\ldots,a_q)^\top\) and
\(b=(b_1,\ldots, b_p)^\top\).
- Using linear algebra, calculus or otherwise, find the projection of
\(y\) onto the model space, i.e., find
the fitted value of \(y_{i,j}\) for
each \((i,j)\).