HW6

Due Sat 09/30/2017 5:00 PM

Please look at before class Tuesday in case there are questions or clarifications needed (or post on Piazza). Use LaTeX or write by hand (must be legible) and scan to submit via Sakai.

1. Suppose $Y \sim N(X\beta, \sigma^2 I_n)$. Consider finding an estimator $a$ for $\mu = X\beta$ to minimize squared error loss, $(\mu- a)^T(\mu - a)$. Show that the posterior mean of $\mu$ minimizes the posterior expected loss: $$E[(\mu- a)^T(\mu - a)]$$ where the expectation is taken with respect to the posterior distribution of $p(\mu \mid Y)$.

2. Suppose that you are using a $g$-prior for $\beta$ given $\phi$: $\beta \mid \phi, g \sim N(0, \frac{g}{\phi} (X^TX)^{-1}$ for the model $Y \sim N(X\beta, I_n/\phi)$ and an independent Jeffreys prior for $\phi,$ $p(\phi) \propto 1/\phi$ for the model in 1. where $X$ is of full column rank $p$.

   a. Find the posterior distribution of $\beta \mid \phi$ and $\phi$. Simplify so that results are functions of sufficient statistics $\hat{\beta}$ and SSE. You should state the distributions and their hyperparameters.

   b. Find the posterior distribution for $\mu = X\beta$ given $\phi$ and unconditionally. You should state the distribution and give the hyperparameters.

   c. Suppose that you decide to reparametrize your model, $X\beta = X U U^{-1} \beta = Z \alpha = \mu$ where $U$ is a $p \times p$ matrix that is full rank. What is the induced prior distribution for $\alpha \mid \phi$? Is it a $g$-prior also? Explain.

   d. Using the prior above, show that posterior distribution for $Z \alpha$ is the same as the posterior distribution for $X\beta$. (Hint: try to express the posterior distribution as a function of $P_X$, the projection onto the columns space of $X.$ Does the posterior distribution of $\mu$ depend the basis that we have used to represent the column space of $X$?

3. R Problem: Write an R function to compute $(1- \alpha) 100\%$ credible intervals for each $\beta_j$ using the output from lm (replacement for confint) for your g-prior above, with a default level of 0.95 and $g=n$.

4. Refer back to the prostate data in HW5. Use your function to create 95% credible intervals for the coefficients in the model with the response lcavol and gleason as a factor and present in a nicely formatted table. How do the intervals compare to the results using the MLE? Are the intervals shorter or wider? Provide an interpretation of the credible intervals.

5. (Challenge) If $W_1$ and $W_2$ have a joint multivariate normal distribution with $$ \left[\begin{array}{l} W_1 \cr W_2 \end{array} \right] \sim N \left( \left[\begin{array}{l} \mu_1 \cr \mu_2 \end{array} \right], \left[\begin{array}{l} \Sigma_{11} & \Sigma_{21} \cr \Sigma_{21} & \Sigma_{22} \end{array} \right] \right)$$

then the conditional distribution of $W_1$ given $W_2$ is $$W_1 \mid W_2 = w_2 \sim N(\mu_1 + \Sigma_{12} \Sigma_{22}^-(w_2 - \mu_2), \Sigma_{11} - \Sigma_{12} \Sigma_{22}^- \Sigma_{21})$$ where $\Sigma_{22}^-$ is a generalized inverse of $\Sigma_{22}$. Suppose that for the model $Y = X\beta + \epsilon$ with $\epsilon \sim N(0, \sigma^2 I)$ we use a $g$-prior for $\beta$ with a generalized inverse $$\beta \mid \phi \sim N(0, g/\phi (X^TX)-).$$ a. Find the joint (normal) distribution of $Y$ and $\beta$ given $\phi$.

b. Use the result about conditional normals above to find the posterior distribution of $\beta$ given $\phi$.

c. Find the posterior distribution of $\mu = X \beta$ given $\phi$. Does the latter depend on the choice of generalized inverse? (Can you express the result as a function of $P_X$?)

Review Chapter 2 and Appendices in Plane Answers to Complex Questions