HW7

Due 10/06/2017 5:00 PM

Please look at before class Tuesday in case there are questions or clarifications needed (or post on Piazza). Use LaTeX or write by hand (must be legible) and scan to submit via Sakai.

Suppose $Y \sim N(1 \alpha + X\beta, I_n/\phi)$ where $X^T1 = 0$ i.e. the columns of $X$ have been centered to have mean 0, and you are using the $g$-prior for $\beta$ specified as follows: $$\beta \mid \alpha, \phi, g \sim N(0, \frac{g}{\phi} (X^TX)^{-1}$$ where $X$ is full column rank and an independent Jeffreys prior for $\alpha$ and $\phi,$
$$p(\alpha, \phi) \propto 1/\phi$$

a. Find the posterior distribution of $\alpha, \beta \mid \phi$ and $\phi$. Simplify so that results are functions of sufficient statistics $\bar{Y}$, $\hat{\beta}$ and SSE. You should state the distributions and their hyperparameters. Hint: decompose $Y$ in terms of projections to factor the likelihood$ (Attempt by Wed)

b. Find $\tilde{\beta} = E_{\beta \mid Y}[\beta \mid Y, g]$, the posterior mean under the Zellner $g$-prior from above.

c. Find the sampling distribution of $\tilde{\beta}$. (i.e as a function of $Y$ given the true parameter what is the distribution of $\tilde{\beta}$?

d. Is the posterior mean under the $g$-prior used here an unbiased estimator of $\beta$. If not, what is the bias?

e. Find $E_{Y \mid \alpha, \beta, \phi}[ || \tilde{\beta} - \beta ||^2]$.

f. The Gauss-Markov Theorem showed that out of the class of unbiased linear estimators that the MLE (OLS) estimator had the smallest variance. If we use the posterior mean as an estimator and instead use squared error loss (see notes) can the posterior mean have a smaller expected loss than the MLE for estimating $\beta$? (Expectation is taken with respect to the distribution of $Y$.) Can it be worse? Make a plot to illustrate with $g/(1+g)$ on the x-axis and MSE (expected loss) on the y-axis for the Bayes estimator (posterior mean). (you may need to assume or fix values for some quantities that go into the loss, if so how sensitive are the plots/conclusions to those assumptions?)

g. Find a value of $g$ that minimizes the expected MSE of the Bayes estimator. Add this point to the graph above. With this value for $g$ will the expects MSE with the Bayes estimator always be smaller that the expected MSE the MLE/OLS estimator? If this depends on unknown parateters, describe how you might estimate $g$.