HW 6 Solutions

STA 211 Spring 2023 (Jiang)

Exercise 1

Suppose an i.i.d. random sample comes from a distribution that is symmetric about its expectation (say, \(N(0, 1)\)). Use numerical simulations to suggest that the sample mean and sample median are both unbiased estimates for the population mean (in this case 0), but that the sample mean is more efficient. Use simulations to further estimate the relative efficiency of these two estimators (the ratio of their variances).

Answer will vary, but all simulations must suggest unbiasedness of both estimators but that the variance of the sample median is lower. As well, some estimate of the relative efficiency by dividing the two sample variances must be provided.

n_sim <- 10000
mean_bias <- med_bias <- rep(NA, n_sim)

for(i in 1:n_sim){
  set.seed(i)
  temp <- rnorm(10000, 0, 1)
  mean_bias[i] <- mean(temp)
  med_bias[i] <- median(temp)
}

options(digits = 3)

cbind(mean(mean_bias), mean(med_bias))

          [,1]      [,2]
[1,] -1.03e-05 -4.19e-05

cbind(var(mean_bias), var(med_bias))

         [,1]     [,2]
[1,] 0.000101 0.000158

var(mean_bias)/var(med_bias)

[1] 0.642

As an aside, the asymptotic relative efficiency (as the sample size for a sample from the standard normal grows to infinity) of the sample median compared to the sample mean for i.i.d. draws from a standard normal distribution is \(2\pi \approx 0.642\). That is, the asymptotic variance of the sample median is \(\pi/2 \approx 1.57\) times larger than that of the sample mean for draws from this distribution.

Exercise 2

Suppose \(X_1, \cdots, X_{16}\) is an i.i.d. sample of size 16, each of which has density \(f(x) = \frac{1}{\lambda}\exp(-x/\lambda)\), where \(x > 0\) and for some \(\lambda > 0\). Consider two estimators of \(\lambda\): \(\widehat{\lambda} = \bar{X}\) and \(\tilde\lambda = 2\). What is the bias of each of these estimators? For what values of \(\lambda\), if any, is the MSE of \(\tilde{\lambda}\) lower than the MSE of \(\widehat{\lambda}\)?

Note:

\[\begin{align*} E(x_i) &= \int_0^\infty x\frac{1}{\lambda}\exp(-x/\lambda) dx\\ &= \lambda\\ Var(x_i) &= \left(\int_0^\infty x^2\frac{1}{\lambda}\exp(-x/\lambda) dx\right) - \lambda^2\\ &= 2\lambda^2 - \lambda^2\\ &= \lambda^2 \end{align*}\]

The expectations of the two estimators are:

\[\begin{align*} E(\bar{X}) &= \frac{1}{16}\sum_{i = 1}^{16} E(x_i)\\ &= \lambda\\ E(\tilde\lambda) &= 2 \end{align*}\]

The biases of the two estimators are:

\[\begin{align*} Bias(\bar{X}) &= 0\\ Bias(\tilde{X}) &= 2 = \lambda \end{align*}\]

The variances of the two estimators are:

\[\begin{align*} Var(\bar{X}) &= \frac{1}{16^2}\sum_{i = 1}^{16} Var(x_i)\\ &= \frac{\lambda^2}{16}\\ Var(\tilde{X}) &= 0 \end{align*}\]

So the MSEs of the two estimators are:

\[\begin{align*} MSE(\bar{X}) &= 0^2 + \frac{\lambda^2}{16}\\ MSE(\tilde{X}) &= (2 - \lambda)^2 + 0\\ \end{align*}\]

Solving the inequality:

\[\begin{align*} MSE(\tilde{X}) &< MSE(\bar{X})\\ (2 - \lambda)^2 &< \frac{\lambda^2}{16}\\ (8 - 5\lambda)(8 - 3\lambda) &< 0\\ \frac{8}{5} < \lambda < \frac{8}{3} \end{align*}\]

Exercise 3

Suppose an i.i.d. random sample comes from \(N(\mu, \sigma^2)\). Demonstrate that the MSE of \(\tilde{\sigma}^2\) is lower than the MSE of \(s^2\).

You may use the fact that if you have an i.i.d. sample from a \(N(\mu, \sigma^2)\) distribution, then the variance of the sample variance, \(Var(s^2) = \frac{2\sigma^4}{n - 1}.\) For this problem you may assume \(\sigma^4 < \infty\).

We know \(s^2\) is an unbiased estimator for \(\sigma^2\), and so the MSE is simply the variance, or \(\frac{2\sigma^4}{n - 1}\). The MSE of \(\tilde{\sigma}^2\) is given by:

\[\begin{align*} \tilde{\sigma}^2 &= \frac{n - 1}{n}s^2\\ E(\tilde{\sigma}^2 - \sigma^2) &= E\left(\frac{n - 1}{n} s^2 \right) - \sigma^2\\ &= \frac{n - 1}{n}\sigma^2 - \sigma^2\\ &= \frac{\sigma^2}{n}\\ Var(\tilde{\sigma}^2) &= Var\left(\frac{n - 1}{n} s^2 \right)\\ &= \left(\frac{n - 1}{n} \right)^2\left(\frac{2\sigma^4}{n - 1} \right)\\ &= \frac{(n - 1)2\sigma^4}{n^2}\\ MSE(\tilde{\sigma}^2) &= \left(\frac{\sigma^2}{n}\right)^2 + \frac{(n - 1)2\sigma^4}{n^2}\\ &= \frac{(2n - 1)\sigma^4}{n^2} \end{align*}\]

Solving the inequality, and assuming that \(0 < \sigma^4 < \infty\):

\[\begin{align*} \frac{(2n - 1)\sigma^4}{n^2} &< \frac{2\sigma^4}{n - 1}\\ \frac{(1 - 3n)\sigma^4}{n^2(n - 1)} &< 0\\ \frac{(1 - 3n)}{n^2(n - 1)} &< 0 \end{align*}\]

This inequality is true when \(n < 0\), \(0 < n < \frac{1}{3}\), and \(n > 1\). Since \(n\) must take on positive integer values, we see that the original statement is true.