HW 9 Solutions

STA 211 Spring 2023 (Jiang)

Exercise 1

Consider a uniform distribution on \((0, \theta)\) (that is, \(f_X(x) = 1/\theta\) for \(x \in (0, \theta)\)). Is this a member of the exponential family? If so, identify the components in canonical form and use the log-partition to calculate the MLE for \(\theta\) from an i.i.d. sample. If not, explain why not.

No, this is not a member of the exponential family. There is no way to disentangle \(x\) from \(\theta\) in the term \(I(x \in (0, \theta))\); there is no possible \(h(x)\) that can do so while being free of \(\theta\) as required.

Exercise 2

Consider a normal distribution \(N(\mu, \mu)\) for \(\mu > 0\) (that is, where the variance equals the mean). Is this a member of the exponential family? If so, identify the components in canonical form. If not, explain why not.

\[\begin{align*} f_X(x) &= \frac{1}{\sqrt{2\pi\mu}}\exp\left(-\frac{1}{2\mu}(x - \mu)^2 \right)I(x \in \mathbb{R})\\ &= I(x \in \mathbb{R})\frac{\exp(x)}{\sqrt{2\pi}} \exp\left(-\frac{x^2}{2\mu} - \frac{\mu}{2} - \frac{1}{2}\log\mu\right). \end{align*}\]

Yes, this is a member of the one-parameter exponential family with the following components:

\[\begin{align*} h(x) &= I(x \in \mathbb{R})\frac{\exp(x)}{\sqrt{2\pi}}\\ \eta(\mu) &= -\frac{1}{2\mu}\\ T(x) &= x^2\\ \psi(\mu) &= \frac{1}{2}(\mu + \log\mu) \end{align*}\]

In canonical form, let \(\eta = \frac{1}{2\mu}\). Then \(\mu = \frac{1}{2\eta}\).

\[\begin{align*} A(\eta) &= \frac{1}{2}\left(\frac{1}{2\eta} + \log\left(\frac{1}{2\eta}\right)\right) \end{align*}\]

Exercise 3

Consider a distribution \(f_X(x) = \frac{\lambda}{x^{1 + \lambda}}\) for \(\lambda > 0\) and \(x > 1\)). Is this a member of the exponential family? If so, identify the components in canonical form and use the log-partition to calculate the MLE for \(\lambda\) from an i.i.d. sample. If not, explain why not.

\[\begin{align*} f_X(x) &= \lambda x^{-\lambda - 1}I(x > 1)\\ &= \exp(-(\lambda + 1)\log(x) + \log(\lambda))I(x > 1). \end{align*}\]

Yes, this is a member of the one-parameter exponential family with the following components:

\[\begin{align*} h(x) &= I(x > 1)\\ \eta(\lambda) &= -(\lambda + 1)\\ T(x) &= \log(x)\\ \psi(\mu) &= -\log(\lambda) \end{align*}\]

In canonical form, let \(\eta = -(\lambda + 1)\). Then \(\lambda = -\eta - 1\).

\[\begin{align*} A(\eta) &= -\log(-\eta - 1) \end{align*}\]

Finding the MLE, note the following:

\[\begin{align*} \nabla_\eta (A(\eta)) &= \frac{1}{-\eta - 1}\\ &= \frac{1}{\lambda} \end{align*}\]

Thus, \(\frac{1}{n}\sum_{i = 1}^n T(x_i) = \frac{1}{n}\sum_{i = 1}^n \log(x_i)\) is the MLE for \(\nabla_\eta (A(\eta)) = \frac{1}{\lambda}\). Hence, \(\frac{n}{\sum_{i = 1}^n \log(x_i)}\) is the MLE for \(\lambda\).