Statistics & Gene Expression Data Analysis Note 11: Multi-Normal Theory

♠ Density and Parameters

• x is p × 1 with mean vector m (p × 1) and variance (covariance) matrix V (p × p)

• Precision (or concentration) matrix K = V^{−1} (assume non-singular)

• Density function

p(x) = c exp(−(x − m)′K(x − m)/2)

with c = |2π|^{p/2}|K|^{1/2}

• x ∼ N(m, V) or x ∼ N(x|m, V)

♠ Linear Transforms

• Any k × p matrix G and constant k−vector a, y = a + Gx is normal y ∼ N(a + Gm, GVG′)

• k < p : Dimension reduction

• k > p : Rank deficient (singular) distribution

♠ Key Properties: Marginal & Conditional Distributions

Partition x as x_{1} and x_{2} and conformably partition m and V so that

x = ( \begin{matrix}x & 1 \\ x & 2 \end{matrix} ) , m = ( \begin{matrix}m & 1 \\ m & 2 \end{matrix} ) & V = ( \begin{matrix}V_{1} & R \\ R′ & V_{2} \end{matrix} )

where C(x_{1}, x_{2}) = R (and of course C(x_{2}, x_{1}) = R′.) Dimensions are conformable – any subsetting of x works

• x_{1} ∼ N(m_{1}, V_{1}) and x_{2} ∼ N(m_{2}, V_{2})

• Really critical to understanding regression are the conditional distributions: Here is p(x_{1}|x_{2}) and the same general theory tells you what p(x_{2}|x_{1}) is

(x_{1}|x_{2}) ∼ N(a_{1} + B_{1}x_{2}, W_{1})

with

a_{1} = m_{1} − B_{1}m_{2}, B_{1} = RV_{2}^{−1} & W_{1} = V_{1} − B_{1}R′

♠ Precision Matrix and Dependencies

Take x_{1} = x_{1}, the first element of x so that x_{2} is all the rest. Another way of writing the conditional distribution above is in terms of the elements of the precision matrix K instead of V as follows (this is just based on standard linear algebra and representations of inverses of partitioned matrices).

• If x_{1} = x_{1}, then B_{1} is the (p − 1) row vector with j^{th} element

b_{1,j} = −K_{1,j}/K_{1,1}

and W_{1} is the scalar variance 1/K_{1,1}

• Shows the linear regression of x_{1} (or any other x_{i}) on all other variables (genes)

• Note: Zeros in precision matrices corresponding to conditional independencies

• Underlies the major area of Gaussian graphical models ♠ Singular Normal

• V is singular; distribution is singular

• rank deficient: rank(V) = k < p – for some k × p matrix G, y = Gx has a non-singular distribution: variance matrix GVG′ is non-singular.

• constrained linear combinations of p − k elements of x – only k “free” dimensions

• density still has same form in terms of K where now K = V^{−} is a generalised inverse of V (i.e., such that KVK = K and VKV = V