Statistics & Gene Expression Data Analysis Note 2: Linear Regression

♠ Linear regression model: Relating two genes

• Straight line regression model:

(dependent variable) response gene y (e.g., ER)

(independent variable, explanatory variable) predictor gene x (e.g., ps2)

• Measurement error model: repeat values i = 1, …, n, - independent expression levels on n tumors

y_{i} = α + βx_{i} + ϵ_{i}

• ϵ_{i} : independent errors (sampling, measurement, lack of fit)

• Model “explains” variability in response y “due to” x

• Bivariate data (y_{i}, x_{i}) BUT focus is asymmetric: explaining y through x

• Non-causal, purely empirical

• Predictive validity: fit model and test in new cases

• Typical assumption: Gaussian (normally) distributed errors ϵ ∼ N(0, σ^{2})

• Analysis and inference:

- Estimate parameters (α, β, σ^{2})

- Assess model fit — adequate? good? if inadequate, how?

- Explore implications: β, βx

- Predict new (“future”) responses at new x_{n+1}, …

♠ Linear regression model: Least squares fitting

• For any chosen α, β,

Q(α, β) = ∑_{i=1}^{n} ϵ_{i}^{2} = ∑_{i=1}^{n}(y_{i} − α − βx_{i})^{2}

measures “fit” of chosen line α + βx to response data

• Choose α̂, β̂ to minimise Q(α, β)

• Least squares estimates (LSE)

• Fitted least squares line: ŷ = α̂ + β̂x

♠ LSE formulæ and interpretation:

• Sample variances and covariances s_{x}, s_{y}, s_{x,y} •

β̂ = \frac{s_{x,y}}{s_{x}} , α̂ = ȳ − β̂x̄

• Or

β̂ = r_{x,y}\sqrt{\frac{s_{y}}{s_{x}}}

• β̂ is correlation coefficient corrected for relative scales of y : x - (so units of the “fitted line” α̂ + β̂x are on scale of y)

• Same variability: s_{y} = s_{x} implies β̂ = r_{x,y}

♠ Significance of fit, residuals, prediction

• See the more general framework of multiple regression models, in Note 3. The model here is a special case.

Mike West  June 2, 2004  Page 0