Statistics & Gene Expression Data Analysis Note 8: Binary Regression

♠ Outcomes and classification probabilities

• Expression level vector x_{j} on array (tumor sample, etc) j

• Binary outcome: z_{j} = 0 or 1 – codes a clinical or physiological endpoint, state or outcome

– e.g., primary breast cancer ER+ (z_{j} = 1) versus ER− (z_{j} = 0)

• Probability model estimates π_{j} = P r(z_{j} = 1) based on available information and data

– Fit a model to estimate the π_{j} in training sample j = 1, …, n classification, discrimination of cases in training sample

– Evaluate/estimate π_{j} for validation cases j = n + 1, n + 2, … predictive classification, validation, prognosis

♠ Binary regression models

• Linear regression model based on regression functions μ_{j} = x′_{j}β – linear combinations, linear scoring, of expression levels of genes

– p−vector of regression parameters β, one for each gene

(plus intercept term β_{0})

• Idea: model π_{j} as a function of x_{j} in a similar fashion – But 0 < π_{j} < 1 for probability, and μ_{j} is real-valued

– Need truncation or transformation

– Standard statistical models transform from real-value to (0, 1) using a specified non-linear function:

mapping μ_{j} to π_{j}

• Logistic regression:

π_{j} = 1/(1 + exp(−μ_{j}))

• Probit regression:

π_{j} = Φ(μ_{j})

where Φ is standard normal cumulative distribution function

• Others ... all similar in form (any continuous distribution function does the trick)

♠ Probit models

• One nice, and important, interpretation: Latent threshold for 0/1

• Multiple regression outcome y_{j} = x′_{j}β + ϵ_{j} on array j i.e., y_{j} = μ_{j} + ϵ_{j}

• with a standard Gaussian (or normal) error term ϵ_{j} ∼ N(0, 1)

• y_{j} is latent – i.e., not observed, unknown, hidden

• The probability that y_{j} is positive is π_{j} = Φ(μ_{j})

• A “hidden” underlying threshold mechanism in which a (weighted, super-gene measure of) expression levels determine the probability of outcome

• Also, z_{j} = 1 if and only if y_{j} > 0 – latent variable is positive for ER+ cases, negative for ER− cases

– could precisely classify cases if we could observe the latent y_{j}, but we do not; result is the binary probability model ♠ SVD regression and Bayesian analysis

• Dimension problem: p = 000’s of genes, n =few microarrays. Ill-posed estimation problem – many more variables than data points. May use SVD regression ideas, to map to factor regression x′_{j}β = f′_{j}θ (see Note 6) using the SVD analysis of expression data matrix X

• As in Note 6, θ = B′β or θ = DA′β

• Multiple regression on the factor variables themselves as predictors

• n predictor variables, not p

• Regression parameter vector θ to estimate

• Dimension reduction of inference/estimation problem when p > n, as is the case in gene expression analyses

• Formal inference and prediction can be based on Bayesian analysis and its implicit stochastic regulari-

sation of the estimation problem – remove some of the “least variable” factors

– apply Bayesian analysis to the rest

• Corresponding estimation of β via β = AD^{−1}θ

♠ Software, Computation and Summary

• Point estimate analysis: iterative computation of estimates of θ that are Bayesian posterior modes (EM

algorithms, MAP estimation)

– Choose a subset of genes to use in X

– e.g., screen genes to choose the “top 100” in terms of raw sample correlation with ER or other binary outcome

– Fit model on this reduced subset, using SVD regression

– Point estimates of θ and corresponding β

• Full Bayesian analysis using stochastic simulation methods (Markov chain Monte Carlo simulation,

Gibbs sampling)

– iterative computation of simulation samples of values of θ whose distribution can be summarised to represent the information in the data about θ in terms of point estimates (the average sample value, for example), and probability intervals (taking fractiles/percentiles of the sample values). Map these values to corresponding values of β to summarise too – effects of individual genes on analysis

• Estimated or fitted classification probabilities in sample: π_{j} = Φ(μ_{j}) with μ_{j} = x′_{j}β – point estimate β̂ implies estimate μ̂_{j} and hence estimates of probabilities π̂_{j} = Φ(μ̂_{j})

– simulation samples of values of β imply corresponding simulation samples of values of μ_{j} and hence of the probabilities π_{j}. Summarise by averages, percentiles for interval estimates, etc

♠ Cross-validation and Prediction

• Include “validation samples” to be predicted to assess the realistic utility of the model in “forecasting” the probabilities of 0/1 outcomes for new cases: inference on π_{j} for j = n + 1, n + 2, ... etc

• Also useful to explore One-at-a-time cross-validation studies, also known as Hold-one-out studies – Hold out sample i = 1, and fit model to cases 2, …, n

– Estimate/infer π_{1} to to assess how well case 1 is predicted based on the others

– Repeat for case i = 2, then i = 3, and so forth

• Formal and “honest” assessment of model fit and adequacy

• Identifies “interesting” cases, those that fit least well

• Reflects real-life context of application of models

• n.b., if screening genes to select a subset, must do so separately in each CV analysis