STA 205           Conditional Expectations                 R Wolpert

Let $(\Omega,\cF,\P)$ be a probability space, and let's consider some random
variable $X: \Omega\to\bbR$, some event $F\in\cF$, and a sub-sigma algebra
$\cG\subset\cF$.

The sigma-algebra $\cG$ represents INFORMATION--- to "know" $\cG$ is to know
whether or not each event $G \in \cG$ occured, and to know the value of every
$\cG$-measurable random variable $Y$.  In many application areas (finance,
for example, or medicine, or statistics with successive sampling, or ...) we
learn things over time, so what we "know" depends on time and grows--- at any
time $t$ we have some knowledge $\cG_t$, and for $s<t$ it has to satisfy
$\cG_s \subset \cG_t$ (i.e. we never forget).

A basic problem in every application area is to make PREDICTIONS of
quantities unobservable until some future time, or to make ESTIMATES of
something otherwise unobservable to us--- i.e., based on our available
information $\cG$, try to guess the value of some random variable $X$ that is
NOT $\cG$ measurable (and hence its value is NOT known to us for certain), or
to guess whether or not some event $F\notin\cG$ occurs (or will occur).
Actually "guessing the event" is a special case of "guessing the random
variable", the special case of an indicator variable $X = 1_F$.

Our guess $Y$ of the random variable $X$ has to be known to us, i.e., has to
be $\cG$-measurable, and in some sense has to approximate $X$.  Let's make
the notion precise:

DEFINITION

   A CONDITIONAL EXPECTATION $E[X|\cG]$ of an integrable random variable
   $X\in L^1(\Omega,\cF,\P)$ is any random variable $Y$ satisfying the two
   conditions:

       1)  $Y$ is $\cG$-measurable (i.e. $Y^{-1}(\cB)\subset\cG$)
       2)  For every $G\in\cG$,  $\int_G (X-Y) dP = 0$.

Today's agenda is to illustrate and motivate this definition.  Here's the
outline:


  *  Uniqueness---  if Y1 and Y2 satisfy 1) and 2), then Y1=Y2 a.s.
  *  Existence---   if X\in L^1 then Y exists (uses Radon/Nikodym thm.
                    Consider X>0 first, then take pos & neg parts)
  *  Familiarity---
     -  Trivial Case 1,  $X \in \cG\\\cB$
     -  Trivial Case 2,  $X \pperp \cG$
     -  Trivial Case 3,  $\cG=\{\emptyset,\Omega\}$ (see [2] above)
     -  Conditional probability P[A | B]
     -  Conditional expectation when $X0,X1,...,Xk ~ f(x)$
        and $\cG=\sigma(X1,...,Xk)$
     -  Orthogonality: Note parallel with orthogonal projection for L^2.

  *  Bayes Formula:

     P[ G | A ] = \int_G P[ A | \cG ] dP  /  \int_\Omega P[ A | \cG ] dP

     P[ G_i | A ] = P[ A | G_i ] P[ G_i ] / \sum_j P[ A | G_j ] P[ G_j ]

  *  Borel Paradox:

     Let X,Y be indep in unit square; find 

                P[ Y > 1/2 | X=Y ]

     Or: Let X,Y be longitude, latitude of point on sphere.  What is the pdf
     of $X\in(-\pi,\pi]$, $Y\in[-\pi/2,\pi/2]$?

          STA 205           Conditional Independence               R Wolpert

Two \sigma-algebras \cE, \cF are CONDITIONALLY INDEPENDENT GIVEN \cG if, for
every E\in\cE and F\in\CF,

       \P[ E \cap F | \cG ]  =  \P[ E | \cG ]   \P[ F | \cG ]

Informally, for someone with knowledge \cG, learning whether or not any event
$E\in\cE$ occured or the value of any $\cE$-measurable random variable does
not affect the (conditional) probability of $F\in\cF$, or the best guess
(conditional expectation) of any $\cF$-measurable random variable $X$.  It
follows that, for $\cF$-measurable $X$,

     \E[ X | \cG \vee \cE ]  =  \E[ X | \cG ]

i.e. the best prediction of $X$ based on both \cG and \cE depends only on
\cG.

DEFINITION:  A MARKOV PROCESS is a stochastic process $X_t$ with the property
that, for every t,

     \sigma{X_s : s < t}      and        \sigma{X_s : s > t} 

are conditionally independent given      \sigma{X_t}.

A MARTINGALE$ is an L^1 stochastic process $X_t$ for which, for any T > t,

     X_t = \E[ X_T | \sigma{X_s: s \le t} ]

(so each "future increment" $[X_T - X_t]$ has conditional expectation zero).