Dependence: Cond'l Dist's

::::::::::::::::::: DISCRETE ::::::::::::::::::::
                                P[ X=x, Y=y ]       f(x, y)
   f(y|x) = P[ Y=y | X=x ] = ------------------ = ----------
                                  P[ X=x ]          f_X(x)

Reversing this:

   f(x, y) = f_X(x) * f(y|x)   = f_Y(y) * f(x|y)

As a distribution:

   E[ g(Y) | x ] = sum_y  g(y) f(y|x)
                 = best guess of g(Y), *if* you know X=x

If X, Y are indep, then = E[ g(Y) ];  otherwise maybe not.  BUT:

  E {   E[ g(Y) | X ]   }  = sum_x [  sum_y  g(y) f(y|x) ]  f_X(x)
                           = sum_x sum_y g(y)  f(x,y)
                           = sum_y g(y)  f_Y(y)
                           = E[ g(Y) ]

Sometimes it's easier to compute expectations this way...

For an *event* A, one can also compute

   E[ g(Y) | A ] = sum_y  g(y)  P[ Y=y | A ]
                 = sum_y  g(y)  P[ Y=y , A ]/ P[ A ]
--------

Let Y = # H's in 3 tosses  A = [ At least one head ]

   P[ Y=y | A ] = (3:y)/8 1_A / (7/8)
                = (0, 3, 3, 1)/7

     E[ Y | A ] = ( 3*1 + 3*2 + 1*3 )/7 = 12/7 = 1 5/7

     E[ Y |!A ] = 0;

     E[ Y ] = (7/8) * (12/7) + (1/8) * 0 = 12/8 = 1.5

--------

Let X ~ Ge(p), Y ~ Bi(X, 1/2).  What's P[ Y = 3 ] ?  E [ Y ] ?

     P[ Y = 3 | X = x ] = 0                         if x <  3
                        = (x:3) (1/2)^3 (1/2)^(x-3) if x >= 3

  => P[ Y = 3 ] = sum_{x=3}^{x=oo} p q^x (x:3) (1/2)^x
                = p sum_{x=3}^{x=oo} (x:3) (q/2)^x

     E[ Y | X = x ]  = x/2
  => E[ Y ] = E[x]/2 = p/2q

----------

:::::::::::::::::::CONTINUOUS::::::::::::::::::::

Y ~ Un(0, theta),  X|Y ~ Un(0,Y)

    f(x,y) = th/y             0 < x < y < th
             0                x<0 or y<x or th<y

    f_X(x) = th log(th/x)     0 < x < th
    f_Y(y) = 1/th             0 < y < th

    f(x|y) = 1/y              0 < x < y < th
    f(y|x) = 1/[y log(th/x)]  0 < x < y < th

    E[Y|X] = (th-x)/log(th/x) 0 < x < th
    E[X|Y] = y/2              0 < y < th


Y ~ Ga(2, lambda), X|Y ~ Un(0,Y)

    f(x,y) = lam^2 exp(-lam y)   0 < x < y < oo

    f_X(x) = lam   exp(-lam x)   0 < x < oo

    f(x|y) = 1/y                 0 < x < y < oo
    f(y|x) = lam exp(-lam[x-y])  0 < x < y < oo

    E[X|Y] = Y/2
    E[Y|X] = X + 1/lam


------ Independence -------

------ Covariance ------

------ Correlation ------

     E[ (X-mu.x) (Y-mu.y) ] =  E[ X Y ] - E[ X ] E[ Y ]

     Var[ X-Y ] = Var[ X ]  +  Var[ Y ] - 2 Cov[ X,Y ]  >= 0

 ->  Cov[ X,Y ] <= (1/2) { Var[ X ] + Var[ Y ] }






Example:  X,Y ~ iid Un(0,1), R = Y/X

                                         1/r          1/r
F(x,r) =        --------------     --------------    --------------
               |       |      |   |       | /    |  |  :    |      |
               |       |     /|r  |       |/     |  |  :    |      |
               |       |  /   |   |       X      |  | :     |      |
               |       X      |   |      /|      |  | :     |      |
               |    /  |      |   |     / |      |  |:      |      |
               |  /    |      |   |   /   |      |  |:      |      |
               |/      |      |   | /     |      |  |.      |      |
                --------------     --------------    --------------.
                       x                  x                 x

F(x,r) =            r x^2/2            r x^2/2          x - 1/2r

 ==>  f(x,r) = x, 0 < x < 1, 0 < r < 1/x   ( or 0 < r x < 1 )

                / 1/2       0 < r < 1
 ==>  f_R(r) = {
                \ 1/2r^2    1 < r < oo


                  / 2x         0 < x < 1,    0 < r < 1
 ==>  f(x | r) = <
                  \ 2x r^2     0 < x < 1/r,  1 < r < oo

                  =  0       -oo < c < 0,    0 < r < oo
                  =  c^2       0 < c < 1,    0 < r < 1
  P[ X < c | r ]  =  c^2 r^2   0 < c < 1/r,  1 < r < oo
                  =  1       1/r < c < oo,   1 < r < oo
                  =  undef   -oo < c < oo, -oo < r < 0

For any r<1 (and in particular as r->0), the conditional
distribution of X given R is Be(2,1), with mean 2/3.

------------------------------------------------------------------

Example:  X,Y ~ iid Un(0,1)


F(x,y) =        -------------- 
               |       |      |
               |       |      |
             y |-------+------|
               |       |      |
               |       |      |
               |       |      |
               |       |      |
                -------------- 
                       x

F(x,y) =            x y        0 < x < 1,  0 < y < 1

 ==>  f(x,y) = 1               0 < x < 1,  0 < y < 1

                / 1            0 < y < 1
 ==>  f_Y(y) = {
                \ 0            otherwise


                  / 1          0 < x < 1,    0 < y < 1
 ==>  f(x | y) = <
                  \ 0        -oo < x < 0  or 1 < x < oo

                  =  0       -oo < c < 0,    0 < y < 1
  P[ X < c | y ]  =  c         0 < c < 1,    0 < y < 1
                  =  0         1 < c < oo,   0 < y < 1
                  =  undef   -oo < c < oo,   y < 0 or y > 1

For any 0 < y < 1 (and in particular as y->0), the conditional
distribution of X given Y is Un(0,1), with mean 1/2.