Dependence: Cond'l Dist's

::::::::::::::::::: DISCRETE ::::::::::::::::::::
                                P[ X=x, Y=y ]       f(x, y)
   f(y|x) = P[ Y=y | X=x ] = ------------------ = ----------
                                  P[ X=x ]          f_X(x)

Reversing this:

   f(x, y) = f_X(x) * f(y|x)   = f_Y(y) * f(x|y)

As a distribution:

   E[ g(Y) | x ] = sum_y  g(y) f(y|x)
                 = best guess of g(Y), *if* you know X=x

If X, Y are indep, then = E[ g(Y) ];  otherwise maybe not.  BUT:

  E {   E[ g(Y) | X ]   }  = sum_x [  sum_y  g(y) f(y|x) ]  f_X(x)
                           = sum_x sum_y g(y)  f(x,y)
                           = sum_y g(y)  f_Y(y)
                           = E[ g(Y) ]

Sometimes it's easier to compute expectations this way...

For an *event* A, one can also compute

   E[ g(Y) | A ] = sum_y  g(y)  P[ Y=y | A ]
                 = sum_y  g(y)  P[ Y=y , A ]/ P[ A ]
--------

Let Y = # H's in 3 tosses  A = [ At least one head ]

   P[ Y=y | A ] = (3:y)/8 1_A / (7/8)
                = (0, 3, 3, 1)/7

     E[ Y | A ] = ( 3*1 + 3*2 + 1*3 )/7 = 12/7 = 1 5/7

     E[ Y |!A ] = 0;

     E[ Y ] = (7/8) * (12/7) + (1/8) * 0 = 12/8 = 1.5

--------

Let X ~ Ge(p), Y ~ Bi(X, 1/2).  What's P[ Y = 3 ] ?  E [ Y ] ?

     P[ Y = 3 | X = x ] = 0                         if x <  3
                        = (x:3) (1/2)^3 (1/2)^(x-3) if x >= 3

  => P[ Y = 3 ] = sum_{x=3}^{x=oo} p q^x (x:3) (1/2)^x
                = p sum_{x=3}^{x=oo} (x:3) (q/2)^x

     E[ Y | X = x ]  = x/2
  => E[ Y ] = E[x]/2 = p/2q

----------

:::::::::::::::::::CONTINUOUS::::::::::::::::::::

X ~ Un(0, theta),  Y|X ~ Un(0,X)
X ~ Ga(2, lambda), Y|X ~ Un(0,X)

------ Independence -------

------ Covariance ------

------ Correlation ------

Example:  X,Y ~ iid Un(0,1), R = Y/X

                                         1/r          1/r
F(x,r) =        --------------     --------------    --------------
               |       |      |   |       | /    |  |  :    |      |
               |       |     /|r  |       |/     |  |  :    |      |
               |       |  /   |   |       X      |  | :     |      |
               |       X      |   |      /|      |  | :     |      |
               |    /  |      |   |     / |      |  |:      |      |
               |  /    |      |   |   /   |      |  |:      |      |
               |/      |      |   | /     |      |  |.      |      |
                --------------     --------------    --------------.
                       x                  x                 x

F(x,r) =            r x^2/2            r x^2/2          x - 1/2r

 ==>  f(x,r) = x, 0 < x < 1, 0 < r < 1/x   ( or 0 < r x < 1 )

                / 1/2       0 < r < 1
 ==>  f_R(r) = {
                \ 1/2r^2    1 < r < oo


                  / 2x         0 < x < 1,   0 < r < 1
 ==>  f(x | r) = <
                  \ 2x r^2     0 < x < 1/r, 1 < r < oo

  P[ X < c | r ]  =  c^2       0 < c < 1,   0 < r < 1
                  =  c^2 r^2   0 < c < 1/r, 1 < r < oo
                  =  1       1/r < c < 1,   1 < r < oo

For any r<1 (and in particular as r->0), the conditional
distribution of X given R is Be(2,1), with mean 2/3.