Pitman STA 230 / MTH 230 Probability Week 9 Independence and Joint Distributions ========== 1. Joint Distribution Functions F(x,y) = P[ X <= x && Y <= y ] = P[ (X,Y) lies south-west of the point (x,y) ] : 3 | /__________|________________________.(x,y) \ 2 | | | | 1 | | | | 0 |__,____,____,____,____,_|__,____,____,____,____,____,___ 0 1 2 3 4 | 5 6 7 8 9 10 | | \|/ V From this we can get: Fx(x) = P[ X<=x ] = P[ X<=x && Y<=oo ] = F(x,oo) <- X "Marginal" DF Fy(y) = P[ Y<=y ] = P[ X<=oo && Y<=y ] = F(oo,y) <- Y "Marginal" DF For CONTINUOUS jointly-distributed random variables: 2 d f(x,y) = --------- F(x,y) = 2nd partial derivative (once x , once y) dx dy = "joint density function"; use it just like ordinary density functions, but now in 2-dimensions; e.g. : oo oo / / : / / P[X in A, Y in B] = | | f(x,y) dx dy : E[g(X,Y)] = | | g(x,y) f(x,y) dx dy / / : / / A B : -oo -oo for DISCRETE jointly-distributed random variables: F(x,y) = SUM P[X=s && Y=t] <--- JOINT PROBABILITY MASS FUNCTION s<=x, t<=y P[X in A, Y in B] = SUM ( p(x,y) : x in A, y in B ) E[ g(X,Y) ] = SUM ( g(x,y) p(x,y) : all x,y ) EXAMPLES: A drawer has 6 socks: 2 red, 2 black, 2 white. Reach in and draw out a PAIR of socks; let R be # of red socks, B = # of black socks drawn. What is the probability mass function, p(i,j) = P(R=i, B=j)??? (HINT: What are the possible values of R, B?) / i j p Socks | 0 0 1/15 WW | 0 1 4/15 WB or BW p(i,j) = P(R=i, B=j) = < 0 2 1/15 BB | 1 0 4/15 WR or RW | 1 1 4/15 BR or RB \ 2 0 1/15 RR /2\ /2\ / 2 \ | | | | | | \i/ \j/ \2-i-j/ Note: COULD use "multinomial": p(i,j) = ------------------------- /6\ | | \2/ but certainly don't have to....... Example 2: Let X and Y be drawn uniformly from the triangle 3 | | | 2 |\ 0 < y < 2, 0 < x < 2-y | \ | \ 1 | \ | \ | \ 0 |__,____,____\____, 0 1 2 3 Then what is the joint distribution function for x, y? What is the (marginal) distribution for x? What is the joint density function? 2. Independent Random Variables What does it MEAN for two random variable X,Y to be INDEPENDENT??? ANSW: P[ X in A && Y in B ] = P[ X in A ] * P[ Y in B ] for EVERY sets A, B; for discrete and continuous RV's, DISCRETE: Must check prob mass func satisfies: p(x,y) = p_x(x) p_y(y) CONTINUOUS: Must check pdf satisfies: f(x,y) = f_x(x) f_y(y) (for EVERY x,y; pay special attention to sets on which f(x,y) or p(x,y) are non-zero; must be rectangles) 3. Sums of Independent Random Variables What's the probability that Z=X+Y is less than z??? The pairs (X,Y) that lead to Z= Y ] c) f_X(x), f_Y(y) EG(2): Let theta ~ Un(0, 2 pi); set X = cos(theta), Y = sin(theta). Find: a) P[ X+Y > 1 ] [ note joint pdf doesn't exist ] b) P[ Y > 1/2 ] Bored? Now let theta ~ Ex(1) and answer the same questions EG(3): Let Y be the rate of calls at a help desk, and X the number of calls between 2pm and 4pm one day; let's say that: x (2y) f(x,y) = --------- exp(-3 y), y>0, x=0,1,2,... x ! Find: a) P[ X = 0 ] b) P[ Y > 2 ] c) P[ X = x ] for all x EG(4): Consider a fishing experiment where we catch lambda fish/hr; let S, T be the times of first, 2nd fish-catch events. Find: a) CDF F(s, t) b) pdf f(s, t) c) Marginal pdf's f_S(s), f_T(t) d) P[ T > S + 2 ] [ what's this tell you about T-S ? ] EG(5): Let (X,Y) be coords of a point drawn uniformly from the unit square; fix numbers eps and c in unit interval, 0 < eps, c < 1. Find: a) pdfs f(x,y), f_X(x), f_Y(y) b) P[ X <= c ] c) P[ X <= c | Y <= eps ] d) P[ X <= c | Y <= eps * X ] e) Set A = {(x,0): -oo < x < oo }. Find P[ X <= c | (X,Y) \in A ] [trick question!] EG(6): In EG(5), set R = Y/X. Find: [ Borel's Paradox ] a) pdf f(r) b) joint pdf f(x, r) ------- Foreshadow -------- P[ X <= c | R=0 ] != P[ X <= c | Y=0 ] --------------------------------------------------------------------------- EG(7): If X and Y are independent and exponentially distributed, with P[X>t] = P[Y>t] = exp(-t) for t>0, then what is the density function for Z=X+Y??? Solution: First find the CDF: oo z / / / -(z-x) \ -x F(z) = | F_y(z-x) f_x(x) dx = | | 1-e | e dx / / \ / -oo 0 -z -z = 1 - e - z e (WHY is the upper integration limit z?) -z -z -z f(z) = F'(z) = e + z e - e (product rule) -z = z e , for z>0 (density is zero for z<0). ========= 2nd sol'n ========= Convolution formula: f(z) = Int f_x(z-y) f_y(y) dy = Int { lam exp[-lam (z-y)] lam exp[-lam y ] } dy = Int { lam^2 exp[ - lam z] } dy = ??? Note LIMITS OF INTEGRATION: 0 0 < y < z = lam^2 z exp[-lam z], z > 0 4. Conditional Distributions: Discrete Case Simply apply the definition of conditional probability: P[ X=x && Y=y ] p(x,y) P(x|y) = P[ X=x | Y=y ] = ----------------- = ---------- P[ Y=y ] p_y(y) So, all you need are the joint probability mass function (pmf) p(x,y) and the marginal pmf for Y. Why is it called "Marginal"??? (draw a table for p(x,y), put row and column sums in the margins) p_y(y) = SUM [ p(x,y) : all x ] Let N, Q, D be the numbers of nickles, quarters and dollars in a pair of coins drawn (without replacement) from a bag containing $2 of each coin; find the conditional distributions, the conditional expectation of Q, given D=1, and the conditional expectation of the value (in cents) of the draw, given D=1. How much more is this than the conditional expectation of the value of the draw, given Q=1? Do you expect it to be 75 cents more? Why? (NOTE: Only need p(q,d) and p_d(d) for d=1) p(q,d) ???? p(q | d) = -------- = ---------- (work it out!) p_d(d) ???? E[Q | D=1] = SUM q p(q | d) = ??? " " 5. Conditional Distributions: Continuous Case P[X=x,Y=y] p(x,y) Discrete pmf: p(x|y) = Pr[X=x | Y=y] = -------------- = -------- P[Y=y] p2(y) f(x,y) Continuous pdf: f(x|y) = -------- (think about tiny square) f (y) | 2 | _ P[X in A | Y in B]: B |_| | ______|____________ ______________ | A Example: f(x,y) = (1/y) exp(-y), 0 < x < y < oo ==> f(y|x) = ?? f(x|y) = ?? (BE CAREFUL about LIMITS) 6. Order Statistics X1...Xn independent; important case: X(1) = smallest, X(n) = largest Pr[X(n) <= x] = Pr[ NONE are > x ] = F(x)^n Pr[X(n-1) <= x] = Pr[ at most 1 is > x ] = C(n:0) F(x)^n + C(n:1) F(x)^(n-1) (1-F(x))^1 Pr[X(k) <= x] = Pr[ at most n-k are > x ] = SUM { C(n:i) F(x)^(n-i) (1-F(x))^i : i = 0..n-k } Pr[X(1) <= x] = Pr[ at least one is <= x ] = 1 - (1-F(x))^n Take derivatives to get joint pdf; turns out to be k-1 n-k fk(x) = C(n:k) F(x) [1-F(x)] f(x)/k <-- pdf for X(k) (k'th smallest) n-1 n-1 f1(x) = n [1-F(x)] f(x) fn(x) = n F(x) f(x) ` smallest ' ` largest ' Drop 3 quarters (diam: 1") into a tin can (diam:4"). What's the pdf for the closest distance from a coin to the rim? What did you assume? (Harder question: distribution of closest distance to CENTER of can) 7. Joint Probability Distribution Functions of Random Variables Functions of random variables: Jacobeans.