Conditional Probability and Independence ========== First, what IS probability??? 0. Symmetry? Try to find equally-likely outcomes. 1. Aymptotic Frequency: P(E) = lim_{n->oo} (#E in n tries)/n 2. Degrees of belief: P(E) = fraction of brass rings needed for betting on event or hat to be equally attractive ========== Most important idea in probability: Probability of A: P(A) Probability of A, GIVEN B: P(A|B) = P(AB)/P(B) Example 1: If I draw a heart first, what's the chance 2nd card is also a heart? ANS 1: 12/51 = 4/17 (Think about the deck after 1st draw) ANS 2: P(AB) = (13:2) / (52:2) = (13*12)/(52*51) = 1/17 P(B) = (13/52) = 1/4 P(A | B) = P(AB) / P(B) = (1/17)/(1/4) = 4/17 Example 2: If I flip Heads first, what's the chance 2nd flip is also Heads? ANS: Assuming equally-likely outcomes of {HH, HT, TH, TT}, P(AB) = P({HH}) = 1/4 P(B) = P({HH, HT}) = 2/4 P(A | B) = P(AB) / P(B) = (1/4)/(2/4) = 1/2 NOTICE: In Example 1, P(A) = 1/4 P(A|B) = 4/17 => B *changes* prob of A In Example 2, P(A) = 1/2 P(A|B) = 1/2 => B *does not change* P(A) DEFINITION: A and B are called INDEPENDENT if P(A|B) = P(A) \ P(AB) = P(A) P(B) >-- All three are equivalent P(B|A) = P(B) / ********** Common Use: Multiplying through by P(A), we see: P(AB) = P(A) * P(B|A) (***) IF A and B are independent then P(B|A) = P(B), but formula (***) is correct for ANY events A and B, even if they are not indep... in fact, it's often the easiest way to compute P(AB): Example 1 (cont): What is the probability I draw 2 hearts in a row? ANS 1: P(H1) * P(H2|H1) = (13/52) * (12/51) = (1/4)*(4/17) = 1/17 ANS 2: (13:2)/(52:2) = (13*12)/(52*51) = 1/17 Which is easier? How about if we keep going.... What is the probability of a flush in Hearts (5 in a row)? ANS 1: (13/52) * (12/51) * (11/50) * (10/49) * (9/48) = 33/66640 = .0004952... ANS 2: (13:5)/(52:5) = ... Example 3: For grad & prof students Duke holds a lottery; each student has about a 1/3 chance of buying season basketball tickets each year. What's the chance a student here for 4 years gets tickets at least once? ANS 1: 4 * (1/3) = 4/3 = 1.333; it's a certainty! (Hmmmmmmmmmm.) ANS 2: ????? --- BAYES FORMULA (aka Bayes Rule, Bayes Theorem, ...) Idea: Use what you observe to help you guess about what you need. Basis of one school of statistics. Example: A new headache treatment either helps 50% of subjects, just like the old treatment, or it helps 75% of the subjects as its inventor claims. Initially we think these are equally likely: If B is the event "New treatment is better", then have P(B) = 0.50 Three subjects are given the treatment; all three are helped. Call this event (which we've observed) "A", and compute: P(A|B) = (.75)*(.75)*(.75) = 27/64 = 42.1875% P(A|B^c) = (.50)*(.50)*(.50) = 1/8 = 12.50% Now that we observed A, what is the new probability of B???? P(B|A) = P(AB) / P(A) P(AB) = P(B) P(A|B) = (1/2) * (27/64) = 27/128 = 21.09% P(AB^c) = P(B^c) P(A|B^c) = (1/2) * (1/8) = 1/16 = 6.25% P(A) = P(AB) + P(AB^c) = 35/128 = 27.34% P(B|A) = P(AB) / P(A) = (27/128) / (35/128) = (27/35) = 77.12% SO: Before observing any data we had P(B) = 0.50; afterwards, it's P(B|A) = 0.77, much bigger. Q: What would have happened if we'd observed zero of three helped by new treatment? ============ Bayes Formula: P(Fj|E) = P(EFj) / P(E) = P(EFj) / Sum P(EFi) ( Just conditional probability formula plus: P(E) = Sum P(EFi) ) =================================== x ================================= A Few Surprising Examples: 3c: Laboratory blood test: Disease: 95% + 5% - Healthy: 99% + 1% - Prevalence: 1/200 (0.5%) have disease P[ Disease | + ] = ??? (.005 * .95) / (.005 * .95 + .995 * .01) = .00475/(.00475+.00995) = 475/1470 = 95/294 = .323129 = 32.3% Among 1000 persons, expect 5 with disease, 995 without; probably find all 5 cases, plus about 10 false positives, so 5/15 = 0.3 is reasonable. 3h Gold & Silver bars: GG, GS, SS equally likely; P[GG|Gx] = ??? xx Roll a fair die, and define an event Ai by: Ai: Number is evenly divisible by i Then A2 and A3 are independent, but A2 and A5 are not. xx If A,B are independent, what is P(AuB)? If A,B are independent, are A^c and B indep? A^c and B^c?? *** How do you find P(A B C)???? 2 coins: A = {HH, HT} B = {HH, TH} C = {HH, TT}; show A&B, A&C, B&C indep. P(A B C) = ??? DEFINITION: Events {An} are indep if P(Ai Aj ... Ak) = P(Ai)*P(Aj)* ... *P(Ak) for every i,j,...,k If A,B and A,C are indep, are A, BC indep? xx If P(B)=0 or P(B)=1 then A,B are always indep for any A 4j Random Walk (="Gambler's Ruin") Adam begins with $a, bank begins with $b, and a fair coin is tossed again and again; with each H bank gives Adam $1, with each tail Adam gives it back. What is the probability that Adam doesn't go broke? Let W be the event "Adam wins it all". Notice that together Adam and the Bank always have the same total fortune, $a+b=N; always adam has 0<= x <= N. Set: f(x) = P(W | x) = Prob[ Adam breaks the bank | now has $x ] = 0 if x=0 = 1 if x=N = (1/2) f(x-1) + (1/2) f(x+1) if 0 c = (1-r)/(1-r^N) ==> f(x) = (r^x-1)/(r^N-1) = [(q/p)^x - 1] / [(q/p)^N - 1] = [(p/q)^(N-x)-(p/q)^N] / [1 - (p/q)^N] EXAMPLE: In US Roulette, you win with probability p = 18/38 = 9/19, lose with probability q = 1-p = 10/19, so p/q = 0.9 and f(x) = [ (10/9)^x - 1] / [ (10/9)^N - 1] With N = 2 and x = 1, for example, f(1) = [ 10/9 - 1 ] / [ 100/81 - 1 ] = 9/19 = 47.37% With N = 10 and x = 5, for example, f(5) = [ (10/9)^5 - 1 ] / [ (10/9)^10- 1 ] = .6935/1.868 = 0.37126 With n = 100 and x = 50, f(50) = [ (10/9)^50 - 1 ] / [ (10/9)^100- 1 ] = 193.03/37647.6 = 0.00513 virtually certain failure. MORAL: In US roulette, big bets are better. WHY??: For every dollar you bet, expect to lose 2/38 = 1/19 = $0.0526 NOTE: In "fair" game, with p=.5=q, f(x) = 1-x/N, so all 3 answers would have been f(N/2) = 1/2. For large n and x, the "-1" doesn't change things much from f(x) approx (q/p)^(N-x). Casino can be sure never to go broke if it forbids large bets, since even for a->infinity, (10/9)^a - 1 1 -------------- <= ---------- (10/9)^a+b - 1 (10/9)^b