Agenda
Agenda
Probability
Conditional probability
Bayes' theorem and simple calculations
Probability
Definition
What does it mean to say that:
The probability of flipping a coin and getting heads is 0.5?
The probability of rolling snake eyes, two ones on two dice, is 1/36?
The probability Apple's stock price goes up today is 0.37?
Interpretations:
Frequency: If you can repeat an experiment indefinitely, the proportion of the experiments where the event occurred.
Belief: If you are indifferent between winning $1 if the event occurs or winning $1 if you draw a blue chip from a box with \(1,000,000 \times p\) blue chips, \(1,000,000 \times (1-p)\) red.
Data Example
Calculating Probabilities
For simple probabilities (equally likely outcomes), calculating probabilities is really just about counting.
\[ P(\text{event E}) = \frac{\text{# of outcomes where E occurs}}{\text{# of possible outcomes}} \]
So if I asked based on this data what is the probability that a respondent had blue eyes?
# of outcomes where E occurs - 1050 respondents have blue eyes
# of possible outcomes - 1770 total respondents
\[P(\text{Blue eyes}) = 1050 \,/\, 1770 = 0.593\]
Joint Probabilities
These are probabilities that involve multiple events that all must occur.
What is the probability that a respondent has blue eyes and black hair?
# of outcomes where E occurs - 200 respondents had blue eyes and black hair
# of possible outcomes - 1770 total respondents
\[P(\text{Blue eyes and black hair}) = 200 \,/\, 1770 = 0.113\]
Adding Probabilities
What is the probability that a respondent has black hair or brown hair?
\[ \begin{aligned} P(\text{Black hair or brown hair}) &= P(\text{Black hair}) + P(\text{brown hair}) \\ &= 400/1770 + 300/1770 \\ &= 0.395 \end{aligned} \]
What is the probability that a respondent has black hair or blue eyes?
\[ P(\text{Black hair or blue eyes}) \ne P(\text{Black hair}) + P(\text{blue eyes}) \]
Why does this work in the first case and not the second case?
Mutually Exclusive Events
Events are mutually exclusive when \(E\) occurring means that \(F\) cannot possibly happen (and vice versa).
Is having Black hair and having brown hair mutually exclusive?
Is having Black hair and having blue eyes mutually exclusive?
Basic Rules of Probability
All probabilities must be between 0 and 1
The sum of the probabilities for all possible outcomes must be 1
Events are mutually exclusive (disjoint) when \(P(E\text{ or }F) = P(E) + P(F)\)\
General addition rule, For non-disjoint events \(P(E\text{ or }F) = P(E) + P(F) - P(E\text{ and }F)\)\
Complement Rule, \(P(\text{not }A) = P(A^c) = 1-P(A)\)
Conditional Probability
Conditional Probability
Often we want to calculate the probability of an event occurring given some other information.
\[ \begin{aligned} P(A~|~B) &= \frac{\# \, A\text{ and }B}{\# \, B} \\ &= \frac{\# \, A\text{ and }B ~/~ \# \, \Omega}{\# \, B ~/~ \# \, \Omega} \\ &= \frac{P(A\text{ and }B)}{P(B)} \end{aligned} \]
Conditioning allows us to limit the scope of the outcomes we are examining and tells us about the relationship between events.
Example
What is the probability that a respondent has blue eyes given they have red hair?
\[ \begin{aligned} P(\text{blue eyes } | \text{ red hair}) &= \frac{\# \, \text{blue eyes and red hair}}{\# \, \text{red hair}} \\ &= 50/70 = 0.714 \\ P(\text{blue eyes } | \text{ red hair}) &= \frac{P(\text{blue eyes and red hair})}{P(\text{red hair})} \\ &= \frac{50/1770}{70/1770} = 0.714 \\ \end{aligned} \]
Independence
Not to be confused with being mutually exclusive - events are said to be independent when if E occurs it tells you nothing about F happening.
Mathematically, we define independence as
\[P(A|B) = P(A) \text{ or } P(B|A) = P(B)\]
Multiplication Rule
Using the definition of conditional probability we can arrive at an alternative definition of joint probabilities
\[ P(A \text{ and } B) = P(A~|~B)~P(B) = P(B~|~A)~P(A) \]
This also gives us another definition of independence, \(A\) and \(B\) are indepedent if
\[P(A \text{ and } B) = P(A)P(B)\]
Example
Is having red hair and blue eyes independent of each other?
\[ \begin{aligned} P(\text{red hair and blue eyes}) &\stackrel{?}{=} P(\text{red hair})~P(\text{blue eyes}) \\ 50/1770 &\stackrel{?}{=} 70/1770 ~ 1050/1770 \\ 0.0282 &\ne 0.0235 \end{aligned} \]
Disjoint vs. independent events
Disjoint (mutually exclusive) events are events that cannot happen at the same time
\[P(A~and~B) = 0\]
Independent events are events that do not share any information
\[ \begin{aligned} P(A~|~B) &= P(A) \text{ or} \\ P(B~|~A) &= P(B) \text{ or} \\ P(A~and~B) &= P(A) \times P(B) \end{aligned} \]
Example
Are brown and black hair disjoint?
Are brown and black hair independent?
Are brown eyes and red hair disjoint?
Are brown eyes and red hair independent?
Law of total probability
If we divide our sample space up into \(n\) mutually exclusive events where \(\sum_{i=1}^n P(A_i) = 1\) then we can find \(P(B)\) using conditional and/or joint probabilities.
\[ \begin{aligned} P(B) &= P(A_1 \text{ and } B) + P(A_2 \text{ and } B) +P(A_3 \text{ and } B) +P(A_4 \text{ and } B) +P(A_5 \text{ and } B) \\ &= P(B | A_1)~P(A_1) + P(B | A_2)~P(A_2) + P(B | A_3)~P(A_3) + P(B | A_4)~P(A_4) + P(B | A_5)~P(A_5) \end{aligned} \]
Hiking Example
Whether or not I go hiking depends on the weather, if it is sunny there is a 60% chance I will go for a hike, while there is only a 10% chance if it is raining and a 30% chance if it is snowing.
The weather forecast for tomorrow calls for 50% chance of sunshine, 40% chance of rain, and a 10% chance of snow.
What is the probability I go for a hike tomorrow?
Bayes Theorem
In some cases we know one conditional probability \(P(A|B)\) but what we want to know is the flipped conditional: \(P(B|A)\).
We can achieve this using Bayes' theorem
\[ P(B~|~A) = \frac{P(B \text{ and } A)}{P(A)} = \frac{P(A ~|~ B)~P(B)}{P(A)} \]
We can take this one step farther using the law of total probability to get:
\[ P(B|A) = \frac{P(A ~|~ B)~P(B)}{P(A)} = \frac{P(A ~|~ B)~P(B)}{P(A|B)~P(B) + P(A|B^c)~P(B^c)} \]
Spam
You have 100 emails in your inbox: 60 are spam, 40 are not. Of the 60 spam emails, 35 contain the word "free". Of the rest, 3 contain the word "free".
spam_tree
If an email contains the word "free", what is the probability that it is spam?
Example - House
If you've ever watched the TV show House, you know that Dr. House regularly states, "It's never lupus."
Lupus is a medical phenomenon where antibodies that are supposed to attack foreign cells to prevent infections instead see plasma proteins as foreign bodies, leading to a high risk of blood clotting. It is believed that 2% of the population suffer from this disease.
The test for lupus is very accurate if the person actually has lupus, however is very inaccurate if the person does not. More specifically, the test is 98% accurate if a person actually has the disease. The test is 74% accurate if a person does not have the disease.
Is Dr. House likely to be correct even if his patient tests positive for Lupus?