1、Lecture 4 : Conditional Probability and Bayes Theorem0/ 261/ 26The conditional sample space Motivating examples1. Roll a fair die onceS =1 2 34 5 6Let A = 6 appearsB = an even number appearsSoP(A) = 16P(B) = 12Lecture 4 : Conditional Probability and Bayes Theorem2/ 26Now what aboutP6 appears given a
2、n evennumber appears!Philosophical Remark(Ignore this remark unless you intend to be a scientist)At present the above probability does not have a formal mathematical definitionbut we can still compute it. Soon we will give the formal definition and ourcomputation “will be justified”. This is the mys
3、terious way mathematics works.Somehow there is a deeper reality underlying the formal theory.Back to Stat 400The above probability will be written written P(AjB) to he read P(A given B).Lecture 4 : Conditional Probability and Bayes Theorem3/ 26Now we know an even number occurred so the sample space
4、changes1 2 34 5 6the conditionalsample spaceSo there are only 3 possible outcomes given an even number occurred soP(6 given an even number occurred) = 13The new sample space is called the conditional sample space.Lecture 4 : Conditional Probability and Bayes Theorem4/ 262. Very Important exampleSupp
5、ose you deal two cards (in the usual way without replacement). What isP() i.e., P(two hearts in a row).Well, P(first heart) = 1352.Now what about the second heart?Many of you will come up with 12=51 andP() = (13=52)(12=51)Lecture 4 : Conditional Probability and Bayes Theorem5/ 26There are TWO theore
6、tical points hidden in the formula.Lets first look atP( on 2nd| z this isnt really correct) = 12=51What we really computed was the conditional probabilityP( on 2nd dealjon first deal) = 12=51Why? Given we got a heart on the first deal the conditional sample space is the“new deck” with 51 cards and 1
7、2 hearts so we getP( on 2nd j on 1st) = 12=51Lecture 4 : Conditional Probability and Bayes Theorem6/ 26The second theoretical point we used was that in the following formula wemultiplied the two probablities P( on 1st) and P( on 2nd j on 1st)together. This is a special case of a the formula for the
8、probability of theintersection of two events that we will state below.P() = P( on 1st)P( on 2nd j on 1st)=1352! 1251!The general formula, the multiplicative formula, we will give as a definitionshortly isP(A B) = P(A)PBjA)Compare this to the additive formula which we already provedP(A B) = P(A) + P(
9、B) P(A B)Lecture 4 : Conditional Probability and Bayes Theorem7/ 26Three Basic QuestionsThese three examples will occur repeatedly in todays lecture. The first is theexample we just discussed, the second is the reverse of what we just discussedand the third is a tricky variant of finding the probabi
10、lity of a heart on the first withno other information.1 What isP( on 2nd j on 1st| z reverse of pg. 5)2 What isP( on 1st j on 2nd| z reverse of pg. 5)3 What is P( on 2nd with no information on what happened on the 1st).Lecture 4 : Conditional Probability and Bayes Theorem8/ 26The Formal Mathematical
11、 Theory of Conditional Probability(S) = n; (A) = a; (B) = b; (A B) = CProblemLet S be a finite set with the equally - likely probability measure and A and B beevents with coordinalities shown in the picture.Lecture 4 : Conditional Probability and Bayes Theorem9/ 26ProblemCompute P(AjB).We are given
12、B occurs so the conditional sample space is BOnly part of A is allowed since we know B occurred namely the part A B. Socounting elements we getP(AjB) = (A B)(B)= cbLecture 4 : Conditional Probability and Bayes Theorem10/ 26We can rewrite this asP(AjB) = cb = c=nb=n= P(A B)P(B)soP(AjB) = P(A B)P(B) (
13、*)This formula for the equally likely probability measure leads to the following.Lecture 4 : Conditional Probability and Bayes Theorem11/ 26Formal Mathematical DefinitionLet A and B be any two events in a sample space S with P(B) , 0. Theconditional probability of A given B is written P(AjB) and is
14、defined byP(AjB) = P(A B)P(B) (*)so, reversing the roles of A and B (so we get the formula that is in the text) ifP(A) , 0 thenP(BjA) = P(BA)P(A) = P(A B)P(A) (*)Since A B BA.Lecture 4 : Conditional Probability and Bayes Theorem12/ 26We wont prove the next theorem but you could do it and it is usefu
15、l.TheoremFix B with P(B) , 0. P( jB), ()so P(AjB) as a function of A), satisfies theaxioms (and theorems) of a probability measure - see Lecture 1.For example1 P(A1A2jB) = P(A1jB) + P(A2jB) P(A1A2jB)2 P(A0jB) = 1 P(AjB)P(Aj ) (so P(AjB) as a function of B) does not satisfy the axioms andtheorems.Lec
16、ture 4 : Conditional Probability and Bayes Theorem13/ 26The Multiplicative Rule for P(A B)Rewrite (*) asP(A B) = P(A)P(BjA)()() is very important, more important then (*).It complement the formulaP(A B) = P(A) + P(B) P(A B)Now we know how P interacts with the basic binary operationsand.Lecture 4 : C
17、onditional Probability and Bayes Theorem14/ 26More generallyP(A BC) = P(A)P(BjA)P(CjA B)ExerciseWrite down P(A BCD).Traditional ExampleAn urn contains 5 white chips, 4 black chips and 3 red chips.Four chips are drawn sequentially without replacement. Find P(WRWB).Lecture 4 : Conditional Probability
18、and Bayes Theorem15/ 2612 chipsP(WRWB) =512! 311! 410! 49!What did we do formally? The answer is we used the following formula for theintersection of four eventsP(WRWB) = P(W) P(RjW) P(WjW R) P(BjW RW)Lecture 4 : Conditional Probability and Bayes Theorem16/ 26Now we make a computation that reverses
19、the usual order, namely, we computeP(on firstjon second)By DefinitionP(AjB) = P(A B)P(B)soP(on firstjon second) = P()Pon 2nd with noother information!Now we know from pg. 5.P() = (13=52)(12=51)Now we needP( on 2nd with no other information) = 13=52Lecture 4 : Conditional Probability and Bayes Theore
20、m17/ 26By DefinitionWe will prove this later (to some people this is intuitively clear). In fact if you writedown any probability statement in this situation, take that statement andeverywhere you see “first” write “second” and everywhere you see “second”write “first” then the resulting event will h
21、ave the same probability as the eventwe started with.So back to our problem we haveP( on 1st j on 2nd) = ( 13=52)( 12=51)( 13=52) =1251= P( on 2nd j on 1st| z pg. 5)This is another instance of the symmetry (in “first” and “second ”) stated threelines above.Lecture 4 : Conditional Probability and Bay
22、es Theorem18/ 26Bayes Theorem (pg. 72)Bayes Theorem is a truly remarkable theorem. It tells you “how to computeP(AjB) if you know P(BjA) and a few other things”.For example - we will get a new way to compute are favorite probabilityP( as 1st j on 2nd) because we know P( on 2nd j on 1st).First we wil
23、l need on preliminary result.Lecture 4 : Conditional Probability and Bayes Theorem19/ 26The Law of Total ProbabilityLet A1, A2;:; Ak be mutually exclusive(Ai Aj =;) and exhaustive.(A1A2:Ak = S = the whole space)Then for any event BP(B) = P(BjA1)P(A1) + P(BjA2)P(A2)+ + P(BjAk)P(Ak) (b)Prove thisFirst
24、 prove P(BjS) = 1 then use the P(B; C) is satisfies the additivity rule for aprobability measure as function of C.Special case k = 2 so we have A and A0P(B) = P(BjA)P(A) + P(BjA0)P(A0) (bb)Lecture 4 : Conditional Probability and Bayes Theorem20/ 26Now we can proveP(on 2nd with no other information)
25、= 13=52Put B = on 2ndA = heart on 1stA0 = a nonheart on 1stLets write for nonheart.So,P( on 1st) = 39=52P( on 2nd= on first) = 13=51Lecture 4 : Conditional Probability and Bayes Theorem21/ 26NowP(B) = P(BjA)P(A) + P(BjA0)P(A0)= P( on 2nd j on 1st)P( on 1st)+ P( on 2nd j on 1st)P( on 1st)= (12=51)(13
26、=52) + (13=51)(39=52)add fractions= (12)(13) + (13)(39)(51)(52)factor out 13 add to get 51= (13)xy(12 + 39)(51)(52) =(13)( 51)(51)(52)= (13)=(52) Done!Lecture 4 : Conditional Probability and Bayes Theorem22/ 26Now we can state Bayes Theorem.Bayes Theorem (pg. 73)Let A1; A2;:; Ak be a collection of n
27、 mutually exclusive and exhaustive eventswith P(Ai) 0i = 1; 2;:; k. Then for any event B with P(B) 0P(AjjB) = P(BjAj)P(Aj)kPi=1P(BjAi)P(Ai)Again we wont give the proof.Lecture 4 : Conditional Probability and Bayes Theorem23/ 26Special Case k = 2Suppose we have two events A and B with P(A) 0, P(A0) 0
28、 and P(B 0).Then)P(AjB) = P(BjA)P(A)P(BjA)P(A) + P(BjA0)P(A0)Now we will compute (for the last time)P( on 1st j on 2nd)Using Bayes Theorem.This is the obvious way toLecture 4 : Conditional Probability and Bayes Theorem24/ 26do it since we know the probability “the other way around”P( on 2nd j on 1st
29、) = 12=51So lets do it.We put A = on 1stso A0 = on 1stand B = on secondplugging into () we getP( on 1st j on 2nd)= P( on 2nd j on 1st)P( on 1st)P( on 2nd j on 1st)P( on 1st) + P( on 2nd j on 1st)P( on 1st)Lecture 4 : Conditional Probability and Bayes Theorem25/ 26swapnumbersusesomethingfromhigh scho
30、olfactor this outLecture 4 : Conditional Probability and Bayes Theorem26/ 26The algebra was hard but the approach was the most natural - a special case ofGeneral PrincipleCompulsory Reading (for your own heath)In case you or someone you love tests positive for a rare (this is the point)disease, read Example 2.31, pg. 81. Misleading (and even bad) statistics isrampant in medicine.Lecture 4 : Conditional Probability and Bayes Theorem
