statistics intermediate probabilitystatistics intermediate probability session6 oscarbarrera...

Statistics IntermediateProbabilitySession 6

Oscar [email protected]

April 3, 2018

Probability Probability and Sampling from a Population

Outline

1 ProbabilityThe Monty Hall ParadoxSome Concepts: Event AlgebraProbability Axioms and Things About Probability that areTrueCounting Rules: Permutations and Combination

2 Probability and Sampling from a Population

Oscar BARRERA Statistics Intermediate


The Monty Hall Paradox

This is a good example of people failing to take information intoaccount when estimating probabilities.




Remember the game show "Let’s Make a Deal"The contestant is presented with three doors, Door 1, Door 2,Door 3.Behind one door was a fabulous prize and behind the othertwo doors were gag prizes (goats).Monty Hall tells the contestant to pick one doors.Then MH opens one of the two remaining doors revealing agoat. "you can decide to stay with your choice or switch tothe other door"

Are you more likely to win by staying with your original choice orare you more likely to win by switching to the other unopened door?





Usual approach:it doesn’t matter whether you stay or switch, because theprobability of winning by staying is 0.5 and the probability ofwinning by switching is 0.5.





Usual approach:it doesn’t matter whether you stay or switch, because theprobability of winning by staying is 0.5 and the probability ofwinning by switching is 0.5.

Unusual approach:

The actual probability of winning by staying is 1/3 and theprobability of winning by switching is 2/3!!

We’ll go over the proof of this later.




Unusual approach:

The actual probability of winning by staying is 1/3 and theprobability of winning by switching is 2/3!!

We’ll go over the proof of this later.

for now think about this: the reason people assume the probabilityof winning by staying and by switching are equal is because people

neglect the fact that Monty Hall knows where the prize is!



Some Concepts: Event Algebra

As a student of the behavioral sciences you should love probability,You will use them a LOT!!

Before getting into more technical aspects of calculatingprobabilities we need to go through a few basic ideas of whereprobability comes from.




Some very important concepts

1. simple random experimentProcedure or operation performed where the outcome isunknown ex-ante

2. Sample Space

The collection of all possible outcomes of a simple randomexperiment

Notation: S




Simple random experiment: Procedure or operation performedwhere the outcome is unknown ex-anteSample Space: The collection of all possible outcomes of asimple random experiment

Some examples




1 Simple random experiment: Procedure or operation performedwhere the outcome is unknown ex-ante

2 Sample Space: The collection of all possible outcomes of asimple random experiment

3. An eventA unique outcome in the sample space of a simple randomexperiment; it is a subset of the sample space.




1 Simple random experiment: Procedure or operation performedwhere the outcome is unknown ex-ante

2 Sample Space: The collection of all possible outcomes of asimple random experiment

3 Event: A unique outcome in the sample space of a simplerandom experiment

4. Event algebra : Is a method of expressing relationships amongevents and among combinations of events. The more importantrelationships between events are

UnionIntersectionComplementMutually exclusiveincluded




4. Event algebra : UnionUnion: (A ∪ B) all outcomes that are associated with event AOR event B OR both

Example

Say A = {2, 4, 6, 8, 10, 12} and A = {4, 8, 12, 15}

(A ∪ B) = {2, 4, 6, 8, 10, 12, 15}




4. Event algebra : IntersectionIntersection: (A ∩ B) all outcomes that are associated withevent A AND event B.

Example

Say A = {2, 4, 6, 8, 10, 12} and A = {4, 8, 12, 15}

(A ∩ B) = {4, 8, 12, }




4. Event algebra : ComplementComplement: (Ac) all the event that consists of all outcomesthat are NOT associated with its own outcomes.

Example

Ac = {1, 3, 5, 7, 9, 11, 13, 14, 15}




4. Event algebra : Mutually exclusiveMutually exclusive events: (A ∩ B = Φ) Where Φ isimpossible.

Example

Events A and B are not mutually exclusive, because the intersectionof events A and B is possible.




4. Event algebra : IncludedIncluded events: (A ⊂ B) if each and every outcome in eventA is also in event B



Probability Axioms and Things About Probability that are True

The probability of the outcomes in a simple random experiment is afunction P that assigns a real number, P(E)=a, to each event E ina sample space S, and satisfies the following three axioms:

P(E ) > 0 : probability of an event must be greater than orequal to zeroP(S) = 1 : probability of observing something in the samplespace is 1P(A ∪ B) = P(A) + P(B) probability of observing at least oneof two mutually exclusive events is the sum of their respectiveprobabilities.

By using algebra we can derive some other properties:




P(E ) > 0P(S) = 1P(A ∪ B) = P(A) + P(B)

By using algebra we can derive some other properties:

For any event, P(Ac) = 1− P(A)

For any event, P(A) 6 1For all events A and B, P(A∪B) = P(A) + P(B)−P(A∩B),whether A and B are mutually exclusive or notFor all events A and B, P(A) = P(A ∩ B) + P(A ∩ Bc),whether A and B are mutually exclusive or not.




To calculate the probability of an event (A), identify the number ofpossible outcomes favoring event A and divide that by the numberof possible outcomes where event A can occur in the sample space(S):

P(A) =A

S

For example, we want to calculate the probability of selecting aJack of Clubs from a deck of cards.




P(A) =A

S

For example, we want to calculate the probability of selecting aJack of Clubs from a deck of cards.

P(A) =152

= 0.019

standard deck of 52 cards there is only one jack of clubs, so A = 1and the number of outcomes where event A can occur is S = 52.




P(A) =A

S

What is the probability of selecting any king?




P(A) =A

S

What is the probability of selecting any king?

P(A) =452

= 0.077

There are four possible kings in a deck of cards king of clubs, kingof spades, king of diamonds, king of hearts.



Counting Rules: Permutations and Combination

3 Prisoners and 5 Hats

There were three prisoners in certain jail,one of whom had normal sight, one of whom had only one eye,and the third of whom was completely blind.All three out and had them stand in a line.He then showed them five hats, three white and two red,which he hid in a bag.From these five hats, he selected three and put one on each ofthe prisoners’ heads.None of the men could see what color hat he himself wore.The jailor offered freedom to the sighted man, if he could tellhim what color his hat was. (and to prevent guessing, hethreatened execution for a wrong answer)




The sighted man could not tell what color hat he wore, anddeclined to answer.The jailor next offered the same terms to the one-eyed man,who also could not tell what color hat he wore, and alsodeclined to answer.At this point the jailor thought his game was done, but wasdelighted when the blind prisoner spoke up and asked if hedidn’t get a turn? Laughing, the jailer offered the blind manthe same terms as the others. The blind prisoner smiled andreplied,

I do not need my sight; From what my friends have said, Iclearly see my hat is !




permutations

In the behavioral sciences it is often the case that researchers havea certain number of conditions that can be presented to subjects.

DefinitionThat is, how many orders in which the conditions can be presented.A permutation is an ordered arrangement of distinct items.

The total number of permutation of n distinct items is:

n!

n factorial.




permutations

The total number of permutation of n distinct items is:

n!

n factorial.The factorial (!) operator, tells you to multiply the number (n) byall numbers less than n not including zero.

Examples2! = 2x1 = 23! = 3x2x1 = 64! = 4x3x2x1 = 24

Remember 1! = 1 and 0! = 0.




Example more concrete:

say an investigator is conducting a taste-preference survey fordifferent types of scotch. The investigator has three types of scotchthat can be presented and wants to know all of the permutationsthat are possible for the three scotches.

3! = 3x2x1 = 6




Permutation

3! = 3x2x1 = 6The researcher may want to present only a subset (r) of all thepossible conditions or outcomes.

The number of permutations of r items out of n items is:

nPr =n!

(n − r)!

In the scotch example: the researcher might want to know thenumber of permutations of one scotches out of the three scotches:

3P1 =3!

(3− 1)!=

62!

=62

= 3




Combination

Alternatively, a researcher want to know how many possiblecombinations of subsets are possible from some larger number ofconditions.

The formula for is:

nPr =n!

(n − r)!r !

Using the scotch example, assume that the researcher wants toknow the number of combinations of 2 out of 3 scotches:

3P1 =3!

(3− 2)!=

61!2!

=62

= 3

We’ll use these ideas of permutations and combinations to calculateprobabilities




Combination

Example 1:What is the probability of getting exactly one club and exactly onejack in two randomly dealt cards; assuming order does not matter?

Event A = one club and one jackThere are 13 clubs and 4 jacks, and the number of unorderedcombinations of clubs and jacks is 13 x 4 =52

Sample Space, S = all unordered two card hands:

52C2 =n!

(n − r)!r !

52C2 =52!

(52− 2)!2!=

52!

(50)!2!=

52x512

= 1326

P(A) =56

1326= 0, 042




Combination

Example 2:What is the probability that a five-card poker hand contains exactlythree clubs?

Event A = three clubs and two non-clubsFor the three clubs

13C3 =13!

(13− 3)!3!=

13x12x116

= 286

For the 2 non clubs

39C2 =39!

(37)!2!=

39x382

= 741

Sample space

52C5 =52!

(47)!5!=

52x51x50x49x48120

= 2598960

P(A) =741 ∗ 2862598960

= 0.082




Combination

Example 2:What is the probability that a five-card poker hand contains exactlythree clubs?

Event A = three clubs and two non-clubs

Sample space

52C5 =52!

(47)!5!=

52x51x50x49x48120

= 2598960

P(A) =741 ∗ 2862598960

= 0.082




Combination

Example 3:

Event A = all five cards are from the same suit

For the 5 cards same suit

13C5 =13!

(13− 5)!5!=

13x12x11x10x9120

= 5148

Sample space

52C5 =52!

(47)!5!=

52x51x50x49x48120

= 2598960

P(A) =5148

2598960= 0.002




Combination

Example 4:

Event A = A poker of aces?



One issue with calculating probability is how the selecting ofsamples from a population can change the probability of futureselections. Sample with replacement or sample withoutreplacement?

Replacement occurs when after a case is selected it is returned(replaced) to the population.Without replacement occurs when after a case is selected it isnot returned to the population.

There are pros and cons of each method.



Sampling with replacementthe population size (N) never changes, so the probability of anevent never changes. =)the population you run the risk of selecting the same casemore than once.=(

Sampling without replacementCorrects for the possibility of selecting a case multiple times(replace the unit) =)The probability of unselected events being selected increasesover time (As N gets lower) =(Most population are large (on the order of millions), alteringthe number of cases that can be selected should not makemuch of a difference.

Most behavioral scientists use sampling without replacement.



As examples of sampling with and without replacement:

N=5025 psychology majors.10 neuroscience majors.10 communications majors5 counseling majors.

Say we select a student at random from the class. The initialprobabilities of selecting a student from each major are



Let’s say we select a Communications major, but the student wasnot returned to the class (sampling without replacement).

Say that we select a second student, a psychology major, and donot return that student to the class.



Say that we select a second student, a psychology major, and donot return that student to the class.

The point is that changing the underlying population size, just likechanging information in a situation (e.g., Monty Hall problem),alter probabilities of selecting events:



Probabilities Based on Multiple Variables

Assume a researcher is interested in the association betweenpolitical orientation and sexual behavior.

The researcher surveys all incoming freshmen at a university.N = 1000 freshmen and two questions

1. political attitude: liberal and conservative2. Whether the student has engaged in intercourse: yes and no.

Thus, the freshmen can be classified into one of four groups:




Thus, the freshmen can be classified into one of four groups:

1 liberals who have had intercourse2 liberals who have not had intercourse3 conservatives who have had intercourse4 conservatives who have not had intercourse

For argument’s sake, assume all freshmen are 18 years old and thatthere are equal numbers of males and females in each group.

Contingency tableA contingency table is used to record the relationship between twoor more variables.




Contingency tableA contingency table is used to record the relationship between twoor more variables.

Now, probabilities that are conditional on (dependent on) someother event can be calculated.




Simple Probabilities: We want to determine the probability ofselecting a liberal (conservative) student from among all incomingfreshmen.

P(L) =numberofliberals

N=

7001000

= 0.7

P(C ) =numberofconservatives

N=

3001000

= 0.3

This is the probability of one political orientation without takinginto account the variable previously engaged in intercourse.




Student can either be a liberal or can be a conservative, but cannotbe a political conservative and a political liberal; this means thatthe two outcomes for Political Attitude are mutually exclusive.

Conditional ProbabilitiesIs the probability of selecting some event (A) given some otherevent (B) is selected, that is, that some other condition is satisfied.

P(A|B) =A ∩ B

B

Say the researcher wants to calculate the probability of selecting astudent that has engaged in intercourse given the student is also apolitical conservative.




P(A|B) =A ∩ B

B

Say the researcher wants to calculate the probability of selecting astudent that has engaged in intercourse given the student is also apolitical conservative.

P(Yes|C ) =Yes ∩ C

C=

100300

= 0.333

33.3 percent chance of selecting a student who has had sexualintercourse, if you have also selected a conservative student.




P(A|B) =A ∩ B

B

Let’s say that the researcher is now interested in the probability ofselecting a liberal student given we have selected a student who hasnot had intercourse.




P(A|B) =A ∩ B

B

Let’s say that the researcher is now interested in the probability ofselecting a liberal student given we have selected a student who hasnot had intercourse.

P(L|No) =L ∩ No

No=

200400

= 0.5

There is about a 50 % chance that the researcher has selected aliberal freshmen student if he has selected a student who has notengaged in intercourse.



Dependent and independent events

Conditional probabilities can be used to determine whether there isa relationship between the two events. Whether the two events areindependent or not.

If two events are independent it means that selecting one eventdoes not depend on the other event.

P(A) = p(A|B)

two events are not independent, that is, there is a relationshipbetween the events

P(A) 6= p(A|B)

Let’s determine whether having previously engaged in intercourse(event A) is independent of being a conservative (event B)...



Dependent and independent events

Let’s determine whether having previously engaged in intercourse(event A) is independent of being a conservative (event B)...

P(Y ) =6001000

= 0.6

P(Y |C ) = 0.33

So P(Y ) 6= p(Y |C ) then some relationship must exist betweenhaving engaged in intercourse and being a conservative



Joint Probabilities

If you wish to determine the probability of a combination of twoevents out of all events you must calculate a joint probability.

DefinitionA joint probability is simply the probability of selecting tow events,A and B, together.

P(A ∩ B) =P(A ∩ B)

Nsay that the researcher wants to calculate the probability ofselecting a student that has not engaged in intercourse (event A)and is a politically liberal student (event B).



Joint Probabilities

P(A ∩ B) =P(A ∩ B)

N

Say that the researcher wants to calculate the probability ofselecting a student that has not engaged in intercourse (event A)and is a politically liberal student (event B).



Joint Probabilities

Say that the researcher wants to calculate the probability ofselecting a student that has not engaged in intercourse (event A)and is a politically liberal student (event B).

P(N ∩ L) =P(N ∩ L)

N=

2001000

= 0.2

the probability of selecting a student who has not had intercourseand is a politically liberal student is p = 0.2.



Addition Rule of Probability

The addition rule of probability is used to determine the probabilityof at least one of two different events, or both events, occurring.

That is, what is the probability of only event A, only event B, orboth events A and B occurring?

P(A ∪ B) = P(A) + P(B)− P(A ∩ B)

Say that the researcher wants to calculate the probability ofselecting a student who has engaged in intercourse (event A) or ispolitically liberal (event B), or had intercourse and is liberal.




P(A ∪ B) = P(A) + P(B)− P(A ∩ B)






P(Y)= 600/1000 = 0.6.P(L) = 0.7P(Y ∩ L) = 0.5

P(Y ∪ L) = 0.6 + 0.7− 0.5 = 0.8

there is an 80 percent chance of selecting at least a student howhas had intercourse or is a politically liberal student.



Multiplication Rules of Probability

DefinitionThe multiplication rule is used to calculate the probability ofselecting two events, A and B, like the joint probability.

There are two ways that the multiplication rule can be applied:1 when events are independent2 when events are not to be independent

If two events, A and B, are independent and there is no relationshipbetween the events. Such as:

P(A) = p(A|B)

the following multiplication rule can be applied to events A and B,

P(A ∩ B) = P(A)P(B)




P(A ∩ B) = P(A)P(B)

Example

Finding the probability of rolling a ’2’ on a six-sided die (event A)and flipping a ’tails’ on a two sided coin (event B).

The joint probability of these two events can be found by applyingthe multiplication rule for independent events:

P(2 ∩ tails) = P(2)P(tails) =1612

=112

= 0.084




P(A ∩ B) = P(A)P(B)

Example 2

what is the probability of flipping tails on a two-sided coin (eventA) and then flipping heads on the same coin (event B).




P(A ∩ B) = P(A)P(B)

Example 2

what is the probability of flipping tails on a two-sided coin (eventA) and then flipping heads on the same coin (event B).

P(tails ∩ head) = P(tails)P(head) =1212

=14

= 0.25




If two events A and B are not independent such that

P(A) 6= P(A|B)

then we determine the probability of observing both events as:

P(A ∩ B) = P(B)P(A|B) = P(A)P(B|A)




From the contingency table, say the researcher wants to calculatethe probability of selecting a student who had intercourse (event A)who is politically conservative (event B).

P(A ∩ B) = P(B)P(A|B) = P(A)P(B|A)

That would be

P(Y ∩ C ) = P(C )P(Y |C ) = (0.33)(0.3) = 0.1

Note that this is equal to the joint probability calculated above;that is, P(Yes ∩ Conservative) = 100/1000 = 0.1.




Now say that the researcher is interested in the probability ofselecting a politically conservative student, not returning thatstudent to the population, and then selecting another politicallyconservative student?

P(A ∩ B) = P(B)P(A|B) = P(A)P(B|A)

That would be

P(C1 ∩ C2) = P(C2)P(C1|C2) = (3001000

)(299999

) = 0.0987

Note that this does round to 0.09, but I wanted to show that thereis a difference between sampling with and sampling withoutreplacement.



Bayes’ Theorem

From the previous slide we know that:

P(B)P(A|B) = P(A)P(B|A)

So

P(A|B) =P(A)P(B|A)

P(B)

This expression is one form of Bayes Theorem, which is usedfor computing the conditional probability of an event A givenevent B.



Bayes’ Theorem

Example, say that we want to calculate the probability of selectinga student who has had intercourse (event A) given they are aconservative student (event B) P(Yes|Conservative). from earlierwas 100/300 = 0.333.We can also find this probability using Bayes’ Theorem:

P(Y |C ) =P(Y )P(C |Y )

P(C )=

(0.6)(0.167)

0.33=

0.10.33

= 0.33

The important point to remember is that you can use relationshipsbetween variables and events to calculate probabilities of observingspecific outcomes from the combinations of events.



Back to Monty Hall

Now that you know a little about probability and understand themost important thing when calculating a probability is to takeinformation into account, let’s go back to the Monty Hall problem.

Remember: After you make your initial selection and Monty Hallopens a door exposing a goat

The probability of winning by staying with your original choiceof door is 1/3.The probability of winning through switching to the otherunopened door is 2/3

The probabilities change only after Monty Hall opens a door



Back to Monty Hall

Winning by staying: Say you initially choose Door 1, which is thedoor that hides the prize.

In this case MH can open Door 2 or Door 3 to expose a goatand the probability that Monty Hall opens either door is 1/2.there is 1/6 chance Monty Hall will open Door 2 and a 1/6chance he will open Door 3. (1/3)(1/2) = 1/6.In either case, if you stay with Door 1 (the door with theprize) you win if you stay and you lose if you switch.



Back to Monty Hall

Winning by switching: say you initially pick Door 2 (which hidesa goat)

because the initial pick was a door with a goat Monty Hall isforced to open the door that hides the other goat.Remember, Monty Hall knows which door hides the prize!From the 1/3 chance you select Door 2, the probability thatMonty Hall will open Door 3 to expose the other goat 1.0.



Back to Monty Hall

Winning by switching: say you initially pick Door 3 (which alsohides a goat)

because the initial pick was a door with a goat Monty Hall isforced to open the door that hides the other goat.From the 1/3 chance you select Door 3, the probability thatMonty Hall opens Door 2 is 1.0.In this case if you switch you win, but if you stay you lose.



Back to Monty Hall

Indeed, if you look at the statistics from the game show "Let’sMake a Deal", people won after a switch about 2/3 of the time,and people won from staying about 1/3 of the time.



Back to Monty Hall

The point is that information about events, whether subtle as inthe case of Monty Hall or more overt, provides information that canseriously alter a probability structure; and this information shouldalways be taken into account.



Statistics IntermediateProbabilitySession 6

Oscar [email protected]

April 3, 2018


statistics intermediate probabilitystatistics intermediate probability session6 oscarbarrera...

Documents