Introduction to Probability and Statistics

Probability & Statistics for Engineers & Scientists, 8th Ed.2007

Handout #1

Instructor: Kuo-Jung Lee

TA: Brian Shea

The pdf file for this class is available on the class web page.http://www.stat.umn.edu/~kjlee/STAT3021_Summer2009.html

1

An Overview of Statistics

2

Whats Statistics?

Statistics is a way to get information from data

3

Statistics is a discipline which is concerned with:

summarizing information to aid understanding,

drawing conclusions from data,

estimating the present or predicting the future, and

designing experiments and other data collection.

In making predictions, Statistics uses the companion subject ofProbability, which models chance mathematically and enablescalculations of chance in complicated cases.

4

Today, statistics has become an important tool in the work of

many academic disciplines such as medicine, psychology, edu-

cation, sociology, engineering and physics, just to name a few.

Statistics is also important in many aspects of society such as

business, industry and government. Because of the increasing

use of statistics in so many areas of our lives, it has become very

desirable to understand and practise statistical thinking. This is

important even if you do not use statistical methods directly.

5

Data

Data consists of information coming from observation, counts,

measurements, or responses.

Statistics

Statistics is the science of collecting, organizing, analyzing, and

interpreting data in order to make decisions.

6

Population

A population is the collection of all outcomes, responses, mea-

surements, or counts that are of interest.

Sample

A sample is a subset of a population.

7

Parameter

A parameter is numerical description of a population character-

istics.

Statistic

A statistic is numerical description of a sample characteristic.

8

Branches of Statistics

Descriptive statistics is the branch of statistics that involves

the organization, summarization, and display of data.

Inferential statistics is the branch of statistics that involves

using a sample to draw conclusions about a population. A basic

tool in the study of inferential statistics is probability.

9

Example

A large sample of men, aged 48, was studied for 18 years. For

unmarried men, 60% to 70% were alive at age 65. For married

men, 90% were alive at age 65. Which part of the study repre-

sents the descriptive branch statistics? What conclusions might

be drawn from this study using inferential statistics?

10

Solution:

Descriptive statistics: For unmarried men, 60% to 70% were

alive at age 65. For married men, 90% were alive at age 65.

A possible inference: Being married is associated with a longer

life for men.

11

Example

An instructor is teaching two separate classes, A and B each

of size is 50. After a midterm, the scores for each class are:

A: 50 47 59 49 72 41 63 79 91 65 49 59 92 42 34 43 53 89 76

93 89 51 42 46 67 48 33 47 68 51 56 53 69 53 43 36 58 85

45 64 57 32 1 60 66 60 63 86 62 55

B: 56 61 53 59 60 55 57 49 67 60 58 56 58 59 55 52 60 68 45

59 67 62 42 50 53 63 61 61 57 70 49 64 52 58 58 70 48 66

58 58 61 58 68 58 54 60 61 61 61 72

12

Histogram of Scores for Class A

Score

Freq

uenc

y

0 20 40 60 80 100

02

46

810

12Histogram of Scores for Class B

Score

Freq

uenc

y

40 45 50 55 60 65 70 75

05

1015

13

Class 1st Q Median 3rd Q Mean Standard Deviation

A 47.00 56.50 66.75 58.34 18.24B 55.25 58.50 61.00 58.56 6.29

14

Chapter 2 Probability

15

Sample Space / Events

Counting Sample Points

Probability of an Event

Additive Rules

Conditional Probability

Multiplicative Rules

Bayes Rule16

2.1 Sample Space

17

Experiment

Experiment is any process that generates a set of data.

Sample space

Sample space is the collection of all possible outcomes at a

probability experiment. We use the notation S for sample space.

18

Example 1

Toss a coin. The possible outcomes are heads, tails. So thesample space is S= {Heads, Tails}.

19

Example 2

Roll a dice. The possible outcomes are the six faces, numbered

1, 2, 3, 4, 5, 6. Hence the sample space is S= {1, 2, 3, 4, 5,6}.

20

Sample Points

Each outcome in a sample space is called an element or amem-

ber of the sample space, or simply a sample point.

Tree Diagram

In some experiments, it is helpful to list the elements of the

sample space systematical by means of a tree diagram.

21

Example 3

An experiment consists of flipping a coin and then flipping it asecond time if head occurs. If a tail occurs on the first flip, thena die is tossed once. The sample space

S = {HH,HT, T1, T2, T3, T4, T5, T6}

22

Statement & Rule

Sample spaces with a large or infinite number of sample pointsare best described by a statement or rule.

Example 4: Statement

The possible outcomes of an experiment are the set of cities inthe world with a population over 1 million, our sample space iswritten S = {x|x is a city with a population over 1 million}.

Example 5: Rule

If S is the set of all points (x, y) on the boundary or the interiorof a circle of radius 2 with center at the origin, we write the rule:S = {(x, y)|x2+ y2 22}

23

2.2 Events

24

Event

Event is a sub-collection of outcomes from the sample space.

We are interested in probabilities of events.

Example 6

In the experiment of tossing a die, consider the event E that the

outcome when a die is tossed is divisible by 3.

S = {1,2,3,4,5,6}E = {3,6}

25

Example 7

Given the sample space S = {t|t 0}, where t is the life in yearsof a certain electronic component, then the event A that the

component fails before the end of the third year is the subset

A = {t|0 t < 3}

26

Null Set

A set contains no elements at all, and denoted by the symbol .

Example 8

Roll a die. One event that you may be interested in is E1 =

{You get an even number} = {2, 4, 6}. Another one could beE2 = {You get a prime number} = {2, 3, 5}. Yet another onecould be E3 = {You get a multiple of 7} = .

E1 = {You get an even number} = {2,4,6},E2 = {You get a prime number} = {2,3,5},E3 = {You get a multiple of 7} = .

27

Complement

The complement of event A with respect to S is the subset ofall elements of S that are not in A. We denote the complementof A by the symbol either A or Ac

28

Intersection

The intersection of two events A and B, denoted by the symbol

A B, is the event containing all elements that are common toA and B.

29

Union

The union of the two events A and B, denoted by the symbol

A B, is the event containing all the elements that belong to Aor B or both.

30

Example 8 Contd

Roll a die. One event that you may be interested in is E1 =

{You get an even number} = {2, 4, 6}. Another one could beE2 = {You get a prime number} = {2, 3, 5}. Yet another onecould be E3 = {You get a multiple of 7} = .

S = {1,2,3,4,5,6},E1 = {You get an even number} = {2,4,6},E2 = {You get a prime number} = {2,3,5},E3 = {You get a multiple of 7} = .

E1 = {1,3,5}, E2 = {1,4,6}, E3 = SE1 E2 = {2}, E1 E2 = {2,3,4,5,6}

31

Mutually Exclusive or Disjoint

Two events A and B are mutually exclusive, or disjoint, if

A B = , that is, if A and B have no elements in common.

Example 9

Roll a die. One event that you may be interested in is E1 = You

get an even number = {2, 4, 6}. Another one could be E2 =You get a odd number = {1, 3, 5}. Then E1 E2 = . E1 andE2 are disjoint.

32

Venn Diagrams

The relationship between events and the corresponding sample

space can be illustrated graphically by means of Venn diagrams.

33

AB = regions 4 and 7. BC = regions 6 and 7. A C = regions 1, 3, 4,5, 6 and 7.

B A = regions 1 and5.

A B C = regions 7. (AB)C = regions 1,2 and 4.

34

Example 10

Suppose that in a senior college class of 600 students it is found

that 230 smoke, 250 drink alcoholic beverages, 240 eat between

meals, 150 smoke and drink alcoholic beverages, 90 eat between

meals and drink alcoholic beverages, 100 smoke and eat between

meals, and 60 engage in all three of these bad health practices.

35

A = Smoker. B = Drinking alcoholicbeverages.

C = Eat between meals.

Smokes but does notdrink alcoholic beverage.

A B={1, 5}. Eats between mealsand drinks alcoholic

beverages but does not

smoke.C B A ={2,3,6}

Neither smokes nor eatsbetween meals. (A C) = A C = {2}

Example 11

If S = {x|0 < x < 12}, A = {x|1 x < 9}, and B = {x|0 < x < 5},find

(a) A B ={x|0 < x < 9}(b) A B ={x|1 x < 5}(c) A B ={x|0 < x < 1,5 x < 12}

36

2.3 Counting Sample Points

37

GOLD

To count the the number of points in the sample space without

actually listing each element.

Multiplication Rule

If an operation can be performed in n1 ways, and if for each

of these ways a second operation can be performed in n2 ways,

then the two operations can performed together in n1n2 ways.

38

Example 12

How many sample points are there in the sample space when apair of 6-sided dice is thrown once?

This first dice can land in any one of n1 = 6 ways. For each ofthese 6 ways the second dice can also land in n2 ways. Therefore,the pair of dice can land in

n1 n2 = 6 6 = 36.39

GOLD

To count the the number of points in the sample space that

contains elements as all possible orders or arrangements of a

group of objects.

Permutation

A permutation is an arrangement of all or part of a set of

objects.

40

Example 13

Consider the three letter a, b, and c. The possible permutations

are abc, acb, bac, bca, cab, and cba. Thus we see that there are 6

distinct arrangements, because there n1 = 3 choices for the first

position, then n2 = 2 for the second, leaving only n3 = 1 choice

for the last position, giving a total of

n1 n2 n3 = 3 2 1 = 6

41

In general, n distinct objects can be arranged in

n(n 1)(n 2) (3)(2)(1) waysWe represent this product by the symbol n!, which is read n

factorial. We define 0! = 1.

Theorems

The number of permutations of n distinct objects is n!.

42

Example 14

Consider the 4 letter a, b c, and d. The number of permutation of

the 4 letters will be 4!. Now consider the number of permutations

that are possible by taking 2 letters at a time from four. The

possible permutations are ab, ac, ad, ba, bc, bd, ca, cb, cd da, db,

and dc. Thus we see that there are 12 distinct arrangements,

because there are n1 = 4 choices for the first position, then

n2 = 3 for the second,giving a total of

n1 n2 = 4 3 = 12

43

In general, n distinct objects taken r at a time can be arranged

in

n(n 1)(n 2) (n r+1) = n!(n r)! ways

Theorems

The number of permutations of n distinct objects when taken r

at a time is

nPr =n!

(n r)!

44

Example 15

A president and a treasurer are to be chosen from a student club

consisting of 50 people. How many different choices of officer

are possible if

1. there are no restrictions;

2. A will serve only if he is president;

3. B and C will serve together or not at all;

4. D and E will not serve together?

45

Solution:

1. 50P2.

2. 49P1+49 P2.

3. 2P2+48 P2.

4. 50P2 2.

Example 16

Consider the three letter a, b, and c. The possible permutations

are abc, acb, bac, bca, cab, and cba. Thus we see that there are 6

distinct arrangements, because there n1 = 3 choices for the first

position, then n2 = 2 for the second, leaving only n3 = 1 choice

for the last position, giving a total of

n1 n2 n3 = 3 2 1 = 6If the letters b and c are both equal, then the 6 permutation

of the letters a, b, and c become only 3 distinct permutations.

Therefore, with 3 letters, 2 begin the same, we have 3!2! distinct

permutations.

46

Theorem

The number of distinct permutations of n things of which n1 are

of one kind, n2 of a second kind, . . . , nk of a kth kind is

n!

n1!n2! nk!

47

Example 17

How many distinct permutation can be made from the letters of

1. aabbcc;

Solution:6!

2!2!2!

48

GOAL

To compute the number of partition of a group.

Partition

The number of ways of partitioning a set of n objects into r cells

with n1 elements in the first cell, n2 elements in the second, and

so forth, is (n

n1, n2, . . . , nr

)=

n!

n1!n2! nr!where n = n1+ n2+ + nr.

49

Example 18

Consider the set {A, B, C, D, E}. The possible partitions intotwo cells in which the first cell contains 4 letters and the second

cell 1 element areA, B, C, D E

A, B, C, E D

A, B, D, E C

A, C, D, E B

B, C, D, E A

50

Example 19

In how many ways can 7 graduate students be assigned to one

triple and two double hotel rooms during a conference?

Solution:

The total number of possible partitions would be

7!

3!2!2!=

(7

3, 2, 2

)= 210.

51

In many problems we are interested in the number of ways of

selecting r objects from n without regard to order. The selection

are called combination. A combination is actually a partition

with two cells, the one cell containing the r objects selected and

the other cell containing the (n r) objects that are left. Thenumber of such combinations, denoted by(

nr, n r

), is usually shortened to

(nr

)

52

Combination

The number of combinations of n distinct objects taken r at a

time is

(nr

)=

n!

(n r)!r!

53

Example 20

A young boy asks his mother to get five Game-BoyTM cartridgesfrom his collection of 10 arcade and 5 sports games. How manyways are there that his mother will get 3 arcade and 2 sportsgames, respectively?Solution:The number of ways selecting 3 cartridges from 10 is(

103

)=

10!

3!(10 3)! = 120

The number of ways of selecting 2 cartridges from 5 is(52

)=

5!

2!(5 2)! = 10

Using the multiplication rule with n1 = 120 and n2 = 10, thereare 120 10 = 1200 ways.

54

2.4 Probability of an Event

55

Probability

Probability is a number associated to events, the number de-

noting the chance of that event occurring.

56

Properties of Probability

Probability is a set function P that assigns to each event A in

the sample space S a number P (A), called the probability of theevent A, such that the following properties are satisfied:

1. 0 P (A) 1.

2. P (S) = 1.

3. If Ai are mutually exclusive, then

P (A1 A2 ) =i=1

P (Ai)

57

Theorem

If an experiment can result in any one of N different equally

likely outcomes, and if exactly n of these outcomes correspond

to event A, then the probability of event A is

P (A) =n

N.

That is,

P (A) =Number of outcomes favorable to A

Total number of outcomes for the experiment.

58

Example 21

A coin is tossed twice. What is the probability that at least on

head occurs?

Solutions:

S = {HH,HT, TH, TT};A = {HH,HT, TH}

P (A) =3

4

59

Poker

60

Example 22

In a poker hand consisting of 5 cards, find the probability of

holding 2 aces and 3 jacks?

Solutions:

The number of ways of being dealt 2 aces from 4 is(42

)=

4!

2!(4 2)! = 6

and the number of ways of being dealt 3 jacks from 4 is(43

)=

4!

3!(4 3)! = 4.

There are n = 6 4 = 24 hands with 2 aces and 3 jacks. Thetotal number of 5-card poker hands, all of which are equally

61

likely, is

N =

(525

)=

52!

5!(52 5)! = 2,598,960.

Therefore, the probability of event C of getting 2 aces and 3

jacks in a 5-card poker hand is

P (C) =24

2,598,960.

2.5 Additive Rule

62

Additive Rule

If A and B are two events, then

P (A B) = P (A) + P (B) P (A B).

If they are mutually exclusive (disjoint), then

P (A B) = P (A) + P (B).63

Example 23

What is the probability of getting a total of 7 or 11 when a pair

of fair dice are tossed?

Solutions:

Let A be the event that 7 occurs and B the event that 11 comes

up. Since events A and B are mutually exclusive, since a total

of 7 and 11 cannot both occur on the same toss. Therefore

P (A B) = P (A) + P (B) = 16+

1

18=

2

9.

64

Theorem

For three events A, B, and C,

P (A B C) =P (A) + P (B) + P (C)P (A B) P (A C) P (B C)+P (A B C).

If A and A are complementary events, AA = and AA = S,then

P (A) + P (A) = 1.

65

2.6 Conditional Probability

66

The probability of an event B occurring when it is known that

some event A has occurred is called a conditional probability

and is denoted by P (B|A). The symbol P (B|A) is usually readthe probability that B occurs given that A occurs or simple

the probability of B, given A.

Conditional Probability

For any two events A and B with P (A) > 0, the conditional

probability of B given that A has occurred is:

P (B|A) = P (A B)P (A)

.

67

Example 24

Roll a dice. What is the chance that youd get a 6, given that

youve gotten an even number?

Solutions:

Let A be the event of even numbers, and B of 6.

A = {2,4,6}, P (A) = 12; (1)

B = {6}, P (B) = 16; (2)

A B = {6}, P (A B) = 13; (3)

P (B|A) = P (A B)P (A)

=1

3

68

Example 25

In an experiment to study the relationship of hypertension andsmoking habits, the following data are collected for 200 individ-uals:

Nonsmokers Moderate Heavysmokers smokers

Hypertension 20 40 30Non-hypertension 60 30 20

If one of these individuals is selected at random, find the proba-bility that the person is

1. experiencing hypertension, given that the person is a heavysmoker;

69

2. a nonsmoker, given that the person is experiencing no hy-

pertension.

Solutions:

We shall concerned with the following events

A =the person experiences hypertension, P (A) =90

200

B =the person experiences no hypertension, P (B) =110

200= P (A)

C =the person is a heavy somker, P (C) =50

200

D =the person is a nonsmoker, P (D) =80

200A C =the person experiences hypertension and is a heavy smoker

P (A C) = 30200

B D =the person experiences no hypertension and is a nonsmokerP (B D) = 60

200

1. experiencing hypertension, given that the person is a heavy

smoker;

P (A|C) = P (A C)P (C)

=30/200

50/200=

3

5

2. a nonsmoker, given that the person is experiencing no hy-

pertension.

P (D|B) = 60/200110/200

=6

11

Example 26

Tossing a 6-sided dice which is construed so that the even num-bers are twice as likely to occur as the odd number.

1. Consider the event B of getting a perfect square when a diceis tossed. P (B) =?

2. What is the probability of getting a perfect square when it isknow that the toss of the dice resulted in a number greaterthan 3?

3. What is the probability of getting a perfect square when it isknow that the toss of the dice resulted in a number greaterthan 4?

70

Solution:

1. B = {1,4}, P (B) = 39

2. P (B|A) = 25, A = {4,5,6}

3. P (B|A) = 0, C = {5,6}

Conditional Probability

The conditional probability of B, given A, denoted by P (B|A) isdefined by

P (B|A) = P (A B)P (A)

provided P (A) > 0.

Independent Event

Two events A and B are independent if and only if

P (B|A) = P (B) or P (A|B) = P (A)provided the existences of the conditional probabilities. Other-

wise, A and B are dependent.

71

Sampling With Replacement

Sampling with replacement occurs when an object is selected

and then replaced before the next object is selected.

Sampling Without Replacement

Sampling without replacement occurs when an object is not re-

placed after it has been selected.

72

Example 27

Consider an experiment in which 2 cards are drawn in successionfrom an ordinary deck, with replacement. The events are definedas

1. A: the first card is an ace,

2. B: the second card is a space.

Since the first card is replaced, our sample for both the firstand second draws consists of 52 cards, containing 4 aces and 13spades. Hence

P (B|A) = 1352

=1

4and P (B) =

1

4.

73

That is, P (B|A) = P (B). When this is true, the events A and Bare said to be independent.

2.7 Multiplicative Rules

74

Multiplicative Rule

If in an experiment the events A and B can both occur, then

P (A B) = P (A)P (B|A) provided P (A) > 0.

Theorem

Two events A and B are independent if and only if

P (A B) = P (A)P (B).Therefore, to obtain the probability that two independent events

will both occur, we simply find the product of their individual

probabilities.

75

Example 28

Three cards are drawn in succession, without replacement, from

an ordinary deck of playing cards. Find the probability that the

event A1 A2 A3 occurs, where A1 is the event that the firstcard is a red ace, A2 is the event that the second card is a 10

or a jack, and A3 is the event that the third card is greater tan

3 but less than 7.

Solution:

1. A1: the first card is a red ace,

2. A2: the second card is a 10 or a jack,

76

3. A3: the third card is greater than 3 but less than 7

P (A1) =2

52, P (A2|A1) =

8

51, P (A3|A1 A2) =

12

50.

and hence, by Multiplicative Rule,

P (A1 A2 A3) = P (A1)P (A2|A1)P (A3|A1 A2) =8

5525.

2.8 Bayes Rule

77

Bayes Rule

The power of Bayes rule is that in many situations where we

want to compute P (A|B) it turns out that it is difficult to doso directly, yet we might have direct information about P (B|A).Bayes rule enables us to compute P (A|B) in terms of P (B|A).

P (A|B) = P (A B)P (B)

=P (B|A)P (A)

P (B)

which is the so-called Bayes Rule.

78

Bayes Theorem

Let A and Ac constitute a partition of the sample space S such

that with P (A) > 0 and P (Ac) > 0, then for any event B in S

such that P (B) > 0,

P (A|B) = P (B|A)P (A)P (B|A)P (A) + P (B|Ac)P (Ac).

79

The denominator P (B) in the equation can be computed,

P (B) =P [(A B) (A B)]=P (A B) + P (A B)=P (A)P (B|A) + P (A)P (B|A) (by multiplicative rule)

80

Example 29

A paint-store chain produces and sells latex and semigloss paint.

Based on long-range sales, the probability that a customer will

purchase latex paint is 0.75. Of those that purchase latex paint,

60% also purchase rollers. But only 30% of semigloss pain buyers

purchase rollers. A randomly selected buyer purchases a roller

and a can of paint. What is the probability that the paint is

latex?

81

Solution:

L ={The customer purchases latex paint.}, P (L) = 0.75S ={The customer purchases semigloss paint.}, P (S) = 0.25R ={The customer purchases roller.}

P (R|L) =0.6P (R|S) =0.3P (R) =P (R|L)P (L) + P (R|S)P (S) = 0.6 0.75+ 0.3 0.25 = 0.525

P (L|R) = P (L R)P (R)

=P (R|L)P (L)

P (R)=

0.6 0.750.6 0.75+ 0.3 0.25 .857

82

The denominator P (R) in the equation can be computed,

P (R) =P [(L R) (S R)]=P (L R) + P (S R)=P (L)P (R|L) + P (S)P (R|S) (by multiplicative rule)

83

Example 30

Suppose that we are interested in diagnosing cancer in patients

who visit a chest clinic.

Let A represent the event Person has cancer

Let B represent the event Person is a smoker

We know the probability of the prior event P (A) = 0.1 on the

basis of past data (10% of patients entering the clinic turn out

to have cancer). We want to compute the probability of the

posterior event P (A|B). It is difficult to find this out directly.84

However, we are likely to know P (B) by considering the percent-

age of patients who smoke, suppose P (B) = 0.5. We are also

likely to know P (B|A) by checking from our record the proportionof smokers among those diagnosed. Suppose P (B|A)=0.8.

We can now use Bayes rule to compute:

P (A|B) = P (A B)P (B)

=P (B|A)P (A)

P (B)=

0.8 0.10.5

Bayes Theorem

Let A1, A2, . . ., Ak constitute a partition of the sample space S

such that P (Ai) > 0 for i = 1,2, . . . , k, then for any event B in S

such that P (B) > 0,

P (Aj|B) =P (B|Aj)P (Aj)ki=1 P (B|Ai)P (Ai)

for i = 1,2, . . . k.

85

Example 31

In a bolt factory 30%, 50%, and 20% of the production is man-

ufactured by machines I, II, and III, respectively. If 4%, 5%,

and 3% of the outputs of these respective machines is defec-

tive, what is the probability that a randomly selected bolt that

is found to be defective is manufactured by machine III?

86

Solution:

P (B3|D) =0.03 0.2

0.04 0.3+ 0.05 0.5+ 0.03 0.2 0.14

D is the event that a random bolt is defective.

Bi is the event that it is manufacture by i, where i = I, II, or III.