chapter 3 discrete probability distributions_2.docx

5/26/2018 Chapter 3 Discrete Probability Distributions_2.docx

1/

Chapter 3 Discrete Probability Distributions

Chapter Outline

Section 3.1: Discrete Random Variables Section 3.2: Terminologies in Discrete Random Variables

Probability Mass Function (pmf) Cumulative Distribution Function (cdf) Expected Value and Variance ( and 2)

Section 3.3: Binomial Distribution Permutations and Combinations ( nPkand nCk ) pmf and cdf of Binomial Distribution

Section 3.4: Poisson Distribution Section 3.5: Poisson Approximation to the Binomial Distribution Section 3.6: Practice Problems

Why Discrete Probability Distribution? Flip a coin 50 times. Observe the number of heads.

What are the probabilities of obtaining 0, 1, 2, ... 50 heads?

An average of 2 holes in 100m2 of paper production. A production of 1000m2of paper.What are the probabilities of obtaining 0, 1, 2,... holes?

Soccer matches between teams A and B. Expected scores between teams A and B are 3:2.What is the probability that the actual score is 1:0 ?

Section 3.1 Discrete Random Variables

I. Random Variable

A Random Variable (RV) is a numeric quantity that takes different values with specified probabilities Two types of random variables: Discrete and Continuous

A random variable for which there exists a discrete set of numeric values is a discrete random variable A random variable which can take a continuous range of value is a continuous random variable.

II. Discrete Random Variable

Discrete random variable: Takes a discrete set of numeric values.

Example 1:

Roll a fair die once

Possible outcomes would be {1,2,... , 6}, which is a set of discrete value. Let X be the value of the outcome. X is a discrete random variable

Pr(X=1) = Pr(X=2) = =Pr(X=6) = 1/6


2/

III. Continuous Random Variable

Continuous Random Variable: takes any value on a continuum (with an uncountable number of values)

Example 2:

Let the random variable Z be the weight of an individual.

Z is a continuous random variable taking any positive real numbers.Pr(1


3/

II. Cumulative Distribution Function

Cumulative Distribution Function (cdf) of a random variable X at value x is the probability that X is less than or equal t

the value x.

Notation: F(x) = Pr(X x). Properties: F(x) is an increasing function from 0 to 1.

Note:

The cdf looks like a series of steps, called the step function.

With the increase in the number of values, the cdf will get smoother and smoother

For a continuous random variable, the cdf is a smooth curve.

Example 3 (Continued):

Toss two fair coins.

Let X be the number of heads observed.

Compute the cumulative distribution function

Solutions:

pmf:

{ The cdf is therefore: {


4/

III. Expected Value

A random variable with many different values

Hard to describe the random variable Summarize the random variable based on some location and spread parameters.

Expected value of X: is the sum of product of all possible values with their corresponding probabilities:

where x1, , xnare the possible values X can take.

Note:

is also known as the mean or the population mean.

Expected value is the analog of the arithmetic meanof a sample, as it represents the average value of the randomvariable

IV. Variance

Variance of X: is the sum of squares of all possible values of xi - with their corresponding probabilities:

2is also called the population variance.

Alternative Formula:

where x1, , xnare the possible values X can take.

Note:

Variance of a random variable X is the analog of the sample variance (s2) of a sample.


5/

V. Expected Value and Variance


X = # of heads from tossing 2 fair coins.

What is the expected value and variance of X?

{ Solutions:

Intuition: On average, we expect 1 head from tossing 2 fair coins.

Alternative formula:

Example:

Consider a discrete random variable X with probability distribution given by

a) Show that k=0.1b) Compute E(X), E(X2) and Var(1-4X)

c) Sketch the CDF of X.


6/

Section 3.3 Binomial DistributionI. Introduction

Binomial distribution: Probability distribution of the number of successes in n independent experiments, each yields a

probability of success p.

Derivation of Binomial Distribution:1) Permutations (rearrange objects and values)

2) Factorial

3) Combinations (rearrange objects and values, ordering doesnt matter)

4) Binomial Distribution

II. Permutation

Example 4:

Consider a container with 3 balls (Numbered 1,2 and 3).

Two balls are drawn at random without replacement.

Possible pairs: (1,2), (1,3), (2,3) , (2,1), (3,1), (3,2)

Does the order selected matter in terms of winning?

If the order matters, then there are 6 possible outcomes, => permutations If the order does not matter, then there are 3 possible outcomes, => combinations

Questions:

Select k objects be selected out of n objects (0kn)

How many ways can we select, if the objects are selected without replacement?

Answer:

There are

If we denote nPk to be the number of permutations of n objects taken k at a time, then


7/

III. Permutations and Factorial

Define the term n factorial (n!) as a product of n terms:

n!= n x (n - 1) x (n - 2) x x 1

Example 5:5! = (5)(4)(3)(2)(1)=120.

Permutation nPk and Factorial:

IV. Combination

Ordering doesnt matter => Usecombination but not permutationDenote nCk (or (), pronounce as n choose k) to be the number of combinations of n objects taken k at a time

It represents the number of ways of selecting k objects out of n where the order of selection does not matter.

Example 6:

Evaluate 3P2and 3C2

Solutions:

Examples 7:

Consider a lottery from a container with 3 balls (numbering 1,2 and 3).

(a) What is the # of permutations if two balls are drawn at random without replacement?

(b) What is the # of combinations if two balls are drawn at random without replacement?

Solutions:

Let n=3 balls and k=2 balls drawn from the container,


8/1

V. Setup of the Binomial Distribution

Examples 8:

Toss a fair coin 10 times

Let X be the total number of heads observed.

What is the distribution of X?

Answer: Binomial Distribution!

General Setup of the Binomial Distribution:

If we

(1) record a fixed number of experiments n,

(2) Probability for the event to happen in each experiment is constant at p.

If X is the total number of events to happen, then

X follows Binomial Distribution with parameters n and p

VI. Probability Mass Function of Binomial Distribution

The probability mass function (pmf) of the Binomial distribution with parameters n and p is given by

Therefore, # of possible outcomes =()


9/1

VII. Binomial Distribution

Example 9:

Toss a fair coin twice and let X be the number of heads.

Find the probability of obtaining x=0,1 and 2 heads.

Solutions:

n=2 and p=0.5. Hence

Therefore,

Example 10:

What is the probability of obtaining exactly one success in 10 experiments, if the probability of success is 0.1?

Solutions: n=10 and p=0.1. Therefore

The above probability is pretty large: On average, one would expect to obtain only one success out of the 10

experiments.

In fact, X=1 is the mode of the distribution, followed by the X=0 as follows:

VIII. White Blood Cells Count

White blood cells: Cells of immune system to defend the body against both infectious disease and foreign material

Major types of White Blood Cells

Neutrophil: 70% (most abundant) Eosinophil Basophil Monocyte Lymphocyte

High number of neutrophils counts: Possible bacterial infection/acute viral infections.

A sample of blood is taken => # of neutrophils follow Binomial distribution


10

Example 11:

Assume that 70% of white blood cells are neutrophils from a healthy person. If 10 white blood cells are examined,

what is the probability mass function of the number of neutrophils?

Solutions:

Let X be the number of neutrophils. Then

X ~Binomial(n=10, p=0.70), with pmf

Note: Pr(X=10) very small at 0.028.

If 10 neutrophils are observed from the sample => Indication on bacterial infection (Chapter 7)

IX. Binomial Distribution --- Shape of the Binomial Distribution

1. Binomial Distribution is Symmetric when p=0.5

Binomial distribution with p=0.5 and various values of n:

Binomial distribution is symmetric when p=0.5. That is,

Pr(X=k) = Pr(X=n-k) for k=0, 1, 2,... N

2. When p0.5, the distribution is left-skewed.

Binomial Distribution with n=4 Binomial Distribution with n=10


11

X. Properties of the Binomial Distribution

Mean = =E(X) = np

Intuition: The expected number of successes in n trials is the probability of success in one trial (p) multiplied by n

Variance =2 = np(1- p) = npq ,where q = 1- p is the probability of failure.

Note: For fixed value of n, 2is maximized when p=0.5

Standard Deviation: = = VI. Variance of the Binomial Distribution

2 = np(1-p) = n[]which is a concave function in p and attains its maximum at p=0.5.

Interpretation:

1) When p=0 or 1: 2 =0 as there is no uncertainty on the outcome (Why?)

2) When p=1/2: the pmf spreads out widely across x=0 to x=n. Hence variability is the largest.

VII: Why Probability Distribution?

Covered in Chapter 1:

Suppose that there are 1,000 students in a class. Eachof them flip the same coin 10 times and record the number of

heads each students observed. Below is the frequency table of the number of heads observed by each student:

Question: Is the coin a fair one? (That is, is p=Pr(Head)=0.5?)


12

Covered in Chapter 3:

Let X be the number of heads observed by flipping a coin 10times. Then X~Binomial(n=10, p)

Below is the probability mass function for p =0.1, 0.3, 0.5, 0.7 and 0.9.

We can then compare the histogram on the previous page with the pmf below to make a guess on what is the true p

Covered in Later Chapter:

Chapter 6 EstimationBased on the data, try to

(1) Find the best estimate of the unknown probability p,

(2) Find the range of probability p that is likely to generate the data.

Example: 90% of chance p is between 0.6 to 0.9.

Chapter 7-8 Hypothesis Testing

Question: Is the coin fair?

Conclusion: We have 95% confidence that the probability of head p is larger than 0.50.


13

Section 3.4 Poisson DistributionI. Poisson Distribution

The second most frequently used discrete distribution Poisson distribution is usually associated with rare events.

Example:

Of every 10m2 of paper being produced, we expect to observe an average of one surface defect.Let X be the number of surface defects per 100m2 of production. What is the distribution of X?

Answer: Poisson Distribution

II. Probability Mass Function of Poisson Distribution

Assume that events occur independently with each other.

The probability of k events occurring for a Poissonrandom variable with parameter is

Where e2.71828 is the Eulers constant, and is expected number of events to occur

Notation:

Example 12:

Assume # of bacterial colonies is 0.02 per cm2. For an area of 100cm2, find the probability distribution of the

number of bacterial colonies.

Solutions:

The expected number of bacterial colonies per 100cm2 is given by = 0.02(100) = 2 (Colonies)

Let X be the number of colonies in 100cm2, then X~Poisson(=2)

Probability Mass Function:


14


Now, suppose we are interested in a larger area of 200cm2. Whats the probability distribution of the number of

bacterial colonies?

Solutions: =0.02(200) = 4(colonies). Let Y be the number of colonies in 200cm2, then Y~Poisson(=4)

Probability Mass Function:

III. Binomial Distribution vs Poisson Distribution

Binomial distribution: Number of trials n is finite, and the number of events cannot be larger than n.

Poisson distribution: Number of trials is essentially infinite and the number of events can be indefinitely large.

However, probability of k events becomes very small as k increases.

Examples of Binomial Distribution:

Flip a coin n times

Number of neutrophils out of n White Blood Cells

Example of Poisson Distribution:

Electrical system: telephone calls arriving in a system.

Astronomy: photons arriving at a telescope.

Biology: the number of mutations on a strand of DNA per unit time.

Management: customers arriving at a counter or call centre.

Civil Engineering: cars arriving at a traffic light.

Finance and Insurance: Number of Losses/Claims occurring in a given period of time.


15

IV. Shape of the Poisson Distribution

Poisson Distribution is a right-skewed distribution When is small (e.g. =0.8), the distribution is heavily skewed to the right. As increases (e.g. =12), the distribution becomes more symmetric, even though its still slightly right-skew

V. Expected Value and Variance of Poisson Distribution

Suppose that X~Poisson ( ), that is,

Then the expected value and the variance of X are given by

Section 3.5 Poisson Approximation to the Binomial Distribution

I. Introduction

Binomial distribution: When n is large and p is small, we can think of observing rare events in a large number of trials

that case, the binomial distribution can be well approximated by the Poisson distribution, with=np.

Rule of Thumb:

Let X~Binomial(n,p). When n20, p

chapter 3 discrete probability distributions_2.docx

Documents

random variable x

discrete random variableprx

random variable z

discrete random variablesi

random variable rv

set of discrete value

types of random variables

value x