binomial and normal distributions - faculty.chicagobooth.edu · a sum of two random variables...

Binomial and normal distributions

Business Statistics 41000

Summer 2019

1

Topics

1. Sums of random variables

2. Binomial distribution

3. Normal distribution

2

Topic: sums of random variables

Sums of random variables are important for two reasons:

1. Because we often care about aggregates and totals (sales, revenue,employees, etc).

2. Because averages are basically sums, and probabilities are basicallyaverages (of dummy variables), when we go to estimateprobabilities, we will end up using sums of random variables a lot.

This second point is the topic of the next lecture. For now, we focus onthe direct case.

3

A sum of two random variables

Suppose X is a random variable denoting the profit from one wager andY is a random variable denoting the profit from another wager.

If we want to consider our total profit, we may consider the randomvariable that is the sum of the two wagers, S = X + Y .

To determine the distribution of S , we must first know the jointdistribution of (X ,Y ).

4


Suppose that (X ,Y ) has the following joint distribution:

-$200 $100 $200

$0 0 19

39

$100 19

29

29

So S can take the values {−200,−100, 100, 200, 300}.

Notice that there are two ways that S can be $200.

5


We can directly determine the distribution of S as:

S

s P(S = s)

-$200 +$0 0

-$200 + $100 19

$100 + $0 19

$100 + $100 or $200 + $0 29 + 3

9 = 59

$200 + $100 29

When determining the distribution of sums of random variables, we loseinformation about individual values and aggregate the probability ofevents giving the same sum.

6

Topic: binomial distribution

A binomial random variable can be constructed as the sum ofindependent Bernoulli random variables.

Familiarity with the binomial distribution eases many practical probabilitycalculations.

7

Sums of Bernoulli RVs

When rolling two dice, what is the probability of rolling two ones?

By independence we can calculate this probability as

P(1, 1) =1

6

(1

6

)=

1

36.

Now with three dice, what is the probability of rolling exactly two 1’s?

8

Sums of Bernoulli RVs (cont’d)

The event A =“rolling a one”, can be described as a Bernoulli randomvariable with p = 1

6 .

We can denote the three independent rolls by writing

Xiiid∼Bernoulli(p), i = 1, 2, 3.

The notation iid is shorthand for “independent and identicallydistributed”.

Determining the probability of rolling exactly two 1’s can be done byconsidering the random variable Y = X1 + X2 + X3 and asking forP(Y = 2).

9

Sums of Bernoulli random variables (cont’d)

Consider the distribution of Y = X1 + X2 + X3.

Y

Event y P(Y = y)

000 0 (1− p)3

001 or 100 or 010 1 (1− p)(1− p)p + p(1− p)(1− p) + (1− p)p(1− p)

011 or 110 or 101 2 (1− p)p2 + p2(1− p) + p(1− p)p

111 3 p3

Remember that for this example p = 16 .

10


Determining the probability of a certain number of successes requiresknowing 1) the probability of each individual success and 2) the numberof ways that number of successes can arise.

Y

Event y P(Y = y)

000 0 (1− p)3

001 or 100 or 010 1 3(1− p)2p

011 or 110 or 101 2 3(1− p)p2

111 3 p3

We find that P(Y = 2) = 3p2(1− p) = 3(1/36)(5/6) = 56(12) = 5

72 .

11


What if we had four rolls, and the probability of success was 13?

0000100001001100001010100110111000011001010111010011101101111111

12


Summing up the probabilities for each of the values of Y , we find:

Y

y P(Y = y)

0 (1− p)4

1 4(1− p)3p2 6(1− p)2p2

3 4(1− p)p3

4 p4

Substituting p = 13 we can now find P(Y = y) for any y = 0, 1, 2, 3, 4.

13

Defintion: N choose y

The number of ways we can arrange y successes among N trials can becalculated efficiently by a computer. We denote this number with aspecial expression.

N choose y

The notation (N

y

)=

N!

(N − y)!y !

designates the number of ways that y items can be assigned to Npossible positions.

This notation can be used to summarize the entries in the previous tablesfor various values of N and y .

14

Definition: Binomial distribution

Binomial distribution

A random variable Y has a binomial distribution with parameters N andp if its probability distribution function is of the form:

p(y) =

(N

y

)py (1− p)N−y

for integer values of y between 0 and N.

15

Example: drunk batter

What is the probability that our alcoholic major-leaguer gets more than 2hits in a game in which he has 5 at bats?

Let X =“number of hits”. We model X as a binomial random variablewith parameters N = 5 and p = 0.316.

X

x P(X = x)

0 (1− p)5

1 5(1− p)4p2 10(1− p)3p2

3 10(1− p)2p3

4 5(1− p)p4

5 p5

Substituting p = 0.316 we calculate P(X > 2) = 0.185.

16

Variance of binomial random variable

Variance of a binomial random variable

A binomial random variable X with parameters N and p has variance

V(X ) = Np(1− p).

17

Variance of a sum of independent random variables

A useful fact:

Variance of linear combinations of independent random variables

A weighted sum/difference of random variables Y =∑m

i aiXi can beexpressed as

V(Y ) =m∑i

a2i V(Xi ).

How can this be used to derive the expression for the variance of abinomial random variable?

18

Variance of a proportion

By dividing through by the total number of babies born each week wecan consider the proportion of girl babies. Define the random variables

P1 =X

N1and P2 =

Y

N2.

Then it follows that

V (P1) =V(X )

N21

=N1p(1− p)

N21

= p(1− p)/N1

and

V (P2) =V(Y )

N22

=N2p(1− p)

N22

= p(1− p)/N2.

19

Law of Large Numbers

An arithmetical average of random variables is itself a random variable.

As more and more individual random variables are averaged up, thevariance decreases but the mean stays the same.

As a result, the distribution of the averaged random variable becomesmore and more concentrated around its expected value.

20


0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.00

0.05

0.10

0.15

0.20

0.25

Distribution of sample proportion (N = 10, p = 0.7)

21


0.00

0.05

0.10

0.15


0 0.7 1

22


0.00

0.02

0.04

0.06

0.08

0.10

0.12


0 0.7 1

23


0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.07


0 0.7 1

24


0.00

0.01

0.02

0.03

0.04

0.05


0 0.7 1

25

Example: Schlitz Super Bowl taste test

26

Bell curve approximation to binomial

The binomial distributions can be approximated by a smooth densityfunction for large N.

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

0.00

0.05

0.10

0.15

0.20

Normal approximation for binomial distribution with N = 20, p = 0.5

x

Pro

babi

lity

mas

s / D

ensi

ty

27


0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

0.00

0.05

0.10

0.15


x

Pro

babi

lity

mas

s / D

ensi

ty

28


340 346 352 358 364 370 376 382 388 394 400 406 412 418 424 430 436 442 448 454 460

0.00

0.01

0.02

0.03

0.04


x

Pro

babi

lity

mas

s / D

ensi

ty

What are some reasons that very small p or small N lead to badapproximations?

29

Central limit theorem

The normal distribution can be “justified” via its relationship to thebinomial distribution. Roughly: if a random outcome is the combinedresult of many individual random events, its distribution will follow anormal curve.

The quincunx or Galton box is a device which physically simulates sucha scenario using ball bearings and pins stuck in a board.

PLAY VIDEO

The CLT can be stated more precisely, but the practical impact is justthis: random variables which arise as sums of many other randomvariables (not necessarily bell shaped) tend to be bell shaped.

30

http://faculty.chicagobooth.edu/richard.hahn/teaching/quincunx.html

Topic: continuous random variables

When measurements/observations can take on continuous (rather thandiscrete) quantities, we have to assign probabilities to ranges of outcomesinstead of individual outcomes.

Example: spinning a wheel of fortune.

31

Discrete quantities

So far we have focused on random variables which can take on a discreteset of values:

I number of hits in a game,

I number of wins in a series,

I number of babies born,

I temperature (in units of one degree).

32

Continuous quantities

It is useful to consider continuous quantities which can, in principle, bemeasured with arbitrary precision: between any two values we can alwaysfind yet another value. Examples are

I temperature,

I height,

I annual rainfall,

I speed of a fastball,

I blood alcohol level.

33

Example: NBA player heights

Consider the heights of NBA players measured with perfect precision.Assume the heights can be any value between 60 and 100 inches.

60 70 80 90 100

0.00

0.02

0.04

0.06

0.08

NBA heights

Height in Inches

Pro

babi

lity

dens

ity

Trick question: what is the probability that a randomly selectedhypothetical player is 6 foot 8 inches tall?

34

Remark: events with probability zero happen

In fact, for a continuous random variable X , the probability of taking onany specific value is zero!

Instead, we associate probabilities to intervals. The probability of Xtaking some value in a given region is equal to the area under the curveover that region.

35

Probability is assigned to intervals

For example, the area of the highlighted region corresponds toP(72 < X < 84) where X =“height of NBA player”.

60 70 80 90 100

0.00

0.02

0.04

0.06

0.08

NBA heights

Height in Inches

Pro

babi

lity

dens

ity

The curve describing the distribution of probability mass is called theprobability density function.

36

Definition: density functions

A density function must satisfy two properties in order to yield validprobabilities for any collection of intervals.

Probability density function

A function f (x) is a probability density function if it satisfies thefollowing two properties:

I f (x) ≥ 0 for any x ,

I the total area under the curve defined by f (x) is equal to one.

37

Example: uniform distributionConsider randomly selected points between 0 and 1/2 where any point isas likely as any other. What must be the height of the density?

?

0 !

Hint: we know the total area must sum to one. 38

Example: uniform distribution (cont’d)

In simple cases, determining probabilities amounts to calculating areas ofgeometric shapes with known formulas. The highlighted region depictsthe set A =

{x | 28 ≤ x ≤ 3

8

}.

-0.5 0.0 0.5 1.0

0.0

0.5

1.0

1.5

2.0

x

f(x)

What is P(A)?39

Example: uniform distribution (cont’d)

We can consider disjoint intervals as well, in which case the probabilitiesadd.

-0.5 0.0 0.5 1.0

0.0

0.5

1.0

1.5

2.0

x

f(x)

The set not-A is shaded. It is straightforward to confirm thatP(not-A) = 1− P(A) as it should.

40

Example: triangular distribution

The shaded region here is the set B = {x | 0.2 ≤ x ≤ 0.4}.

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

x

f(x)

What is P(B)? Recall that the area of a triangle is 12 (base × height).

41

Normal distributions

The normal family of densities has two parameters, typically denoted µand σ2, which govern the location and scale, respectively.

-4 -2 0 2 4

0.0

0.1

0.2

0.3

0.4

Gaussian densities for various location parameters

x

f(x)

42

Normal distributions (cont’d)

I will use the terms normal distribution, normal density and normalrandom variable more or less interchangeably.

-4 -2 0 2 4

0.0

0.2

0.4

0.6

0.8

Mean-zero Gaussian densities with differing scale parameters

x

f(x)

The normal distribution is also called the Gaussian distribution or thebell curve.

43

Normal means and variances

Mean and variance of a normal random variable

A normal random variable X , with parameters µ and σ2, is denoted

X ∼ N(µ, σ2).

The mean and variance of X are

E (X ) = µ,

V (X ) = σ2.

The density function is symmetric and unimodal, so the median andmode of X are also given by the location parameter µ. The standarddeviation of X is given by σ.

44

Normal approximation to binomial

The binomial distributions can be approximated by a normal distribution.

Normal approximation to the binomial

A Bin(N, p) distribution can be approximated by a N(Np,Np(1− p))distribution for N “large enough”.

Notice that this just “matches” the mean and variance of the twodistributions.

45

Linear transformation of normal RVs

We can add a fixed number to a normal random variable and/or multiplyit by a fixed number and get a new normal random variable. This sort ofoperation is called a linear transformation.

Linear transformation of normal random variables

If X ∼ N(µ, σ2) and Y = a + bX for fixed numbers a and b, thenY ∼ N(a + bµ, b2σ2).

For example, if X ∼ N(1, 2) and Y = 3− 5X , then Y ∼ N(−2, 50).

46

Standard normal RV

Standard normal

A standard normal random variable is one with mean 0 and variance 1.It is often denoted by the letter Z :

Z ∼ N(0, 1).

We can write any normal random variable as a linear transformation of astandard normal RV. For normal random variable X ∼ N(µ, σ2), we canwrite

X = µ+ σZ .

47

The “empirical rule”

It is convenient to characterize where the “bulk” of the probability massof a normal distribution resides by providing an interval, in terms ofstandard deviations, about the mean.

0.0

0.1

0.2

0.3

0.4

N(µ,σ)

x

Density

µ − 4σ µ − 3σ µ − 2σ µ − σ µ µ + σ µ + 2σ µ + 3σ µ + 4σ

68 %

48

The “empirical rule” (cont’d)

The widespread application of the normal distribution has lead this to bedubbed the empirical rule.

0.0

0.1

0.2

0.3

0.4

N(µ,σ)

x

Density


95 %

49


It is, for obvious reasons, sometimes called the 68-95-99.7 rule.0.0

0.1

0.2

0.3

0.4

N(µ,σ)

x

Density


99.7 %

50


To revisit some earlier examples:

I 68% of Chicago daily highs in the winter season are between 19 and48 degrees.

I 95% of NBA players are between 6ft and 7ft 2in.

I In 99.7% of weeks, the proportion of baby girls born at City Generalis between 0.4985 and 0.5015.

51

Sums of normal random variables

Weighted sums of normal random variables are also normally distributed.

For example if

X1 ∼ N(5, 20) and X2 ∼ N(1, 0.5)

then for Y = 0.1X1 + 0.9X2

Y ∼ N(m, v).

where m = 0.1(5) + 0.9(1) = 1.4 and v = 0.12(20) + 0.92(0.5) = 0.605.

52

Linear combinations of normal RVs

Linear combinations of independent normal random variables

For i = 1, . . . , n, let

Xiiid∼N(µi , σ

2i ).

Define Y =∑n

i=1 aiXi for weights a1, a2, . . . , an. Then

Y ∼ N(m, v)

where

m =n∑

i=1

aiµi and v =n∑

i=1

a2i σ2i .

53

Example: two-stock portfolio

Consider two stocks, A and B, with annual returns (in percent ofinvestment) distributed according to normal distributions

XA ∼ N(5, 20) and XB ∼ N(1, 0.5).

What fraction of our investment should we put into stock A, with theremainder put in stock B?

54

Example: two-stock portfolio (cont’d)

For a given fraction α, the total return on our portfolio is

Y = αXA + (1− α)XB

with distribution

Y ∼ N(m, v).

where m = 5α + (1− α) and v = 20α2 + 0.5(1− α)2.

55


Suppose we want to find α so that P(Y ≤ 0) is as small as possible.

-5 0 5 10 15 20

0.0

0.1

0.2

0.3

0.4

0.5

0.6

Two-stock portfolio

Percent return

Density

Stock AStock B

The blue distributions correspond to varying values of α.56


We can plot the probability of a loss as a function of α.0.04

0.06

0.08

0.10

0.12

Probability of a loss

α

Probability

0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00

We see that this probability is minimized when α = 11% approximately.This is the LLN at work!

57

Variance of a sum of correlated random variables

For correlated (dependent) random variables, we have a modified formula:

Variance of linear combinations of two correlated random variables

A weighted sum/difference of random variables Y = a1X1 + a2X2 can beexpressed as

V(Y ) = a21V(X1) + a22V(X2) + 2a1a2Cov(X1,X2).

There is a homework problem that asks you to find the variance ofportfolios of stocks, as in the example above, for stocks which are relatedto one another (in a common industry, for example).

58

binomial and normal distributions - faculty.chicagobooth.edu · a sum of two random variables...

Documents