dave goldsman - gatech.edu

4. Distributions

Dave Goldsman

H. Milton Stewart School of Industrial and Systems EngineeringGeorgia Institute of Technology

8/5/20

ISYE 6739 — Goldsman 8/5/20 1 / 108

Outline

1 Bernoulli and Binomial Distributions2 Hypergeometric Distribution3 Geometric and Negative Binomial Distributions4 Poisson Distribution5 Uniform, Exponential, and Friends6 Other Continuous Distributions7 Normal Distribution: Basics8 Standard Normal Distribution9 Sample Mean of Normals

10 The Central Limit Theorem + Proof11 Central Limit Theorem Examples12 Extensions — Multivariate Normal Distribution13 Extensions — Lognormal Distribution14 Computer Stuff

ISYE 6739 — Goldsman 8/5/20 2 / 108

Bernoulli and Binomial Distributions

Lesson 4.1 — Bernoulli and Binomial Distributions

Goal: We’ll discuss lots of interesting distributions in this module.

The module will be a compendium of results, some of which we’ll prove, andsome of which we’ve already seen previously.

Special emphasis will be placed on the Normal distribution, because it’s soimportant and has so many implications, including the Central LimitTheorem.

In the next few lessons we’ll discuss some important discrete distributions:

Hypergeometric Distribution

Geometric and Negative Binomial Distributions

Poisson Distribution

ISYE 6739 — Goldsman 8/5/20 3 / 108

Definition: The Bernoulli distribution with parameter p is given by

{1 w.p. p (“success”)

0 w.p. q (“failure”)

Recall: E[X] = p, Var(X) = pq, and MX(t) = pet + q.

Definition: The Binomial distribution with parameters n and p is givenby

P (Y = k) =

)pkqn−k, k = 0, 1, . . . , n.

ISYE 6739 — Goldsman 8/5/20 4 / 108

P (Y = k) =

)pkqn−k, k = 0, 1, . . . , n.

ISYE 6739 — Goldsman 8/5/20 4 / 108

P (Y = k) =

)pkqn−k, k = 0, 1, . . . , n.

ISYE 6739 — Goldsman 8/5/20 4 / 108

Example: Toss 2 dice and take the sum; repeat 5 times. Let Y be the numberof 7’s you see.

Y ∼ Bin(5, 1/6). Then, e.g.,

P (Y = 4) =

)5−4. 2

Theorem: X1, . . . , Xniid∼ Bern(p)⇒ Y ≡

∑ni=1Xi ∼ Bin(n, p).

Proof: This kind of result can easily be proved by a moment generatingfunction uniqueness argument, as we mentioned in previous modules. 2

Think of the Binomial as the number of successes from n Bern(p) trials.

ISYE 6739 — Goldsman 8/5/20 5 / 108

Example: Toss 2 dice and take the sum; repeat 5 times. Let Y be the numberof 7’s you see. Y ∼ Bin(5, 1/6). Then, e.g.,

P (Y = 4) =

)5−4. 2

ISYE 6739 — Goldsman 8/5/20 5 / 108

P (Y = 4) =

)5−4. 2

ISYE 6739 — Goldsman 8/5/20 5 / 108

P (Y = 4) =

)5−4. 2

ISYE 6739 — Goldsman 8/5/20 5 / 108

P (Y = 4) =

)5−4. 2

ISYE 6739 — Goldsman 8/5/20 5 / 108

P (Y = 4) =

)5−4. 2

ISYE 6739 — Goldsman 8/5/20 5 / 108

Theorem: Y ∼ Bin(n, p) implies

E[Y ] = E[ n∑i=1

n∑i=1

E[Xi] = np.

Similarly,Var(Y ) = npq.

We’ve already seen that

MY (t) = (pet + q)n.

Theorem: Certain Binomials add up: If Y1, . . . , Yk are independent andYi ∼ Bin(ni, p), then

k∑i=1

Yi ∼ Bin( k∑i=1

ni, p).

ISYE 6739 — Goldsman 8/5/20 6 / 108

E[Y ] = E[ n∑i=1

n∑i=1

E[Xi] = np.

k∑i=1

Yi ∼ Bin( k∑i=1

ni, p).

ISYE 6739 — Goldsman 8/5/20 6 / 108

E[Y ] = E[ n∑i=1

n∑i=1

E[Xi] = np.

k∑i=1

Yi ∼ Bin( k∑i=1

ni, p).

ISYE 6739 — Goldsman 8/5/20 6 / 108

E[Y ] = E[ n∑i=1

n∑i=1

E[Xi] = np.

Theorem: Certain Binomials add up: If Y1, . . . , Yk are independent andYi ∼ Bin(ni, p),

k∑i=1

Yi ∼ Bin( k∑i=1

ni, p).

ISYE 6739 — Goldsman 8/5/20 6 / 108

E[Y ] = E[ n∑i=1

n∑i=1

E[Xi] = np.

k∑i=1

Yi ∼ Bin( k∑i=1

ni, p).

ISYE 6739 — Goldsman 8/5/20 6 / 108

1 Bernoulli and Binomial Distributions

2 Hypergeometric Distribution3 Geometric and Negative Binomial Distributions

4 Poisson Distribution5 Uniform, Exponential, and Friends6 Other Continuous Distributions7 Normal Distribution: Basics8 Standard Normal Distribution9 Sample Mean of Normals

10 The Central Limit Theorem + Proof

11 Central Limit Theorem Examples

12 Extensions — Multivariate Normal Distribution13 Extensions — Lognormal Distribution

14 Computer Stuff

ISYE 6739 — Goldsman 8/5/20 7 / 108

Lesson 4.2 — Hypergeometric Distribution

Definition: You have a objects of type 1 and b objects of type 2.

Select n objects without replacement from the a+ b.

Let X be the number of type 1’s selected. Then X has the Hypergeometricdistribution with pmf

P (X = k) =

n−k)(

) , k = 0, 1, . . . ,min(a, n− b, n).

Example: 25 sox in a box. 15 red, 10 blue. Pick 7 w/o replacement.

P (exactly 3 reds are picked) =

ISYE 6739 — Goldsman 8/5/20 8 / 108

P (X = k) =

n−k)(

) , k = 0, 1, . . . ,min(a, n− b, n).

ISYE 6739 — Goldsman 8/5/20 8 / 108

P (X = k) =

n−k)(

) , k = 0, 1, . . . ,min(a, n− b, n).

ISYE 6739 — Goldsman 8/5/20 8 / 108

P (X = k) =

n−k)(

) , k = 0, 1, . . . ,min(a, n− b, n).

ISYE 6739 — Goldsman 8/5/20 8 / 108

P (X = k) =

n−k)(

) , k = 0, 1, . . . ,min(a, n− b, n).

ISYE 6739 — Goldsman 8/5/20 8 / 108

P (X = k) =

n−k)(

) , k = 0, 1, . . . ,min(a, n− b, n).

ISYE 6739 — Goldsman 8/5/20 8 / 108

P (X = k) =

n−k)(

) , k = 0, 1, . . . ,min(a, n− b, n).

ISYE 6739 — Goldsman 8/5/20 8 / 108

Theorem: After some algebra, it turns out that

E[X] = n

Var(X) = n

)(1− a

)(a+ b− na+ b− 1

Remark: Here, aa+b here plays the role of p in the Binomial distribution.

And then the corresponding Y ∼ Bin(n, p) results would be

E[Y ] = n

Var(Y ) = n

)(1− a

So same mean as the Hypergeometric, but slightly larger variance.

ISYE 6739 — Goldsman 8/5/20 9 / 108

E[X] = n

Var(X) = n

)(1− a

)(a+ b− na+ b− 1

E[Y ] = n

Var(Y ) = n

)(1− a

ISYE 6739 — Goldsman 8/5/20 9 / 108

E[X] = n

Var(X) = n

)(1− a

)(a+ b− na+ b− 1

E[Y ] = n

Var(Y ) = n

)(1− a

ISYE 6739 — Goldsman 8/5/20 9 / 108

E[X] = n

Var(X) = n

)(1− a

)(a+ b− na+ b− 1

E[Y ] = n

Var(Y ) = n

)(1− a

ISYE 6739 — Goldsman 8/5/20 9 / 108

E[X] = n

Var(X) = n

)(1− a

)(a+ b− na+ b− 1

E[Y ] = n

Var(Y ) = n

)(1− a

ISYE 6739 — Goldsman 8/5/20 9 / 108

E[X] = n

Var(X) = n

)(1− a

)(a+ b− na+ b− 1

E[Y ] = n

Var(Y ) = n

)(1− a

So same mean as the Hypergeometric, but slightly larger variance.ISYE 6739 — Goldsman 8/5/20 9 / 108

14 Computer Stuff

ISYE 6739 — Goldsman 8/5/20 10 / 108

Lesson 4.3 — Geometric and Negative Binomial Distributions

Definition: Suppose we consider an infinite sequence of independentBern(p) trials.

Let Z equal the number of trials until the first success is obtained. The eventZ = k corresponds to k − 1 failures, and then a success. Thus,

P (Z = k) = qk−1p, k = 1, 2, . . . ,

and we say that Z has the Geometric distribution with parameter p.

Notation: X ∼ Geom(p).

We’ll get the mean and variance of the Geometric via the mgf. . . .

ISYE 6739 — Goldsman 8/5/20 11 / 108

P (Z = k) = qk−1p, k = 1, 2, . . . ,

ISYE 6739 — Goldsman 8/5/20 11 / 108

P (Z = k) = qk−1p, k = 1, 2, . . . ,

ISYE 6739 — Goldsman 8/5/20 11 / 108

P (Z = k) = qk−1p, k = 1, 2, . . . ,

ISYE 6739 — Goldsman 8/5/20 11 / 108

P (Z = k) = qk−1p, k = 1, 2, . . . ,

ISYE 6739 — Goldsman 8/5/20 11 / 108

P (Z = k) = qk−1p, k = 1, 2, . . . ,

ISYE 6739 — Goldsman 8/5/20 11 / 108

P (Z = k) = qk−1p, k = 1, 2, . . . ,

ISYE 6739 — Goldsman 8/5/20 11 / 108

Theorem: The mgf of the Geom(p) is

MZ(t) =pet

1− qet, for t < `n(1/q).

Proof:

MZ(t) = E[etZ ]

∞∑k=1

etkqk−1p

= pet∞∑k=0

(qet)k

1− qet, for qet < 1. 2

ISYE 6739 — Goldsman 8/5/20 12 / 108

MZ(t) =pet

1− qet, for t < `n(1/q).

Proof:

MZ(t) = E[etZ ]

∞∑k=1

etkqk−1p

= pet∞∑k=0

(qet)k

ISYE 6739 — Goldsman 8/5/20 12 / 108

MZ(t) =pet

1− qet, for t < `n(1/q).

Proof:

MZ(t) = E[etZ ]

∞∑k=1

etkqk−1p

= pet∞∑k=0

(qet)k

ISYE 6739 — Goldsman 8/5/20 12 / 108

MZ(t) =pet

1− qet, for t < `n(1/q).

Proof:

MZ(t) = E[etZ ]

∞∑k=1

etkqk−1p

= pet∞∑k=0

(qet)k

ISYE 6739 — Goldsman 8/5/20 12 / 108

MZ(t) =pet

1− qet, for t < `n(1/q).

Proof:

MZ(t) = E[etZ ]

∞∑k=1

etkqk−1p

= pet∞∑k=0

(qet)k

ISYE 6739 — Goldsman 8/5/20 12 / 108

Corollary: E[Z] = 1/p.

Proof:

E[Z] =d

dtMZ(t)

∣∣∣t=0

=(1− qet)(pet)− (−qet)(pet)

(1− qet)2∣∣∣t=0

(1− q)2=

Remark: We could also have proven this directly from the definition ofexpected value.

ISYE 6739 — Goldsman 8/5/20 13 / 108

Proof:

E[Z] =d

dtMZ(t)

∣∣∣t=0

(1− qet)2∣∣∣t=0

(1− q)2=

ISYE 6739 — Goldsman 8/5/20 13 / 108

Proof:

E[Z] =d

dtMZ(t)

∣∣∣t=0

(1− qet)2∣∣∣t=0

(1− q)2=

ISYE 6739 — Goldsman 8/5/20 13 / 108

Proof:

E[Z] =d

dtMZ(t)

∣∣∣t=0

(1− qet)2∣∣∣t=0

(1− q)2=

ISYE 6739 — Goldsman 8/5/20 13 / 108

Proof:

E[Z] =d

dtMZ(t)

∣∣∣t=0

(1− qet)2∣∣∣t=0

(1− q)2=

ISYE 6739 — Goldsman 8/5/20 13 / 108

Proof:

E[Z] =d

dtMZ(t)

∣∣∣t=0

(1− qet)2∣∣∣t=0

(1− q)2=

ISYE 6739 — Goldsman 8/5/20 13 / 108

Similarly, after a lot of algebra,

E[Z2] =d2

dt2MZ(t)

∣∣∣t=0

=2− pp2

and thenVar(Z) = E[Z2]− (E[Z])2 = q/p2. 2

Example: Toss a die repeatedly. What’s the probability that we observe a 3for the first time on the 8th toss?

Answer: The number of tosses we need is Z ∼ Geom(1/6).P (Z = 8) = qk−1p = (5/6)7(1/6). 2

How many tosses would we expect to take?

Answer: E[Z] = 1/p = 6 tosses. 2

ISYE 6739 — Goldsman 8/5/20 14 / 108

E[Z2] =d2

dt2MZ(t)

∣∣∣t=0

=2− pp2

ISYE 6739 — Goldsman 8/5/20 14 / 108

E[Z2] =d2

dt2MZ(t)

∣∣∣t=0

=2− pp2

ISYE 6739 — Goldsman 8/5/20 14 / 108

E[Z2] =d2

dt2MZ(t)

∣∣∣t=0

=2− pp2

ISYE 6739 — Goldsman 8/5/20 14 / 108

E[Z2] =d2

dt2MZ(t)

∣∣∣t=0

=2− pp2

Answer: The number of tosses we need is Z ∼ Geom(1/6).P (Z = 8) = qk−1p

= (5/6)7(1/6). 2

ISYE 6739 — Goldsman 8/5/20 14 / 108

E[Z2] =d2

dt2MZ(t)

∣∣∣t=0

=2− pp2

ISYE 6739 — Goldsman 8/5/20 14 / 108

E[Z2] =d2

dt2MZ(t)

∣∣∣t=0

=2− pp2

ISYE 6739 — Goldsman 8/5/20 14 / 108

E[Z2] =d2

dt2MZ(t)

∣∣∣t=0

=2− pp2

ISYE 6739 — Goldsman 8/5/20 14 / 108

Memoryless Property of Geometric

Theorem: Suppose Z ∼ Geom(p). Then for positive integers s, t, we have

P (Z > s+ t|Z > s) = P (Z > t).

Why is it called the Memoryless Property? If an event hasn’t occurred by times, the probability that it will occur after an additional t time units is the sameas the (unconditional) probability that it will occur after time t — it forgot thatit made it past time s!

ISYE 6739 — Goldsman 8/5/20 15 / 108

P (Z > s+ t|Z > s) = P (Z > t).

ISYE 6739 — Goldsman 8/5/20 15 / 108

P (Z > s+ t|Z > s) = P (Z > t).

Why is it called the Memoryless Property?

If an event hasn’t occurred by times, the probability that it will occur after an additional t time units is the sameas the (unconditional) probability that it will occur after time t — it forgot thatit made it past time s!

ISYE 6739 — Goldsman 8/5/20 15 / 108

P (Z > s+ t|Z > s) = P (Z > t).

ISYE 6739 — Goldsman 8/5/20 15 / 108

Proof: First of all, for any t = 0, 1, 2, . . ., the tail probability is

P (Z > t) = P (t Bern(p) failures in a row) = qt. (∗)

P (Z > s+ t|Z > s) =P (Z > s+ t ∩ Z > s)

P (Z > s)

=P (Z > s+ t)

P (Z > s)

qs(by (∗))

= P (Z > t). 2

ISYE 6739 — Goldsman 8/5/20 16 / 108

P (Z > s+ t|Z > s)

=P (Z > s+ t ∩ Z > s)

P (Z > s)

=P (Z > s+ t)

P (Z > s)

qs(by (∗))

= P (Z > t). 2

ISYE 6739 — Goldsman 8/5/20 16 / 108

P (Z > s+ t|Z > s) =P (Z > s+ t ∩ Z > s)

P (Z > s)

=P (Z > s+ t)

P (Z > s)

qs(by (∗))

= P (Z > t). 2

ISYE 6739 — Goldsman 8/5/20 16 / 108

P (Z > s+ t|Z > s) =P (Z > s+ t ∩ Z > s)

P (Z > s)

=P (Z > s+ t)

P (Z > s)

qs(by (∗))

= P (Z > t). 2

ISYE 6739 — Goldsman 8/5/20 16 / 108

P (Z > s+ t|Z > s) =P (Z > s+ t ∩ Z > s)

P (Z > s)

=P (Z > s+ t)

P (Z > s)

qs(by (∗))

= P (Z > t). 2

ISYE 6739 — Goldsman 8/5/20 16 / 108

P (Z > s+ t|Z > s) =P (Z > s+ t ∩ Z > s)

P (Z > s)

=P (Z > s+ t)

P (Z > s)

qs(by (∗))

= P (Z > t). 2

ISYE 6739 — Goldsman 8/5/20 16 / 108

P (Z > s+ t|Z > s) =P (Z > s+ t ∩ Z > s)

P (Z > s)

=P (Z > s+ t)

P (Z > s)

qs(by (∗))

= P (Z > t). 2

ISYE 6739 — Goldsman 8/5/20 16 / 108

Example: Let’s toss a die until a 5 appears for the first time.

Suppose thatwe’ve already made 4 tosses without success. What’s the probability thatwe’ll need more than 2 additional tosses before we observe a 5?

Let Z be the number of tosses required. By the Memoryless Property (withs = 4 and t = 2) and (∗), we want

P (Z > 6|Z > 4) = P (Z > 2) = (5/6)2. 2

Fun Fact: The Geom(p) is the only discrete distribution with thememoryless property.

Not-as-Fun Fact: Some books define the Geom(p) as the number ofBern(p) failures until you observe a success. # failures = # trials − 1. Youshould be aware of this inconsistency, but don’t worry about it now.

ISYE 6739 — Goldsman 8/5/20 17 / 108

Example: Let’s toss a die until a 5 appears for the first time. Suppose thatwe’ve already made 4 tosses without success.

What’s the probability thatwe’ll need more than 2 additional tosses before we observe a 5?

P (Z > 6|Z > 4) = P (Z > 2) = (5/6)2. 2

ISYE 6739 — Goldsman 8/5/20 17 / 108

Example: Let’s toss a die until a 5 appears for the first time. Suppose thatwe’ve already made 4 tosses without success. What’s the probability thatwe’ll need more than 2 additional tosses before we observe a 5?

P (Z > 6|Z > 4) = P (Z > 2) = (5/6)2. 2

ISYE 6739 — Goldsman 8/5/20 17 / 108

Let Z be the number of tosses required.

By the Memoryless Property (withs = 4 and t = 2) and (∗), we want

P (Z > 6|Z > 4) = P (Z > 2) = (5/6)2. 2

ISYE 6739 — Goldsman 8/5/20 17 / 108

P (Z > 6|Z > 4) = P (Z > 2) = (5/6)2. 2

ISYE 6739 — Goldsman 8/5/20 17 / 108

P (Z > 6|Z > 4) = P (Z > 2) = (5/6)2. 2

ISYE 6739 — Goldsman 8/5/20 17 / 108

P (Z > 6|Z > 4) = P (Z > 2) = (5/6)2. 2

ISYE 6739 — Goldsman 8/5/20 17 / 108

P (Z > 6|Z > 4) = P (Z > 2) = (5/6)2. 2

ISYE 6739 — Goldsman 8/5/20 17 / 108

Now let W equal the number of trials until the rth success is obtained.W = r, r + 1, . . .. The event W = k corresponds to exactly r − 1 successesby time k − 1, and then the rth success at time k.

We say that W has the Negative Binomial distribution (aka the Pascaldistribution) with parameters r and p.

Example: ‘FFFFSFS’ corresponds to W = 7 trials until the r = 2nd success.

Notation: W ∼ NegBin(r, p).

Remark: As with the Geom(p), the exact definition of the NegBin dependson what book you’re reading.

ISYE 6739 — Goldsman 8/5/20 18 / 108

Now let W equal the number of trials until the rth success is obtained.W = r, r + 1, . . ..

The event W = k corresponds to exactly r − 1 successesby time k − 1, and then the rth success at time k.

ISYE 6739 — Goldsman 8/5/20 18 / 108

Theorem: If Z1, . . . , Zriid∼ Geom(p), then W =

∑ri=1 Zi ∼ NegBin(r, p).

In other words, Geometrics add up to a NegBin.

Proof: Won’t do it here, but you can use the mgf technique. 2

Anyhow, it makes sense if you think of Zi as the number of trials after the(i− 1)st success up to and including the ith success.

Since the Zi’s are i.i.d., the above theorem gives:

E[W ] = rE[Zi] = r/p,

Var(W ) = rVar(Zi) = rq/p2,

MW (t) = [MZi(t)]r =

1− qet

ISYE 6739 — Goldsman 8/5/20 19 / 108

E[W ] = rE[Zi] = r/p,

1− qet

ISYE 6739 — Goldsman 8/5/20 19 / 108

E[W ] = rE[Zi] = r/p,

1− qet

ISYE 6739 — Goldsman 8/5/20 19 / 108

E[W ] = rE[Zi] = r/p,

1− qet

ISYE 6739 — Goldsman 8/5/20 19 / 108

E[W ] = rE[Zi] = r/p,

1− qet

ISYE 6739 — Goldsman 8/5/20 19 / 108

E[W ] = rE[Zi] = r/p,

1− qet

ISYE 6739 — Goldsman 8/5/20 19 / 108

E[W ] = rE[Zi] = r/p,

1− qet

ISYE 6739 — Goldsman 8/5/20 19 / 108

E[W ] = rE[Zi] = r/p,

1− qet

ISYE 6739 — Goldsman 8/5/20 19 / 108

Just to be complete, let’s get the pmf of W .

Note that W = k iff you getexactly r − 1 successes by time k − 1, and then the rth success at time k.So. . .

P (W = k) =

[(k − 1

r − 1

)pr−1qk−r

(k − 1

r − 1

)prqk−r, k = r, r + 1, . . .

Example: Toss a die until a 5 appears for the third time. What’s theprobability that we’ll need exactly 7 tosses?

Let W be the number of tosses required. Clearly, W ∼ NegBin(3, 1/6).

P (W = 7) =

(7− 1

3− 1

)(1/6)3(5/6)7−3.

ISYE 6739 — Goldsman 8/5/20 20 / 108

Just to be complete, let’s get the pmf of W . Note that W = k iff you getexactly r − 1 successes by time k − 1, and then the rth success at time k.So. . .

P (W = k) =

[(k − 1

r − 1

)pr−1qk−r

(k − 1

r − 1

)prqk−r, k = r, r + 1, . . .

P (W = 7) =

(7− 1

3− 1

)(1/6)3(5/6)7−3.

ISYE 6739 — Goldsman 8/5/20 20 / 108

P (W = k) =

[(k − 1

r − 1

)pr−1qk−r

(k − 1

r − 1

)prqk−r, k = r, r + 1, . . .

P (W = 7) =

(7− 1

3− 1

)(1/6)3(5/6)7−3.

ISYE 6739 — Goldsman 8/5/20 20 / 108

P (W = k) =

[(k − 1

r − 1

)pr−1qk−r

(k − 1

r − 1

)prqk−r, k = r, r + 1, . . .

P (W = 7) =

(7− 1

3− 1

)(1/6)3(5/6)7−3.

ISYE 6739 — Goldsman 8/5/20 20 / 108

P (W = k) =

[(k − 1

r − 1

)pr−1qk−r

(k − 1

r − 1

)prqk−r, k = r, r + 1, . . .

Example: Toss a die until a 5 appears for the third time.

What’s theprobability that we’ll need exactly 7 tosses?

P (W = 7) =

(7− 1

3− 1

)(1/6)3(5/6)7−3.

ISYE 6739 — Goldsman 8/5/20 20 / 108

P (W = k) =

[(k − 1

r − 1

)pr−1qk−r

(k − 1

r − 1

)prqk−r, k = r, r + 1, . . .

P (W = 7) =

(7− 1

3− 1

)(1/6)3(5/6)7−3.

ISYE 6739 — Goldsman 8/5/20 20 / 108

P (W = k) =

[(k − 1

r − 1

)pr−1qk−r

(k − 1

r − 1

)prqk−r, k = r, r + 1, . . .

P (W = 7) =

(7− 1

3− 1

)(1/6)3(5/6)7−3.

ISYE 6739 — Goldsman 8/5/20 20 / 108

P (W = k) =

[(k − 1

r − 1

)pr−1qk−r

(k − 1

r − 1

)prqk−r, k = r, r + 1, . . .

P (W = 7) =

(7− 1

3− 1

)(1/6)3(5/6)7−3.

ISYE 6739 — Goldsman 8/5/20 20 / 108

How are the Binomial and NegBin Related?

X1, . . . , Xniid∼ Bern(p) ⇒ Y ≡

Z1, . . . , Zriid∼ Geom(p) ⇒ W ≡

E[Y ] = np, Var(Y ) = npq.

E[W ] = r/p, Var(W ) = rq/p2.

ISYE 6739 — Goldsman 8/5/20 21 / 108

X1, . . . , Xniid∼ Bern(p) ⇒ Y ≡

Z1, . . . , Zriid∼ Geom(p) ⇒ W ≡

E[W ] = r/p, Var(W ) = rq/p2.

ISYE 6739 — Goldsman 8/5/20 21 / 108

X1, . . . , Xniid∼ Bern(p) ⇒ Y ≡

Z1, . . . , Zriid∼ Geom(p) ⇒ W ≡

E[W ] = r/p, Var(W ) = rq/p2.

ISYE 6739 — Goldsman 8/5/20 21 / 108

X1, . . . , Xniid∼ Bern(p) ⇒ Y ≡

Z1, . . . , Zriid∼ Geom(p) ⇒ W ≡

E[W ] = r/p, Var(W ) = rq/p2.

ISYE 6739 — Goldsman 8/5/20 21 / 108

X1, . . . , Xniid∼ Bern(p) ⇒ Y ≡

Z1, . . . , Zriid∼ Geom(p) ⇒ W ≡

E[W ] = r/p, Var(W ) = rq/p2.

ISYE 6739 — Goldsman 8/5/20 21 / 108

14 Computer Stuff

ISYE 6739 — Goldsman 8/5/20 22 / 108

Lesson 4.4 — Poisson Distribution

We’ll first talk about Poisson processes.

Let N(t) be a counting process. That is, N(t) is the number ofoccurrences (or arrivals, or events) of some process over the time interval[0, t]. N(t) looks like a step function.

Examples: N(t) could be any of the following.(a) Cars entering a shopping center (by time t).(b) Defects on a wire (of length t).(c) Raisins in cookie dough (of volume t).

Let λ > 0 be the average number of occurrences per unit time (or length orvolume).

In the above examples, we might have:(a) λ = 10/min. (b) λ = 0.5/ft. (c) λ = 4/in3.

ISYE 6739 — Goldsman 8/5/20 23 / 108

Examples: N(t) could be any of the following.

(a) Cars entering a shopping center (by time t).(b) Defects on a wire (of length t).(c) Raisins in cookie dough (of volume t).

ISYE 6739 — Goldsman 8/5/20 23 / 108

Examples: N(t) could be any of the following.(a) Cars entering a shopping center (by time t).

(b) Defects on a wire (of length t).(c) Raisins in cookie dough (of volume t).

ISYE 6739 — Goldsman 8/5/20 23 / 108

Examples: N(t) could be any of the following.(a) Cars entering a shopping center (by time t).(b) Defects on a wire (of length t).

(c) Raisins in cookie dough (of volume t).

ISYE 6739 — Goldsman 8/5/20 23 / 108

In the above examples, we might have:

(a) λ = 10/min. (b) λ = 0.5/ft. (c) λ = 4/in3.

ISYE 6739 — Goldsman 8/5/20 23 / 108

In the above examples, we might have:(a) λ = 10/min.

(b) λ = 0.5/ft. (c) λ = 4/in3.

ISYE 6739 — Goldsman 8/5/20 23 / 108

In the above examples, we might have:(a) λ = 10/min. (b) λ = 0.5/ft.

(c) λ = 4/in3.

ISYE 6739 — Goldsman 8/5/20 23 / 108

A Poisson process is a specific counting process. . . .

First, some notation: o(h) is a generic function that goes to zero faster than hgoes to zero.

Definition: A Poisson process (PP) is one that satisfies the following:

(i) There is a short enough interval of time h, such that, for all t,

P (N(t+ h)−N(t) = 0) = 1− λh+ o(h)

P (N(t+ h)−N(t) = 1) = λh+ o(h)

P (N(t+ h)−N(t) ≥ 2) = o(h)

(ii) The distribution of the “increment” N(t+ h)−N(t) only depends onthe length h.

(iii) If a < b < c < d, then the two “increments” N(d)−N(c) andN(b)−N(a) are independent RVs.

ISYE 6739 — Goldsman 8/5/20 24 / 108

P (N(t+ h)−N(t) = 0) = 1− λh+ o(h)

P (N(t+ h)−N(t) = 1) = λh+ o(h)

P (N(t+ h)−N(t) ≥ 2) = o(h)

ISYE 6739 — Goldsman 8/5/20 24 / 108

P (N(t+ h)−N(t) = 0) = 1− λh+ o(h)

P (N(t+ h)−N(t) = 1) = λh+ o(h)

P (N(t+ h)−N(t) ≥ 2) = o(h)

ISYE 6739 — Goldsman 8/5/20 24 / 108

P (N(t+ h)−N(t) = 0) = 1− λh+ o(h)

P (N(t+ h)−N(t) = 1) = λh+ o(h)

P (N(t+ h)−N(t) ≥ 2) = o(h)

ISYE 6739 — Goldsman 8/5/20 24 / 108

P (N(t+ h)−N(t) = 0) = 1− λh+ o(h)

P (N(t+ h)−N(t) = 1) = λh+ o(h)

P (N(t+ h)−N(t) ≥ 2) = o(h)

ISYE 6739 — Goldsman 8/5/20 24 / 108

P (N(t+ h)−N(t) = 0) = 1− λh+ o(h)

P (N(t+ h)−N(t) = 1) = λh+ o(h)

P (N(t+ h)−N(t) ≥ 2) = o(h)

ISYE 6739 — Goldsman 8/5/20 24 / 108

P (N(t+ h)−N(t) = 0) = 1− λh+ o(h)

P (N(t+ h)−N(t) = 1) = λh+ o(h)

P (N(t+ h)−N(t) ≥ 2) = o(h)

ISYE 6739 — Goldsman 8/5/20 24 / 108

P (N(t+ h)−N(t) = 0) = 1− λh+ o(h)

P (N(t+ h)−N(t) = 1) = λh+ o(h)

P (N(t+ h)−N(t) ≥ 2) = o(h)

ISYE 6739 — Goldsman 8/5/20 24 / 108

P (N(t+ h)−N(t) = 0) = 1− λh+ o(h)

P (N(t+ h)−N(t) = 1) = λh+ o(h)

P (N(t+ h)−N(t) ≥ 2) = o(h)

ISYE 6739 — Goldsman 8/5/20 24 / 108

English translation of Poisson process assumptions.

(i) Arrivals basically occur one-at-a-time, and then at rate λ/unit time. (Wemust make sure that λ doesn’t change over time.)

(ii) The arrival pattern is stationary — it doesn’t change over time.

(iii) The numbers of arrivals in two disjoint time intervals are independent

Poisson Process Example: Neutrinos hit a detector. Occurrences are rareenough so that they really do happen one-at-a-time. You never get arrivals ofgroups of neutrinos. Further, the rate doesn’t vary over time, and all arrivalsare independent of each other. 2

Anti-Example: Customers arrive at a restaurant. They show up in groups,not one-at-a-time. The rate varies over the day (more at dinnertime). Arrivalsmay not be independent. This ain’t a Poisson process. 2

ISYE 6739 — Goldsman 8/5/20 25 / 108

Definition: Let X be the number of occurrences in a Poisson(λ) process in aunit interval of time.

Then X has the Poisson distribution with parameterλ.

Notation: X ∼ Pois(λ).

Theorem/Definition: X ∼ Pois(λ)⇒ P (X = k) = e−λλk/k!,k = 0, 1, 2, . . ..

Proof: The proof follows from the PP assumptions and involves some simpledifferential equations.

To begin with, let’s define Px(t) ≡ P (N(t) = x), i.e., the probability ofexactly x arrivals by time t.

Note that the probability that there haven’t been any arrivals by time t+ h canbe written in terms of the probability that there haven’t been any arrivals bytime t. . . .

ISYE 6739 — Goldsman 8/5/20 26 / 108

Definition: Let X be the number of occurrences in a Poisson(λ) process in aunit interval of time. Then X has the Poisson distribution with parameterλ.

ISYE 6739 — Goldsman 8/5/20 26 / 108

P0(t+ h)

= P (N(t+ h) = 0)

= P (no arrivals by time t and then no arrivals by time t+ h)

= P({N(t) = 0} ∩ {N(t+ h)−N(t) = 0}

)= P (N(t) = 0)P (N(t+ h)−N(t) = 0) (by indep increments (iii)).= P (N(t) = 0)(1− λh) (by (i) and a little bit of (ii))

= P0(t)(1− λh).

Thus,P0(t+ h)− P0(t)

.= −λP0(t).

Taking the limit as h→ 0, we have

P ′0(t) = −λP0(t). (1)

ISYE 6739 — Goldsman 8/5/20 27 / 108

P0(t+ h)

= P (N(t+ h) = 0)

= P({N(t) = 0} ∩ {N(t+ h)−N(t) = 0}

= P0(t)(1− λh).

.= −λP0(t).

P ′0(t) = −λP0(t). (1)

ISYE 6739 — Goldsman 8/5/20 27 / 108

P0(t+ h)

= P (N(t+ h) = 0)

= P({N(t) = 0} ∩ {N(t+ h)−N(t) = 0}

= P (N(t) = 0)P (N(t+ h)−N(t) = 0) (by indep increments (iii)).= P (N(t) = 0)(1− λh) (by (i) and a little bit of (ii))

= P0(t)(1− λh).

.= −λP0(t).

P ′0(t) = −λP0(t). (1)

ISYE 6739 — Goldsman 8/5/20 27 / 108

P0(t+ h)

= P (N(t+ h) = 0)

= P({N(t) = 0} ∩ {N(t+ h)−N(t) = 0}

)= P (N(t) = 0)P (N(t+ h)−N(t) = 0) (by indep increments (iii))

.= P (N(t) = 0)(1− λh) (by (i) and a little bit of (ii))

= P0(t)(1− λh).

.= −λP0(t).

P ′0(t) = −λP0(t). (1)

ISYE 6739 — Goldsman 8/5/20 27 / 108

P0(t+ h)

= P (N(t+ h) = 0)

= P({N(t) = 0} ∩ {N(t+ h)−N(t) = 0}

= P0(t)(1− λh).

.= −λP0(t).

P ′0(t) = −λP0(t). (1)

ISYE 6739 — Goldsman 8/5/20 27 / 108

P0(t+ h)

= P (N(t+ h) = 0)

= P({N(t) = 0} ∩ {N(t+ h)−N(t) = 0}

= P0(t)(1− λh).

.= −λP0(t).

P ′0(t) = −λP0(t). (1)

ISYE 6739 — Goldsman 8/5/20 27 / 108

P0(t+ h)

= P (N(t+ h) = 0)

= P({N(t) = 0} ∩ {N(t+ h)−N(t) = 0}

= P0(t)(1− λh).

.= −λP0(t).

P ′0(t) = −λP0(t). (1)

ISYE 6739 — Goldsman 8/5/20 27 / 108

P0(t+ h)

= P (N(t+ h) = 0)

= P({N(t) = 0} ∩ {N(t+ h)−N(t) = 0}

= P0(t)(1− λh).

.= −λP0(t).

P ′0(t) = −λP0(t). (1)

ISYE 6739 — Goldsman 8/5/20 27 / 108

P0(t+ h)

= P (N(t+ h) = 0)

= P({N(t) = 0} ∩ {N(t+ h)−N(t) = 0}

= P0(t)(1− λh).

.= −λP0(t).

P ′0(t) = −λP0(t). (1)

ISYE 6739 — Goldsman 8/5/20 27 / 108

Similarly, for x > 0, we have

Px(t+ h) = P (N(t+ h) = x)

= P (N(t+ h) = x and no arrivals during [t, t+ h])

+P (N(t+ h) = x and ≥ 1 arrival during [t, t+ h])

(Law of Total Probability).= P

({N(t) = x} ∩ {N(t+ h)−N(t) = 0}

)+P({N(t) = x− 1} ∩ {N(t+ h)−N(t) = 1}

)(by (i), only consider case of one arrival in [t, t+ h])

= P (N(t) = x)P (N(t+ h)−N(t) = 0)

+P (N(t) = x− 1)P (N(t+ h)−N(t) = 1)

(by independent increments (iii)).= Px(t)(1− λh) + Px−1(t)λh.

ISYE 6739 — Goldsman 8/5/20 28 / 108

Px(t+ h) = P (N(t+ h) = x)

({N(t) = x} ∩ {N(t+ h)−N(t) = 0}

)+P({N(t) = x− 1} ∩ {N(t+ h)−N(t) = 1}

= P (N(t) = x)P (N(t+ h)−N(t) = 0)

+P (N(t) = x− 1)P (N(t+ h)−N(t) = 1)

ISYE 6739 — Goldsman 8/5/20 28 / 108

Px(t+ h) = P (N(t+ h) = x)

(Law of Total Probability)

({N(t) = x} ∩ {N(t+ h)−N(t) = 0}

)+P({N(t) = x− 1} ∩ {N(t+ h)−N(t) = 1}

= P (N(t) = x)P (N(t+ h)−N(t) = 0)

+P (N(t) = x− 1)P (N(t+ h)−N(t) = 1)

ISYE 6739 — Goldsman 8/5/20 28 / 108

Px(t+ h) = P (N(t+ h) = x)

({N(t) = x} ∩ {N(t+ h)−N(t) = 0}

)+P({N(t) = x− 1} ∩ {N(t+ h)−N(t) = 1}

= P (N(t) = x)P (N(t+ h)−N(t) = 0)

+P (N(t) = x− 1)P (N(t+ h)−N(t) = 1)

ISYE 6739 — Goldsman 8/5/20 28 / 108

Px(t+ h) = P (N(t+ h) = x)

({N(t) = x} ∩ {N(t+ h)−N(t) = 0}

)+P({N(t) = x− 1} ∩ {N(t+ h)−N(t) = 1}

= P (N(t) = x)P (N(t+ h)−N(t) = 0)

+P (N(t) = x− 1)P (N(t+ h)−N(t) = 1)

(by independent increments (iii))

.= Px(t)(1− λh) + Px−1(t)λh.

ISYE 6739 — Goldsman 8/5/20 28 / 108

Px(t+ h) = P (N(t+ h) = x)

({N(t) = x} ∩ {N(t+ h)−N(t) = 0}

)+P({N(t) = x− 1} ∩ {N(t+ h)−N(t) = 1}

= P (N(t) = x)P (N(t+ h)−N(t) = 0)

+P (N(t) = x− 1)P (N(t+ h)−N(t) = 1)

ISYE 6739 — Goldsman 8/5/20 28 / 108

Taking the limits as h→ 0, we obtain

P ′x(t) = λ[Px−1(t)− Px(t)

], x = 1, 2, . . . . (2)

The solution of differential equations (1) and (2) is easily shown to be

Px(t) =(λt)xe−λt

x!, x = 0, 1, 2, . . . .

(If you don’t believe me, just plug in and see for yourself!)

Noting that t = 1 for the Pois(λ) finally completes the proof. 2

Remark: λ can be changed simply by changing the units of time.

Examples:X = # calls to a switchboard in 1 minute ∼ Pois(3 / min)Y = # calls to a switchboard in 5 minutes ∼ Pois(15 / 5 min)Z = # calls to a switchboard in 10 sec ∼ Pois(0.5 / 10 sec)

ISYE 6739 — Goldsman 8/5/20 29 / 108

P ′x(t) = λ[Px−1(t)− Px(t)

], x = 1, 2, . . . . (2)

x!, x = 0, 1, 2, . . . .

ISYE 6739 — Goldsman 8/5/20 29 / 108

P ′x(t) = λ[Px−1(t)− Px(t)

], x = 1, 2, . . . . (2)

x!, x = 0, 1, 2, . . . .

ISYE 6739 — Goldsman 8/5/20 29 / 108

P ′x(t) = λ[Px−1(t)− Px(t)

], x = 1, 2, . . . . (2)

x!, x = 0, 1, 2, . . . .

ISYE 6739 — Goldsman 8/5/20 29 / 108

P ′x(t) = λ[Px−1(t)− Px(t)

], x = 1, 2, . . . . (2)

x!, x = 0, 1, 2, . . . .

ISYE 6739 — Goldsman 8/5/20 29 / 108

P ′x(t) = λ[Px−1(t)− Px(t)

], x = 1, 2, . . . . (2)

x!, x = 0, 1, 2, . . . .

ISYE 6739 — Goldsman 8/5/20 29 / 108

P ′x(t) = λ[Px−1(t)− Px(t)

], x = 1, 2, . . . . (2)

x!, x = 0, 1, 2, . . . .

Examples:X = # calls to a switchboard in 1 minute ∼ Pois(3 / min)

Y = # calls to a switchboard in 5 minutes ∼ Pois(15 / 5 min)Z = # calls to a switchboard in 10 sec ∼ Pois(0.5 / 10 sec)

ISYE 6739 — Goldsman 8/5/20 29 / 108

P ′x(t) = λ[Px−1(t)− Px(t)

], x = 1, 2, . . . . (2)

x!, x = 0, 1, 2, . . . .

Examples:X = # calls to a switchboard in 1 minute ∼ Pois(3 / min)Y = # calls to a switchboard in 5 minutes ∼ Pois(15 / 5 min)

Z = # calls to a switchboard in 10 sec ∼ Pois(0.5 / 10 sec)

ISYE 6739 — Goldsman 8/5/20 29 / 108

P ′x(t) = λ[Px−1(t)− Px(t)

], x = 1, 2, . . . . (2)

x!, x = 0, 1, 2, . . . .

ISYE 6739 — Goldsman 8/5/20 29 / 108

Theorem: X ∼ Pois(λ)⇒ mgf is MX(t) = eλ(et−1).

Proof:

MX(t) = E[etX ] =∞∑k=0

etk(e−λλk

)= e−λ

∞∑k=0

(λet)k

k!= e−λeλe

Theorem: X ∼ Pois(λ)⇒ E[X] = Var(X) = λ.

Proof (using mgf):

E[X] =d

dtMX(t)

∣∣∣t=0

dteλ(e

t−1)∣∣∣t=0

= λetMX(t)∣∣∣t=0

ISYE 6739 — Goldsman 8/5/20 30 / 108

Proof:

MX(t) = E[etX ]

=∞∑k=0

etk(e−λλk

)= e−λ

∞∑k=0

(λet)k

k!= e−λeλe

Proof (using mgf):

E[X] =d

dtMX(t)

∣∣∣t=0

dteλ(e

t−1)∣∣∣t=0

ISYE 6739 — Goldsman 8/5/20 30 / 108

Proof:

MX(t) = E[etX ] =

∞∑k=0

etk(e−λλk

= e−λ∞∑k=0

(λet)k

k!= e−λeλe

Proof (using mgf):

E[X] =d

dtMX(t)

∣∣∣t=0

dteλ(e

t−1)∣∣∣t=0

ISYE 6739 — Goldsman 8/5/20 30 / 108

Proof:

MX(t) = E[etX ] =

∞∑k=0

etk(e−λλk

)= e−λ

∞∑k=0

(λet)k

= e−λeλet. 2

Proof (using mgf):

E[X] =d

dtMX(t)

∣∣∣t=0

dteλ(e

t−1)∣∣∣t=0

ISYE 6739 — Goldsman 8/5/20 30 / 108

Proof:

MX(t) = E[etX ] =

∞∑k=0

etk(e−λλk

)= e−λ

∞∑k=0

(λet)k

k!= e−λeλe

Proof (using mgf):

E[X] =d

dtMX(t)

∣∣∣t=0

dteλ(e

t−1)∣∣∣t=0

ISYE 6739 — Goldsman 8/5/20 30 / 108

Proof:

MX(t) = E[etX ] =

∞∑k=0

etk(e−λλk

)= e−λ

∞∑k=0

(λet)k

k!= e−λeλe

Proof (using mgf):

E[X] =d

dtMX(t)

∣∣∣t=0

dteλ(e

t−1)∣∣∣t=0

ISYE 6739 — Goldsman 8/5/20 30 / 108

Proof:

MX(t) = E[etX ] =

∞∑k=0

etk(e−λλk

)= e−λ

∞∑k=0

(λet)k

k!= e−λeλe

Proof (using mgf):

E[X] =d

dtMX(t)

∣∣∣t=0

dteλ(e

t−1)∣∣∣t=0

ISYE 6739 — Goldsman 8/5/20 30 / 108

Proof:

MX(t) = E[etX ] =

∞∑k=0

etk(e−λλk

)= e−λ

∞∑k=0

(λet)k

k!= e−λeλe

Proof (using mgf):

E[X] =d

dtMX(t)

∣∣∣t=0

dteλ(e

t−1)∣∣∣t=0

ISYE 6739 — Goldsman 8/5/20 30 / 108

Proof:

MX(t) = E[etX ] =

∞∑k=0

etk(e−λλk

)= e−λ

∞∑k=0

(λet)k

k!= e−λeλe

Proof (using mgf):

E[X] =d

dtMX(t)

∣∣∣t=0

dteλ(e

t−1)∣∣∣t=0

ISYE 6739 — Goldsman 8/5/20 30 / 108

Proof:

MX(t) = E[etX ] =

∞∑k=0

etk(e−λλk

)= e−λ

∞∑k=0

(λet)k

k!= e−λeλe

Proof (using mgf):

E[X] =d

dtMX(t)

∣∣∣t=0

dteλ(e

t−1)∣∣∣t=0

ISYE 6739 — Goldsman 8/5/20 30 / 108

Similarly,

E[X2] =d2

dt2MX(t)

∣∣∣t=0

( ddtMX(t)

)∣∣∣t=0

(etMX(t)

)∣∣∣t=0

= λ[etMX(t) + et

dtMX(t)

]∣∣∣t=0

= λet[MX(t) + λetMX(t)

]∣∣∣t=0

= λ(1 + λ).

Thus, Var(X) = E[X2]− (E[X])2 = λ(1 + λ)− λ2 = λ. 2

ISYE 6739 — Goldsman 8/5/20 31 / 108

Similarly,

E[X2] =d2

dt2MX(t)

∣∣∣t=0

( ddtMX(t)

)∣∣∣t=0

(etMX(t)

)∣∣∣t=0

= λ[etMX(t) + et

dtMX(t)

]∣∣∣t=0

= λ(1 + λ).

ISYE 6739 — Goldsman 8/5/20 31 / 108

Similarly,

E[X2] =d2

dt2MX(t)

∣∣∣t=0

( ddtMX(t)

)∣∣∣t=0

(etMX(t)

)∣∣∣t=0

= λ[etMX(t) + et

dtMX(t)

]∣∣∣t=0

= λ(1 + λ).

ISYE 6739 — Goldsman 8/5/20 31 / 108

Similarly,

E[X2] =d2

dt2MX(t)

∣∣∣t=0

( ddtMX(t)

)∣∣∣t=0

(etMX(t)

)∣∣∣t=0

= λ[etMX(t) + et

dtMX(t)

]∣∣∣t=0

= λ(1 + λ).

ISYE 6739 — Goldsman 8/5/20 31 / 108

Similarly,

E[X2] =d2

dt2MX(t)

∣∣∣t=0

( ddtMX(t)

)∣∣∣t=0

(etMX(t)

)∣∣∣t=0

= λ[etMX(t) + et

dtMX(t)

]∣∣∣t=0

= λ(1 + λ).

ISYE 6739 — Goldsman 8/5/20 31 / 108

Similarly,

E[X2] =d2

dt2MX(t)

∣∣∣t=0

( ddtMX(t)

)∣∣∣t=0

(etMX(t)

)∣∣∣t=0

= λ[etMX(t) + et

dtMX(t)

]∣∣∣t=0

= λ(1 + λ).

ISYE 6739 — Goldsman 8/5/20 31 / 108

Similarly,

E[X2] =d2

dt2MX(t)

∣∣∣t=0

( ddtMX(t)

)∣∣∣t=0

(etMX(t)

)∣∣∣t=0

= λ[etMX(t) + et

dtMX(t)

]∣∣∣t=0

= λ(1 + λ).

ISYE 6739 — Goldsman 8/5/20 31 / 108

Example: Calls to a switchboard arrive as a Poisson process with rate 3calls/min.

Let X = number of calls in 1 minute.

So X ∼ Pois(3), E[X] = Var(X) = 3, P (X ≤ 4) =∑4

k=0 e−33k/k!.

Let Y = number of calls in 40 sec.

So Y ∼ Pois(2), E[Y ] = Var(Y ) = 2, P (Y ≤ 4) =∑4

k=0 e−22k/k!. 2

ISYE 6739 — Goldsman 8/5/20 32 / 108

k=0 e−33k/k!.

k=0 e−22k/k!. 2

ISYE 6739 — Goldsman 8/5/20 32 / 108

k=0 e−33k/k!.

k=0 e−22k/k!. 2

ISYE 6739 — Goldsman 8/5/20 32 / 108

k=0 e−33k/k!.

k=0 e−22k/k!. 2

ISYE 6739 — Goldsman 8/5/20 32 / 108

k=0 e−33k/k!.

k=0 e−22k/k!. 2

ISYE 6739 — Goldsman 8/5/20 32 / 108

Theorem (Additive Property of Poissons): Suppose X1, . . . , Xn areindependent with Xi ∼ Pois(λi), i = 1, . . . , n. Then

Y ≡n∑i=1

Xi ∼ Pois( n∑i=1

Proof: Since the Xi’s are independent, we have

MY (t) =

n∏i=1

MXi(t) =

n∏i=1

eλi(et−1) = e(

∑ni=1 λi)(e

t−1),

which is the mgf of the Pois(∑n

i=1 λi)

distribution. 2

ISYE 6739 — Goldsman 8/5/20 33 / 108

Y ≡n∑i=1

MY (t) =

n∏i=1

MXi(t) =

n∏i=1

eλi(et−1) = e(

∑ni=1 λi)(e

t−1),

i=1 λi)

distribution. 2

ISYE 6739 — Goldsman 8/5/20 33 / 108

Y ≡n∑i=1

MY (t) =

n∏i=1

MXi(t) =

n∏i=1

eλi(et−1) = e(

∑ni=1 λi)(e

t−1),

i=1 λi)

distribution. 2

ISYE 6739 — Goldsman 8/5/20 33 / 108

Y ≡n∑i=1

MY (t) =

n∏i=1

MXi(t)

n∏i=1

eλi(et−1) = e(

∑ni=1 λi)(e

t−1),

i=1 λi)

distribution. 2

ISYE 6739 — Goldsman 8/5/20 33 / 108

Y ≡n∑i=1

MY (t) =

n∏i=1

MXi(t) =

n∏i=1

eλi(et−1)

= e(∑n

i=1 λi)(et−1),

i=1 λi)

distribution. 2

ISYE 6739 — Goldsman 8/5/20 33 / 108

Y ≡n∑i=1

MY (t) =

n∏i=1

MXi(t) =

n∏i=1

eλi(et−1) = e(

∑ni=1 λi)(e

t−1),

i=1 λi)

distribution. 2

ISYE 6739 — Goldsman 8/5/20 33 / 108

Y ≡n∑i=1

MY (t) =

n∏i=1

MXi(t) =

n∏i=1

eλi(et−1) = e(

∑ni=1 λi)(e

t−1),

i=1 λi)

distribution. 2

ISYE 6739 — Goldsman 8/5/20 33 / 108

Example: Cars driven by males [females] arrive at a parking lot according toa Poisson process with a rate of λ1 = 3 / hr [λ2 = 5 / hr]. All arrivals areindependent.

What’s the probability of exactly 2 arrivals in the next 30 minutes?

The total number of arrivals is Pois(λ1 + λ2 = 8/hr), and so the total in thenext 30 minutes is X ∼ Pois(4). So P (X = 2) = e−442/2!. 2

ISYE 6739 — Goldsman 8/5/20 34 / 108

The total number of arrivals is Pois(λ1 + λ2 = 8/hr), and so the total in thenext 30 minutes is X ∼ Pois(4).

So P (X = 2) = e−442/2!. 2

ISYE 6739 — Goldsman 8/5/20 34 / 108

Uniform, Exponential, and Friends

14 Computer Stuff

ISYE 6739 — Goldsman 8/5/20 35 / 108

Lesson 4.5 — Uniform, Exponential, and Friends

Definition: The RV X has the Uniform distribution if it has pdf and cdf

f(x) =

{1b−a , a ≤ x ≤ b0, otherwise

and F (x) =

0, x < axb−a , a ≤ x ≤ b1, x ≥ b.

Notation: X ∼ Unif(a, b).

Previous work showed that

E[X] =a+ b

2and Var(X) =

(a− b)2

We can also derive the mgf,

MX(t) = E[etX ] =

b− adx =

etb − eta

t(b− a).

ISYE 6739 — Goldsman 8/5/20 36 / 108

f(x) =

and F (x) =

0, x < axb−a , a ≤ x ≤ b1, x ≥ b.

E[X] =a+ b

2and Var(X) =

(a− b)2

MX(t) = E[etX ] =

b− adx =

etb − eta

t(b− a).

ISYE 6739 — Goldsman 8/5/20 36 / 108

f(x) =

and F (x) =

0, x < axb−a , a ≤ x ≤ b1, x ≥ b.

E[X] =a+ b

2and Var(X) =

(a− b)2

MX(t) = E[etX ] =

b− adx =

etb − eta

t(b− a).

ISYE 6739 — Goldsman 8/5/20 36 / 108

f(x) =

and F (x) =

0, x < axb−a , a ≤ x ≤ b1, x ≥ b.

E[X] =a+ b

2and Var(X) =

(a− b)2

MX(t) = E[etX ] =

b− adx =

etb − eta

t(b− a).

ISYE 6739 — Goldsman 8/5/20 36 / 108

f(x) =

and F (x) =

0, x < axb−a , a ≤ x ≤ b1, x ≥ b.

E[X] =a+ b

2and Var(X) =

(a− b)2

MX(t) = E[etX ] =

b− adx =

etb − eta

t(b− a).

ISYE 6739 — Goldsman 8/5/20 36 / 108

f(x) =

and F (x) =

0, x < axb−a , a ≤ x ≤ b1, x ≥ b.

E[X] =a+ b

2and Var(X) =

(a− b)2

MX(t) = E[etX ] =

b− adx =

etb − eta

t(b− a).

ISYE 6739 — Goldsman 8/5/20 36 / 108

Definition: The Exponential(λ) distribution has pdf

f(x) =

{λe−λx, x > 0

0, otherwise.

Previous work showed that the cdf F (x) = 1− e−λx,

E[X] = 1/λ, and Var(X) = 1/λ2.

We also derived the mgf,

MX(t) = E[etX ] =

∫ ∞0

etxf(x) dx =λ

λ− t, t < λ.

ISYE 6739 — Goldsman 8/5/20 37 / 108

f(x) =

{λe−λx, x > 0

0, otherwise.

E[X] = 1/λ, and Var(X) = 1/λ2.

MX(t) = E[etX ] =

∫ ∞0

etxf(x) dx =λ

λ− t, t < λ.

ISYE 6739 — Goldsman 8/5/20 37 / 108

f(x) =

{λe−λx, x > 0

0, otherwise.

E[X] = 1/λ, and Var(X) = 1/λ2.

MX(t) = E[etX ] =

∫ ∞0

etxf(x) dx =λ

λ− t, t < λ.

ISYE 6739 — Goldsman 8/5/20 37 / 108

f(x) =

{λe−λx, x > 0

0, otherwise.

E[X] = 1/λ, and Var(X) = 1/λ2.

MX(t) = E[etX ] =

∫ ∞0

etxf(x) dx =λ

λ− t, t < λ.

ISYE 6739 — Goldsman 8/5/20 37 / 108

f(x) =

{λe−λx, x > 0

0, otherwise.

E[X] = 1/λ, and Var(X) = 1/λ2.

MX(t) = E[etX ] =

∫ ∞0

etxf(x) dx =λ

λ− t, t < λ.

ISYE 6739 — Goldsman 8/5/20 37 / 108

f(x) =

{λe−λx, x > 0

0, otherwise.

E[X] = 1/λ, and Var(X) = 1/λ2.

MX(t) = E[etX ] =

∫ ∞0

etxf(x) dx =λ

λ− t, t < λ.

ISYE 6739 — Goldsman 8/5/20 37 / 108

Memoryless Property of Exponential

Theorem: Suppose that X ∼ Exp(λ). Then for positive s, t, we have

P (X > s+ t|X > s) = P (X > t).

Similar to the discrete Geometric distribution, the probability that X willsurvive an additional t time units is the (unconditional) probability that it willsurvive at least t — it forgot that it made it past time s! It’s always “like new”!

Proof:

P (X > s+ t|X > s) =P (X > s+ t ∩X > s)

P (X > s)=

P (X > s+ t)

P (X > s)

=e−λ(s+t)

e−λs= e−λt = P (X > t). 2

ISYE 6739 — Goldsman 8/5/20 38 / 108

P (X > s+ t|X > s) = P (X > t).

Proof:

P (X > s+ t|X > s) =P (X > s+ t ∩X > s)

P (X > s)=

P (X > s+ t)

P (X > s)

=e−λ(s+t)

e−λs= e−λt = P (X > t). 2

ISYE 6739 — Goldsman 8/5/20 38 / 108

P (X > s+ t|X > s) = P (X > t).

Proof:

P (X > s+ t|X > s) =P (X > s+ t ∩X > s)

P (X > s)=

P (X > s+ t)

P (X > s)

=e−λ(s+t)

e−λs= e−λt = P (X > t). 2

ISYE 6739 — Goldsman 8/5/20 38 / 108

P (X > s+ t|X > s) = P (X > t).

Similar to the discrete Geometric distribution, the probability that X willsurvive an additional t time units is the (unconditional) probability that it willsurvive at least t

— it forgot that it made it past time s! It’s always “like new”!

Proof:

P (X > s+ t|X > s) =P (X > s+ t ∩X > s)

P (X > s)=

P (X > s+ t)

P (X > s)

=e−λ(s+t)

e−λs= e−λt = P (X > t). 2

ISYE 6739 — Goldsman 8/5/20 38 / 108

P (X > s+ t|X > s) = P (X > t).

Proof:

P (X > s+ t|X > s) =P (X > s+ t ∩X > s)

P (X > s)=

P (X > s+ t)

P (X > s)

=e−λ(s+t)

e−λs= e−λt = P (X > t). 2

ISYE 6739 — Goldsman 8/5/20 38 / 108

P (X > s+ t|X > s) = P (X > t).

Proof:

P (X > s+ t|X > s) =P (X > s+ t ∩X > s)

P (X > s)

=P (X > s+ t)

P (X > s)

=e−λ(s+t)

e−λs= e−λt = P (X > t). 2

ISYE 6739 — Goldsman 8/5/20 38 / 108

P (X > s+ t|X > s) = P (X > t).

Proof:

P (X > s+ t|X > s) =P (X > s+ t ∩X > s)

P (X > s)=

P (X > s+ t)

P (X > s)

=e−λ(s+t)

e−λs= e−λt = P (X > t). 2

ISYE 6739 — Goldsman 8/5/20 38 / 108

P (X > s+ t|X > s) = P (X > t).

Proof:

P (X > s+ t|X > s) =P (X > s+ t ∩X > s)

P (X > s)=

P (X > s+ t)

P (X > s)

=e−λ(s+t)

e−λs

= e−λt = P (X > t). 2

ISYE 6739 — Goldsman 8/5/20 38 / 108

P (X > s+ t|X > s) = P (X > t).

Proof:

P (X > s+ t|X > s) =P (X > s+ t ∩X > s)

P (X > s)=

P (X > s+ t)

P (X > s)

=e−λ(s+t)

e−λs= e−λt = P (X > t). 2

ISYE 6739 — Goldsman 8/5/20 38 / 108

Example: Suppose that the life of a lightbulb is exponential with a mean of10 months.

If the light survives 20 months, what’s the probability that it’llsurvive another 10?

P (X > 30|X > 20) = P (X > 10) = e−λx = e−(1/10)(10) = e−1. 2

Example: If the time to the next bus is exponentially distributed with a meanof 10 minutes, and you’ve already been waiting 20 minutes, you can expect towait 10 more. 2

Remark: The exponential is the only continuous distribution with theMemoryless Property.

Remark: Look at E[X] and Var(X) for the Geometric distribution and seehow they’re similar to those for the exponential. (Not a coincidence.)

ISYE 6739 — Goldsman 8/5/20 39 / 108

Example: Suppose that the life of a lightbulb is exponential with a mean of10 months. If the light survives 20 months, what’s the probability that it’llsurvive another 10?

P (X > 30|X > 20) = P (X > 10) = e−λx = e−(1/10)(10) = e−1. 2

ISYE 6739 — Goldsman 8/5/20 39 / 108

P (X > 30|X > 20) = P (X > 10)

= e−λx = e−(1/10)(10) = e−1. 2

ISYE 6739 — Goldsman 8/5/20 39 / 108

P (X > 30|X > 20) = P (X > 10) = e−λx

= e−(1/10)(10) = e−1. 2

ISYE 6739 — Goldsman 8/5/20 39 / 108

P (X > 30|X > 20) = P (X > 10) = e−λx = e−(1/10)(10)

= e−1. 2

ISYE 6739 — Goldsman 8/5/20 39 / 108

P (X > 30|X > 20) = P (X > 10) = e−λx = e−(1/10)(10) = e−1. 2

ISYE 6739 — Goldsman 8/5/20 39 / 108

P (X > 30|X > 20) = P (X > 10) = e−λx = e−(1/10)(10) = e−1. 2

ISYE 6739 — Goldsman 8/5/20 39 / 108

P (X > 30|X > 20) = P (X > 10) = e−λx = e−(1/10)(10) = e−1. 2

ISYE 6739 — Goldsman 8/5/20 39 / 108

P (X > 30|X > 20) = P (X > 10) = e−λx = e−(1/10)(10) = e−1. 2

ISYE 6739 — Goldsman 8/5/20 39 / 108

Definition: If X is a cts RV with pdf f(x) and cdf F (x), then its failurerate function is

S(t) ≡ f(t)

P (X > t)=

1− F (t),

which can loosely be regarded as X’s instantaneous rate of death, given that ithas so far survived to time t.

Example: If X ∼ Exp(λ), then S(t) = λe−λt/e−λt = λ. So if X is theexponential lifetime of a lightbulb, then its instantaneous burn-out rate isalways λ — always good as new! This is clearly a result of the MemorylessProperty. 2

ISYE 6739 — Goldsman 8/5/20 40 / 108

S(t) ≡ f(t)

P (X > t)=

1− F (t),

ISYE 6739 — Goldsman 8/5/20 40 / 108

S(t) ≡ f(t)

P (X > t)=

1− F (t),

ISYE 6739 — Goldsman 8/5/20 40 / 108

S(t) ≡ f(t)

P (X > t)=

1− F (t),

ISYE 6739 — Goldsman 8/5/20 40 / 108

The Exponential is also related to the Poisson!

Theorem: Let X be the amount of time until the first arrival in a Poissonprocess with rate λ. Then X ∼ Exp(λ).

Proof:

F (x) = P (X ≤ x) = 1− P (no arrivals in [0, x])

= 1− e−λx(λx)0

0!(since # arrivals in [0, x] is Pois(λx))

= 1− e−λx. 2

Theorem: Amazingly, it can be shown (after a lot of work) that theinterarrival times of a PP are all iid Exp(λ)! See for yourself when you take astochastic processes course.

ISYE 6739 — Goldsman 8/5/20 41 / 108

Proof:

= 1− e−λx(λx)0

= 1− e−λx. 2

ISYE 6739 — Goldsman 8/5/20 41 / 108

Proof:

= 1− e−λx(λx)0

= 1− e−λx. 2

ISYE 6739 — Goldsman 8/5/20 41 / 108

Proof:

= 1− e−λx(λx)0

= 1− e−λx. 2

ISYE 6739 — Goldsman 8/5/20 41 / 108

Proof:

= 1− e−λx(λx)0

= 1− e−λx. 2

ISYE 6739 — Goldsman 8/5/20 41 / 108

Proof:

= 1− e−λx(λx)0

= 1− e−λx. 2

ISYE 6739 — Goldsman 8/5/20 41 / 108

Example: Suppose that arrivals to a shopping center are from a PP with rateλ = 20/hr.

What’s the probability that the time between the 13th and 14thcustomers will be at least 4 minutes?

Let the time between customers 13 and 14 be X . Since we have a PP, theinterarrivals are iid Exp(λ = 20/hr), so

P (X > 4 min) = P (X > 1/15 hr) = e−λt = e−20/15. 2

ISYE 6739 — Goldsman 8/5/20 42 / 108

Example: Suppose that arrivals to a shopping center are from a PP with rateλ = 20/hr. What’s the probability that the time between the 13th and 14thcustomers will be at least 4 minutes?

P (X > 4 min) = P (X > 1/15 hr) = e−λt = e−20/15. 2

ISYE 6739 — Goldsman 8/5/20 42 / 108

P (X > 4 min) = P (X > 1/15 hr) = e−λt = e−20/15. 2

ISYE 6739 — Goldsman 8/5/20 42 / 108

P (X > 4 min) = P (X > 1/15 hr) = e−λt = e−20/15. 2

ISYE 6739 — Goldsman 8/5/20 42 / 108

Definition/Theorem: Suppose X1, . . . , Xkiid∼ Exp(λ), and let

S =∑k

i=1Xi. Then S has the Erlang distribution with parameters k andλ, denoted S ∼ Erlangk(λ).

The Erlang is simply the sum of iid exponentials.

Special Case: Erlang1(λ) ∼ Exp(λ).

The pdf and cdf of the Erlang are

f(s) =λke−λssk−1

(k − 1)!, s ≥ 0, and F (s) = 1−

k−1∑i=0

e−λs(λs)i

Notice that the cdf is the sum of a bunch of Poisson probabilities. (Won’t do ithere, but this observation helps in the derivation of the cdf.)

ISYE 6739 — Goldsman 8/5/20 43 / 108

S =∑k

(k − 1)!, s ≥ 0, and F (s) = 1−

k−1∑i=0

e−λs(λs)i

ISYE 6739 — Goldsman 8/5/20 43 / 108

S =∑k

(k − 1)!, s ≥ 0, and F (s) = 1−

k−1∑i=0

e−λs(λs)i

ISYE 6739 — Goldsman 8/5/20 43 / 108

S =∑k

(k − 1)!, s ≥ 0, and F (s) = 1−

k−1∑i=0

e−λs(λs)i

ISYE 6739 — Goldsman 8/5/20 43 / 108

S =∑k

(k − 1)!, s ≥ 0, and

F (s) = 1−k−1∑i=0

e−λs(λs)i

ISYE 6739 — Goldsman 8/5/20 43 / 108

S =∑k

(k − 1)!, s ≥ 0, and F (s) = 1−

k−1∑i=0

e−λs(λs)i

ISYE 6739 — Goldsman 8/5/20 43 / 108

S =∑k

(k − 1)!, s ≥ 0, and F (s) = 1−

k−1∑i=0

e−λs(λs)i

ISYE 6739 — Goldsman 8/5/20 43 / 108

Expected value, variance, and mgf:

E[S] = E[ k∑i=1

k∑i=1

E[Xi] = k/λ

Var(S) = k/λ2

MS(t) =

λ− t

Example: Suppose X and Y are iid Exp(2). Find P (X + Y < 1).

P (X + Y < 1) = 1−k−1∑i=0

e−λs(λs)i

i!= 1−

2−1∑i=0

e−(2·1)(2 · 1)i

i!= 0.594. 2

ISYE 6739 — Goldsman 8/5/20 44 / 108

E[S] = E[ k∑i=1

k∑i=1

E[Xi] = k/λ

Var(S) = k/λ2

MS(t) =

λ− t

P (X + Y < 1) = 1−k−1∑i=0

e−λs(λs)i

i!= 1−

2−1∑i=0

e−(2·1)(2 · 1)i

i!= 0.594. 2

ISYE 6739 — Goldsman 8/5/20 44 / 108

E[S] = E[ k∑i=1

k∑i=1

E[Xi] = k/λ

Var(S) = k/λ2

MS(t) =

λ− t

P (X + Y < 1) = 1−k−1∑i=0

e−λs(λs)i

i!= 1−

2−1∑i=0

e−(2·1)(2 · 1)i

i!= 0.594. 2

ISYE 6739 — Goldsman 8/5/20 44 / 108

E[S] = E[ k∑i=1

k∑i=1

E[Xi] = k/λ

Var(S) = k/λ2

MS(t) =

λ− t

P (X + Y < 1) = 1−k−1∑i=0

e−λs(λs)i

i!= 1−

2−1∑i=0

e−(2·1)(2 · 1)i

i!= 0.594. 2

ISYE 6739 — Goldsman 8/5/20 44 / 108

E[S] = E[ k∑i=1

k∑i=1

E[Xi] = k/λ

Var(S) = k/λ2

MS(t) =

λ− t

P (X + Y < 1) = 1−k−1∑i=0

e−λs(λs)i

= 1−2−1∑i=0

e−(2·1)(2 · 1)i

i!= 0.594. 2

ISYE 6739 — Goldsman 8/5/20 44 / 108

E[S] = E[ k∑i=1

k∑i=1

E[Xi] = k/λ

Var(S) = k/λ2

MS(t) =

λ− t

P (X + Y < 1) = 1−k−1∑i=0

e−λs(λs)i

i!= 1−

2−1∑i=0

e−(2·1)(2 · 1)i

i!= 0.594. 2

ISYE 6739 — Goldsman 8/5/20 44 / 108

Definition: X has the Gamma distribution with parameters α > 0 andλ > 0 if it has pdf

f(x) =λαe−λxxα−1

Γ(α), x ≥ 0,

whereΓ(α) ≡

∫ ∞0

tα−1e−t dt

is the gamma function.

Remark: The Gamma distribution generalizes the Erlang distribution (whereα has to be a positive integer). It has the same expected value and variance asthe Erlang, with α in place of k.

Remark: If α is a positive integer, then Γ(α) = (α− 1)!. Party trick:Γ(1/2) =

√π.

ISYE 6739 — Goldsman 8/5/20 45 / 108

Γ(α), x ≥ 0,

whereΓ(α) ≡

∫ ∞0

tα−1e−t dt

√π.

ISYE 6739 — Goldsman 8/5/20 45 / 108

Γ(α), x ≥ 0,

whereΓ(α) ≡

∫ ∞0

tα−1e−t dt

√π.

ISYE 6739 — Goldsman 8/5/20 45 / 108

Γ(α), x ≥ 0,

whereΓ(α) ≡

∫ ∞0

tα−1e−t dt

√π.

ISYE 6739 — Goldsman 8/5/20 45 / 108

Γ(α), x ≥ 0,

whereΓ(α) ≡

∫ ∞0

tα−1e−t dt

√π.

ISYE 6739 — Goldsman 8/5/20 45 / 108

Other Continuous Distributions

14 Computer Stuff

ISYE 6739 — Goldsman 8/5/20 46 / 108

Lesson 4.6 — Other Continuous Distributions

Triangular(a, b, c) Distribution — good for modeling RVs on the basis oflimited data (minimum, mode, maximum).

f(x) =

2(x−a)

(b−a)(c−a) , a < x ≤ b2(c−x)

(c−b)(c−a) , b < x < c

0, otherwise.

E[X] =a+ b+ c

3and Var(X) = mess

ISYE 6739 — Goldsman 8/5/20 47 / 108

f(x) =

2(x−a)

(b−a)(c−a) , a < x ≤ b2(c−x)

(c−b)(c−a) , b < x < c

0, otherwise.

E[X] =a+ b+ c

3and Var(X) = mess

ISYE 6739 — Goldsman 8/5/20 47 / 108

f(x) =

2(x−a)

(b−a)(c−a) , a < x ≤ b2(c−x)

(c−b)(c−a) , b < x < c

0, otherwise.

E[X] =a+ b+ c

3and Var(X) = mess

ISYE 6739 — Goldsman 8/5/20 47 / 108

f(x) =

2(x−a)

(b−a)(c−a) , a < x ≤ b2(c−x)

(c−b)(c−a) , b < x < c

0, otherwise.

E[X] =a+ b+ c

Var(X) = mess

ISYE 6739 — Goldsman 8/5/20 47 / 108

f(x) =

2(x−a)

(b−a)(c−a) , a < x ≤ b2(c−x)

(c−b)(c−a) , b < x < c

0, otherwise.

E[X] =a+ b+ c

3and Var(X) = mess

ISYE 6739 — Goldsman 8/5/20 47 / 108

Beta(a, b) Distribution — good for modeling RVs that are restricted to aninterval.

f(x) =Γ(a+ b)

Γ(a)Γ(b)xa−1(1− x)b−1, 0 < x < 1.

E[X] =a

a+ band Var(X) =

(a+ b)2(a+ b+ 1).

This distribution gets its name from the beta function, which is defined as

β(a, b) ≡ Γ(a)Γ(b)

Γ(a+ b)=

0xa−1(1− x)b−1 dx.

ISYE 6739 — Goldsman 8/5/20 48 / 108

f(x) =Γ(a+ b)

Γ(a)Γ(b)xa−1(1− x)b−1, 0 < x < 1.

E[X] =a

a+ band Var(X) =

(a+ b)2(a+ b+ 1).

β(a, b) ≡ Γ(a)Γ(b)

Γ(a+ b)=

0xa−1(1− x)b−1 dx.

ISYE 6739 — Goldsman 8/5/20 48 / 108

f(x) =Γ(a+ b)

Γ(a)Γ(b)xa−1(1− x)b−1, 0 < x < 1.

E[X] =a

a+ band

Var(X) =ab

(a+ b)2(a+ b+ 1).

β(a, b) ≡ Γ(a)Γ(b)

Γ(a+ b)=

0xa−1(1− x)b−1 dx.

ISYE 6739 — Goldsman 8/5/20 48 / 108

f(x) =Γ(a+ b)

Γ(a)Γ(b)xa−1(1− x)b−1, 0 < x < 1.

E[X] =a

a+ band Var(X) =

(a+ b)2(a+ b+ 1).

β(a, b) ≡ Γ(a)Γ(b)

Γ(a+ b)=

0xa−1(1− x)b−1 dx.

ISYE 6739 — Goldsman 8/5/20 48 / 108

f(x) =Γ(a+ b)

Γ(a)Γ(b)xa−1(1− x)b−1, 0 < x < 1.

E[X] =a

a+ band Var(X) =

(a+ b)2(a+ b+ 1).

β(a, b) ≡ Γ(a)Γ(b)

Γ(a+ b)=

0xa−1(1− x)b−1 dx.

ISYE 6739 — Goldsman 8/5/20 48 / 108

f(x) =Γ(a+ b)

Γ(a)Γ(b)xa−1(1− x)b−1, 0 < x < 1.

E[X] =a

a+ band Var(X) =

(a+ b)2(a+ b+ 1).

β(a, b) ≡ Γ(a)Γ(b)

Γ(a+ b)=

0xa−1(1− x)b−1 dx.

ISYE 6739 — Goldsman 8/5/20 48 / 108

The Beta distribution is very flexible. Here are a few family portraits.

Remark: Certain versions of the Uniform (a = b = 1) and Triangulardistributions are special cases of the Beta.

ISYE 6739 — Goldsman 8/5/20 49 / 108

Weibull(a, b) Distribution — good for modeling reliability models. a isthe “scale” parameter, and b is the “shape” parameter.

f(x) = ab(ax)b−1e−(ax)b, x > 0.

F (x) = 1− exp[−(ax)b], x > 0.

E[X] = (1/a)Γ(1 + (1/b)) and Var(X) = slight mess

Remark: The Exponential is a special case of the Weibull.

Example: Time-to-failure T for a transmitter has a Weibull distribution withparameters a = 1/(200 hrs) and b = 1/3. Then

E[T ] = 200Γ(1 + 3) = 1200 hrs.

The probability that it fails before 2000 hrs is

F (2000) = 1− exp[−(2000/200)1/3] = 0.884. 2

ISYE 6739 — Goldsman 8/5/20 50 / 108

f(x) = ab(ax)b−1e−(ax)b, x > 0.

F (x) = 1− exp[−(ax)b], x > 0.

E[T ] = 200Γ(1 + 3) = 1200 hrs.

F (2000) = 1− exp[−(2000/200)1/3] = 0.884. 2

ISYE 6739 — Goldsman 8/5/20 50 / 108

f(x) = ab(ax)b−1e−(ax)b, x > 0.

F (x) = 1− exp[−(ax)b], x > 0.

E[T ] = 200Γ(1 + 3) = 1200 hrs.

F (2000) = 1− exp[−(2000/200)1/3] = 0.884. 2

ISYE 6739 — Goldsman 8/5/20 50 / 108

f(x) = ab(ax)b−1e−(ax)b, x > 0.

F (x) = 1− exp[−(ax)b], x > 0.

E[T ] = 200Γ(1 + 3) = 1200 hrs.

F (2000) = 1− exp[−(2000/200)1/3] = 0.884. 2

ISYE 6739 — Goldsman 8/5/20 50 / 108

f(x) = ab(ax)b−1e−(ax)b, x > 0.

F (x) = 1− exp[−(ax)b], x > 0.

E[T ] = 200Γ(1 + 3) = 1200 hrs.

F (2000) = 1− exp[−(2000/200)1/3] = 0.884. 2

ISYE 6739 — Goldsman 8/5/20 50 / 108

f(x) = ab(ax)b−1e−(ax)b, x > 0.

F (x) = 1− exp[−(ax)b], x > 0.

E[T ] = 200Γ(1 + 3) = 1200 hrs.

F (2000) = 1− exp[−(2000/200)1/3] = 0.884. 2

ISYE 6739 — Goldsman 8/5/20 50 / 108

f(x) = ab(ax)b−1e−(ax)b, x > 0.

F (x) = 1− exp[−(ax)b], x > 0.

E[T ] = 200Γ(1 + 3) = 1200 hrs.

F (2000) = 1− exp[−(2000/200)1/3] = 0.884. 2

ISYE 6739 — Goldsman 8/5/20 50 / 108

f(x) = ab(ax)b−1e−(ax)b, x > 0.

F (x) = 1− exp[−(ax)b], x > 0.

E[T ] = 200Γ(1 + 3) = 1200 hrs.

F (2000) = 1− exp[−(2000/200)1/3] = 0.884. 2

ISYE 6739 — Goldsman 8/5/20 50 / 108

Cauchy Distribution — A “fat-tailed” distribution good for disprovingthings!

f(x) =1

π(1 + x2)and F (x) =

arctan(x)

π, x ∈ R.

Theorem: The Cauchy distribution has an undefined mean and infinitevariance!

Weird Fact: X1, . . . , Xniid∼ Cauchy⇒

∑ni=1Xi/n ∼ Cauchy. Even if you

take the average of a bunch of Cauchys, you’re right back where you started!

ISYE 6739 — Goldsman 8/5/20 51 / 108

f(x) =1

π(1 + x2)and

F (x) =1

arctan(x)

π, x ∈ R.

ISYE 6739 — Goldsman 8/5/20 51 / 108

f(x) =1

π(1 + x2)and F (x) =

arctan(x)

π, x ∈ R.

ISYE 6739 — Goldsman 8/5/20 51 / 108

f(x) =1

π(1 + x2)and F (x) =

arctan(x)

π, x ∈ R.

ISYE 6739 — Goldsman 8/5/20 51 / 108

f(x) =1

π(1 + x2)and F (x) =

arctan(x)

π, x ∈ R.

ISYE 6739 — Goldsman 8/5/20 51 / 108

Alphabet Soup of Other Distributions

χ2 distribution — coming up when we talk Statistics

t distribution — coming up

F distribution — coming up

Pareto, LaPlace, Rayleigh, Gumbel, Johnson distributions

Etc. . . .

ISYE 6739 — Goldsman 8/5/20 52 / 108

Etc. . . .

ISYE 6739 — Goldsman 8/5/20 52 / 108

Etc. . . .

ISYE 6739 — Goldsman 8/5/20 52 / 108

Etc. . . .

ISYE 6739 — Goldsman 8/5/20 52 / 108

Etc. . . .

ISYE 6739 — Goldsman 8/5/20 52 / 108

Etc. . . .

ISYE 6739 — Goldsman 8/5/20 52 / 108

Normal Distribution: Basics

14 Computer Stuff

ISYE 6739 — Goldsman 8/5/20 53 / 108

Lesson 4.7 — Normal Distribution: Basics

The Normal Distribution is so important that we’re giving it an entire section.

Definition: X ∼ Nor(µ, σ2) if it has pdf

f(x) =1√

2πσ2exp

[−(x− µ)2

], ∀x ∈ R.

Remark: The Normal distribution is also called the Gaussian distribution.

Examples: Heights, weights, SAT scores, crop yields, and averages ofthings tend to be normal.

ISYE 6739 — Goldsman 8/5/20 54 / 108

f(x) =1√

2πσ2exp

[−(x− µ)2

], ∀x ∈ R.

ISYE 6739 — Goldsman 8/5/20 54 / 108

f(x) =1√

2πσ2exp

[−(x− µ)2

], ∀x ∈ R.

ISYE 6739 — Goldsman 8/5/20 54 / 108

f(x) =1√

2πσ2exp

[−(x− µ)2

], ∀x ∈ R.

ISYE 6739 — Goldsman 8/5/20 54 / 108

f(x) =1√

2πσ2exp

[−(x− µ)2

], ∀x ∈ R.

ISYE 6739 — Goldsman 8/5/20 54 / 108

The pdf f(x) is “bell-shaped” and symmetric around x = µ, with tails fallingoff quickly as you move away from µ.

Small σ2 corresponds to a “tall, skinny” bell curve; large σ2 gives a “short,fat” bell curve.

ISYE 6739 — Goldsman 8/5/20 55 / 108

Fun Fact (1):∫R f(x) dx = 1.

Proof: Transform to polar coordinates. Good luck. 2

Fun Fact (2): The cdf is

F (x) =

−∞

1√2πσ2

[−(t− µ)2

]dt = ??

Remark: No closed-form solution for this. Stay tuned.

Fun Facts (3) and (4): E[X] = µ and Var(X) = σ2.

Proof: Integration by parts or mgf (below).

ISYE 6739 — Goldsman 8/5/20 56 / 108

F (x) =

−∞

1√2πσ2

[−(t− µ)2

]dt = ??

ISYE 6739 — Goldsman 8/5/20 56 / 108

F (x) =

−∞

1√2πσ2

[−(t− µ)2

]dt = ??

ISYE 6739 — Goldsman 8/5/20 56 / 108

F (x) =

−∞

1√2πσ2

[−(t− µ)2

]dt = ??

ISYE 6739 — Goldsman 8/5/20 56 / 108

F (x) =

−∞

1√2πσ2

[−(t− µ)2

]dt = ??

ISYE 6739 — Goldsman 8/5/20 56 / 108

F (x) =

−∞

1√2πσ2

[−(t− µ)2

]dt = ??

ISYE 6739 — Goldsman 8/5/20 56 / 108

Fun Fact (5): If X is any Normal RV, then

P (µ− σ < X < µ+ σ) = 0.6827

P (µ− 2σ < X < µ+ 2σ) = 0.9545

P (µ− 3σ < X < µ+ 3σ) = 0.9973.

So almost all of the probability is contained within 3 standard deviations ofthe mean. (This is sort of what Toyota is referring to when it brags about“six-sigma” quality.)

Fun Fact (6): The mgf is MX(t) = exp(µt+ 12σ

Proof: Calculus (or look it up in a table of integrals).

ISYE 6739 — Goldsman 8/5/20 57 / 108

P (µ− σ < X < µ+ σ) = 0.6827

P (µ− 2σ < X < µ+ 2σ) = 0.9545

P (µ− 3σ < X < µ+ 3σ) = 0.9973.

ISYE 6739 — Goldsman 8/5/20 57 / 108

P (µ− σ < X < µ+ σ) = 0.6827

P (µ− 2σ < X < µ+ 2σ) = 0.9545

P (µ− 3σ < X < µ+ 3σ) = 0.9973.

ISYE 6739 — Goldsman 8/5/20 57 / 108

P (µ− σ < X < µ+ σ) = 0.6827

P (µ− 2σ < X < µ+ 2σ) = 0.9545

P (µ− 3σ < X < µ+ 3σ) = 0.9973.

ISYE 6739 — Goldsman 8/5/20 57 / 108

P (µ− σ < X < µ+ σ) = 0.6827

P (µ− 2σ < X < µ+ 2σ) = 0.9545

P (µ− 3σ < X < µ+ 3σ) = 0.9973.

ISYE 6739 — Goldsman 8/5/20 57 / 108

P (µ− σ < X < µ+ σ) = 0.6827

P (µ− 2σ < X < µ+ 2σ) = 0.9545

P (µ− 3σ < X < µ+ 3σ) = 0.9973.

ISYE 6739 — Goldsman 8/5/20 57 / 108

Theorem (Additive Property of Normals): If X1, . . . , Xn are independentwith Xi ∼ Nor(µi, σ

2i ), i = 1, . . . , n, then

Y ≡n∑i=1

aiXi + b ∼ Nor( n∑i=1

aiµi + b,

n∑i=1

a2iσ2i

So a linear combination of independent normals is itself normal.

ISYE 6739 — Goldsman 8/5/20 58 / 108

2i ), i = 1, . . . , n, then

Y ≡n∑i=1

aiµi + b,

n∑i=1

a2iσ2i

ISYE 6739 — Goldsman 8/5/20 58 / 108

2i ), i = 1, . . . , n, then

Y ≡n∑i=1

aiµi + b,

n∑i=1

a2iσ2i

ISYE 6739 — Goldsman 8/5/20 58 / 108

Proof: Since Y is a linear function,

MY (t) = M∑i aiXi+b(t)

= etbM∑i aiXi

= etbn∏i=1

MaiXi(t) (Xi’s independent)

= etbn∏i=1

MXi(ait) (mgf of linear function)

= etbn∏i=1

exp[µi(ait) +

2σ2i (ait)

(normal mgf)

[( n∑i=1

µiai + b)t+

( n∑i=1

a2iσ2i

and we are done by mgf uniqueness. 2

ISYE 6739 — Goldsman 8/5/20 59 / 108

MY (t) = M∑i aiXi+b(t) = etbM∑

i aiXi(t)

= etbn∏i=1

exp[µi(ait) +

2σ2i (ait)

(normal mgf)

[( n∑i=1

µiai + b)t+

( n∑i=1

a2iσ2i

ISYE 6739 — Goldsman 8/5/20 59 / 108

i aiXi(t)

= etbn∏i=1

exp[µi(ait) +

2σ2i (ait)

(normal mgf)

[( n∑i=1

µiai + b)t+

( n∑i=1

a2iσ2i

ISYE 6739 — Goldsman 8/5/20 59 / 108

i aiXi(t)

= etbn∏i=1

exp[µi(ait) +

2σ2i (ait)

(normal mgf)

[( n∑i=1

µiai + b)t+

( n∑i=1

a2iσ2i

ISYE 6739 — Goldsman 8/5/20 59 / 108

i aiXi(t)

= etbn∏i=1

exp[µi(ait) +

2σ2i (ait)

(normal mgf)

[( n∑i=1

µiai + b)t+

( n∑i=1

a2iσ2i

ISYE 6739 — Goldsman 8/5/20 59 / 108

i aiXi(t)

= etbn∏i=1

exp[µi(ait) +

2σ2i (ait)

(normal mgf)

[( n∑i=1

µiai + b)t+

( n∑i=1

a2iσ2i

ISYE 6739 — Goldsman 8/5/20 59 / 108

i aiXi(t)

= etbn∏i=1

exp[µi(ait) +

2σ2i (ait)

(normal mgf)

[( n∑i=1

µiai + b)t+

( n∑i=1

a2iσ2i

ISYE 6739 — Goldsman 8/5/20 59 / 108

Remark: A normal distribution is completely characterized by its mean andvariance.

By the above, we know that a linear combination of independent normals isstill normal. Therefore, when we add up independent normals, all we have todo is figure out the mean and variance — the normality of the sum comes forfree.

Example: X ∼ Nor(3, 4), Y ∼ Nor(4, 6), and X,Y are independent. Findthe distribution of 2X − 3Y .

Solution: This is normal with

E[2X − 3Y ] = 2E[X]− 3E[Y ] = 2(3)− 3(4) = −6

andVar(2X − 3Y ) = 4Var(X) + 9Var(Y ) = 70.

Thus, 2X − 3Y ∼ Nor(−6, 70). 2

ISYE 6739 — Goldsman 8/5/20 60 / 108

By the above, we know that a linear combination of independent normals isstill normal.

Therefore, when we add up independent normals, all we have todo is figure out the mean and variance — the normality of the sum comes forfree.

E[2X − 3Y ] = 2E[X]− 3E[Y ] = 2(3)− 3(4) = −6

Thus, 2X − 3Y ∼ Nor(−6, 70). 2

ISYE 6739 — Goldsman 8/5/20 60 / 108

E[2X − 3Y ] = 2E[X]− 3E[Y ] = 2(3)− 3(4) = −6

Thus, 2X − 3Y ∼ Nor(−6, 70). 2

ISYE 6739 — Goldsman 8/5/20 60 / 108

E[2X − 3Y ] = 2E[X]− 3E[Y ] = 2(3)− 3(4) = −6

Thus, 2X − 3Y ∼ Nor(−6, 70). 2

ISYE 6739 — Goldsman 8/5/20 60 / 108

E[2X − 3Y ] = 2E[X]− 3E[Y ] = 2(3)− 3(4) = −6

Thus, 2X − 3Y ∼ Nor(−6, 70). 2

ISYE 6739 — Goldsman 8/5/20 60 / 108

E[2X − 3Y ] = 2E[X]− 3E[Y ] = 2(3)− 3(4) = −6

Thus, 2X − 3Y ∼ Nor(−6, 70). 2

ISYE 6739 — Goldsman 8/5/20 60 / 108

E[2X − 3Y ] = 2E[X]− 3E[Y ] = 2(3)− 3(4) = −6

Thus, 2X − 3Y ∼ Nor(−6, 70). 2

ISYE 6739 — Goldsman 8/5/20 60 / 108

Corollary (of Additive Property Theorem):

X ∼ Nor(µ, σ2) ⇒ aX + b ∼ Nor(aµ+ b, a2σ2).

Proof: Immediate from the Additive Property after noting thatE[aX + b] = aµ+ b and Var(aX + b) = a2σ2. 2

Corollary (of Corollary):

X ∼ Nor(µ, σ2) ⇒ Z ≡ X − µσ

∼ Nor(0, 1).

Proof: Use above Corollary with a = 1/σ and b = −µ/σ. 2

The manipulation described in this corollary is referred to asstandardization.

ISYE 6739 — Goldsman 8/5/20 61 / 108

X ∼ Nor(µ, σ2) ⇒ Z ≡ X − µσ

∼ Nor(0, 1).

ISYE 6739 — Goldsman 8/5/20 61 / 108

X ∼ Nor(µ, σ2) ⇒ Z ≡ X − µσ

∼ Nor(0, 1).

ISYE 6739 — Goldsman 8/5/20 61 / 108

X ∼ Nor(µ, σ2) ⇒ Z ≡ X − µσ

∼ Nor(0, 1).

ISYE 6739 — Goldsman 8/5/20 61 / 108

X ∼ Nor(µ, σ2) ⇒ Z ≡ X − µσ

∼ Nor(0, 1).

ISYE 6739 — Goldsman 8/5/20 61 / 108

X ∼ Nor(µ, σ2) ⇒ Z ≡ X − µσ

∼ Nor(0, 1).

ISYE 6739 — Goldsman 8/5/20 61 / 108

X ∼ Nor(µ, σ2) ⇒ Z ≡ X − µσ

∼ Nor(0, 1).

ISYE 6739 — Goldsman 8/5/20 61 / 108

Standard Normal Distribution

14 Computer Stuff

ISYE 6739 — Goldsman 8/5/20 62 / 108

Lesson 4.8 — Standard Normal Distribution

Definition: The Nor(0, 1) is called the standard normal distribution,and is often denoted by Z.

The Nor(0, 1) is nice because there are tables available for its cdf.

You can standardize any normal RV X into a standard normal by applying thetransformation Z = (X − µ)/σ. Then you can use the cdf tables.

The pdf of the Nor(0, 1) is

φ(z) ≡ 1√2πe−z

2/2, z ∈ R.

The cdf isΦ(z) ≡

−∞φ(t) dt, z ∈ R.

ISYE 6739 — Goldsman 8/5/20 63 / 108

φ(z) ≡ 1√2πe−z

2/2, z ∈ R.

The cdf isΦ(z) ≡

−∞φ(t) dt, z ∈ R.

ISYE 6739 — Goldsman 8/5/20 63 / 108

φ(z) ≡ 1√2πe−z

2/2, z ∈ R.

The cdf isΦ(z) ≡

−∞φ(t) dt, z ∈ R.

ISYE 6739 — Goldsman 8/5/20 63 / 108

φ(z) ≡ 1√2πe−z

2/2, z ∈ R.

The cdf isΦ(z) ≡

−∞φ(t) dt, z ∈ R.

ISYE 6739 — Goldsman 8/5/20 63 / 108

φ(z) ≡ 1√2πe−z

2/2, z ∈ R.

The cdf isΦ(z) ≡

−∞φ(t) dt, z ∈ R.

ISYE 6739 — Goldsman 8/5/20 63 / 108

φ(z) ≡ 1√2πe−z

2/2, z ∈ R.

The cdf isΦ(z) ≡

−∞φ(t) dt, z ∈ R.

ISYE 6739 — Goldsman 8/5/20 63 / 108

Remarks: The following results are easy to derive, usually via symmetryarguments.

P (Z ≤ a) = Φ(a)

P (Z ≥ b) = 1− Φ(b)P (a ≤ Z ≤ b) = Φ(b)− Φ(a)

Φ(0) = 1/2Φ(−b) = P (Z ≤ −b) = P (Z ≥ b) = 1− Φ(b)P (−b ≤ Z ≤ b) = Φ(b)− Φ(−b) = 2Φ(b)− 1

P (µ− kσ ≤ X ≤ µ+ kσ) = P (−k ≤ Z ≤ k) = 2Φ(k)− 1.

So the probability that any normal RV is within k standard deviations of itsmean doesn’t depend on the mean or variance.

ISYE 6739 — Goldsman 8/5/20 64 / 108

P (Z ≤ a) = Φ(a)P (Z ≥ b) = 1− Φ(b)

P (a ≤ Z ≤ b) = Φ(b)− Φ(a)

P (µ− kσ ≤ X ≤ µ+ kσ) = P (−k ≤ Z ≤ k) = 2Φ(k)− 1.

ISYE 6739 — Goldsman 8/5/20 64 / 108

P (Z ≤ a) = Φ(a)P (Z ≥ b) = 1− Φ(b)P (a ≤ Z ≤ b) = Φ(b)− Φ(a)

P (µ− kσ ≤ X ≤ µ+ kσ) = P (−k ≤ Z ≤ k) = 2Φ(k)− 1.

ISYE 6739 — Goldsman 8/5/20 64 / 108

Φ(0) = 1/2

Φ(−b) = P (Z ≤ −b) = P (Z ≥ b) = 1− Φ(b)P (−b ≤ Z ≤ b) = Φ(b)− Φ(−b) = 2Φ(b)− 1

P (µ− kσ ≤ X ≤ µ+ kσ) = P (−k ≤ Z ≤ k) = 2Φ(k)− 1.

ISYE 6739 — Goldsman 8/5/20 64 / 108

Φ(0) = 1/2Φ(−b) = P (Z ≤ −b) = P (Z ≥ b) = 1− Φ(b)

P (−b ≤ Z ≤ b) = Φ(b)− Φ(−b) = 2Φ(b)− 1

P (µ− kσ ≤ X ≤ µ+ kσ) = P (−k ≤ Z ≤ k) = 2Φ(k)− 1.

ISYE 6739 — Goldsman 8/5/20 64 / 108

P (µ− kσ ≤ X ≤ µ+ kσ) = P (−k ≤ Z ≤ k) = 2Φ(k)− 1.

ISYE 6739 — Goldsman 8/5/20 64 / 108

P (µ− kσ ≤ X ≤ µ+ kσ)

= P (−k ≤ Z ≤ k) = 2Φ(k)− 1.

ISYE 6739 — Goldsman 8/5/20 64 / 108

P (µ− kσ ≤ X ≤ µ+ kσ) = P (−k ≤ Z ≤ k)

= 2Φ(k)− 1.

ISYE 6739 — Goldsman 8/5/20 64 / 108

P (µ− kσ ≤ X ≤ µ+ kσ) = P (−k ≤ Z ≤ k) = 2Φ(k)− 1.

ISYE 6739 — Goldsman 8/5/20 64 / 108

P (µ− kσ ≤ X ≤ µ+ kσ) = P (−k ≤ Z ≤ k) = 2Φ(k)− 1.

ISYE 6739 — Goldsman 8/5/20 64 / 108

Famous Nor(0, 1) table values. You can memorize these.

Or you can usesoftware calls, like NORMDIST in Excel (which calculates the cdf for anynormal distribution.)

z Φ(z) = P (Z ≤ z)0.00 0.5000

1.00 0.8413

1.28 0.8997 ≈ 0.90

1.645 0.9500

1.96 0.9750

2.33 0.9901 ≈ 0.99

3.00 0.9987

4.00 ≈ 1.0000

ISYE 6739 — Goldsman 8/5/20 65 / 108

Famous Nor(0, 1) table values. You can memorize these. Or you can usesoftware calls, like NORMDIST in Excel (which calculates the cdf for anynormal distribution.)

z Φ(z) = P (Z ≤ z)0.00 0.5000

1.00 0.8413

1.28 0.8997 ≈ 0.90

1.645 0.9500

1.96 0.9750

2.33 0.9901 ≈ 0.99

3.00 0.9987

4.00 ≈ 1.0000

ISYE 6739 — Goldsman 8/5/20 65 / 108

z Φ(z) = P (Z ≤ z)

0.00 0.5000

1.00 0.8413

1.28 0.8997 ≈ 0.90

1.645 0.9500

1.96 0.9750

2.33 0.9901 ≈ 0.99

3.00 0.9987

4.00 ≈ 1.0000

ISYE 6739 — Goldsman 8/5/20 65 / 108

z Φ(z) = P (Z ≤ z)0.00 0.5000

1.00 0.8413

1.28 0.8997 ≈ 0.90

1.645 0.9500

1.96 0.9750

2.33 0.9901 ≈ 0.99

3.00 0.9987

4.00 ≈ 1.0000

ISYE 6739 — Goldsman 8/5/20 65 / 108

z Φ(z) = P (Z ≤ z)0.00 0.5000

1.00 0.8413

1.28 0.8997 ≈ 0.90

1.645 0.9500

1.96 0.9750

2.33 0.9901 ≈ 0.99

3.00 0.9987

4.00 ≈ 1.0000

ISYE 6739 — Goldsman 8/5/20 65 / 108

z Φ(z) = P (Z ≤ z)0.00 0.5000

1.00 0.8413

1.28 0.8997 ≈ 0.90

1.645 0.9500

1.96 0.9750

2.33 0.9901 ≈ 0.99

3.00 0.9987

4.00 ≈ 1.0000

ISYE 6739 — Goldsman 8/5/20 65 / 108

z Φ(z) = P (Z ≤ z)0.00 0.5000

1.00 0.8413

1.28 0.8997 ≈ 0.90

1.645 0.9500

1.96 0.9750

2.33 0.9901 ≈ 0.99

3.00 0.9987

4.00 ≈ 1.0000

ISYE 6739 — Goldsman 8/5/20 65 / 108

z Φ(z) = P (Z ≤ z)0.00 0.5000

1.00 0.8413

1.28 0.8997 ≈ 0.90

1.645 0.9500

1.96 0.9750

2.33 0.9901 ≈ 0.99

3.00 0.9987

4.00 ≈ 1.0000

ISYE 6739 — Goldsman 8/5/20 65 / 108

z Φ(z) = P (Z ≤ z)0.00 0.5000

1.00 0.8413

1.28 0.8997 ≈ 0.90

1.645 0.9500

1.96 0.9750

2.33 0.9901 ≈ 0.99

3.00 0.9987

4.00 ≈ 1.0000

ISYE 6739 — Goldsman 8/5/20 65 / 108

z Φ(z) = P (Z ≤ z)0.00 0.5000

1.00 0.8413

1.28 0.8997 ≈ 0.90

1.645 0.9500

1.96 0.9750

2.33 0.9901 ≈ 0.99

3.00 0.9987

4.00 ≈ 1.0000

ISYE 6739 — Goldsman 8/5/20 65 / 108

z Φ(z) = P (Z ≤ z)0.00 0.5000

1.00 0.8413

1.28 0.8997 ≈ 0.90

1.645 0.9500

1.96 0.9750

2.33 0.9901 ≈ 0.99

3.00 0.9987

4.00 ≈ 1.0000

ISYE 6739 — Goldsman 8/5/20 65 / 108

By the earlier “Fun Facts” and then the discussion on the last two pages, theprobability that any normal RV is within k standard deviations of its mean is

P (µ− kσ ≤ X ≤ µ+ kσ) = 2Φ(k)− 1.

For k = 1, this probability is 2(0.8413)− 1 = 0.6827.

There is a 95% chance that a normal observation will be within 2 s.d.’s of itsmean.

99.7% of all observations are within 3 standard deviations of the mean!

Finally, note that Toyota’s six-sigma corresponds to 2Φ(6)− 1.= 1.0000.

ISYE 6739 — Goldsman 8/5/20 66 / 108

P (µ− kσ ≤ X ≤ µ+ kσ) = 2Φ(k)− 1.

ISYE 6739 — Goldsman 8/5/20 66 / 108

P (µ− kσ ≤ X ≤ µ+ kσ) = 2Φ(k)− 1.

ISYE 6739 — Goldsman 8/5/20 66 / 108

P (µ− kσ ≤ X ≤ µ+ kσ) = 2Φ(k)− 1.

ISYE 6739 — Goldsman 8/5/20 66 / 108

P (µ− kσ ≤ X ≤ µ+ kσ) = 2Φ(k)− 1.

ISYE 6739 — Goldsman 8/5/20 66 / 108

P (µ− kσ ≤ X ≤ µ+ kσ) = 2Φ(k)− 1.

ISYE 6739 — Goldsman 8/5/20 66 / 108

Famous Inverse Nor(0, 1) table values. Can also use software, such asExcel’s NORMINV function, which actually calculates inverses for any normaldistribution, not just standard normal.

Φ−1(p) is the value of z such that Φ(z) = p. Φ−1(p) is called the pthquantile of Z.

p Φ−1(p)

0.90 1.28

0.95 1.645

0.975 1.96

0.99 2.33

0.995 2.58

ISYE 6739 — Goldsman 8/5/20 67 / 108

p Φ−1(p)

0.90 1.28

0.95 1.645

0.975 1.96

0.99 2.33

0.995 2.58

ISYE 6739 — Goldsman 8/5/20 67 / 108

p Φ−1(p)

0.90 1.28

0.95 1.645

0.975 1.96

0.99 2.33

0.995 2.58

ISYE 6739 — Goldsman 8/5/20 67 / 108

p Φ−1(p)

0.90 1.28

0.95 1.645

0.975 1.96

0.99 2.33

0.995 2.58

ISYE 6739 — Goldsman 8/5/20 67 / 108

p Φ−1(p)

0.90 1.28

0.95 1.645

0.975 1.96

0.99 2.33

0.995 2.58

ISYE 6739 — Goldsman 8/5/20 67 / 108

p Φ−1(p)

0.90 1.28

0.95 1.645

0.975 1.96

0.99 2.33

0.995 2.58

ISYE 6739 — Goldsman 8/5/20 67 / 108

p Φ−1(p)

0.90 1.28

0.95 1.645

0.975 1.96

0.99 2.33

0.995 2.58

ISYE 6739 — Goldsman 8/5/20 67 / 108

p Φ−1(p)

0.90 1.28

0.95 1.645

0.975 1.96

0.99 2.33

0.995 2.58

ISYE 6739 — Goldsman 8/5/20 67 / 108

Example: X ∼ Nor(21, 4). Find P (19 < X < 22.5). Standardizing, weget

P (19 < X < 22.5)

(19− µσ

<X − µσ

<22.5− µ

(19− 21

2< Z <

22.5− 21

)= P (−1 < Z < 0.75)

= Φ(0.75)− Φ(−1)

= Φ(0.75)− [1− Φ(1)]

= 0.7734− [1− 0.8413] = 0.6147. 2

ISYE 6739 — Goldsman 8/5/20 68 / 108

P (19 < X < 22.5)

(19− µσ

<X − µσ

<22.5− µ

(19− 21

2< Z <

22.5− 21

)= P (−1 < Z < 0.75)

= Φ(0.75)− Φ(−1)

= Φ(0.75)− [1− Φ(1)]

= 0.7734− [1− 0.8413] = 0.6147. 2

ISYE 6739 — Goldsman 8/5/20 68 / 108

P (19 < X < 22.5)

(19− µσ

<X − µσ

<22.5− µ

(19− 21

2< Z <

22.5− 21

= P (−1 < Z < 0.75)

= Φ(0.75)− Φ(−1)

= Φ(0.75)− [1− Φ(1)]

= 0.7734− [1− 0.8413] = 0.6147. 2

ISYE 6739 — Goldsman 8/5/20 68 / 108

P (19 < X < 22.5)

(19− µσ

<X − µσ

<22.5− µ

(19− 21

2< Z <

22.5− 21

)= P (−1 < Z < 0.75)

= Φ(0.75)− Φ(−1)

= Φ(0.75)− [1− Φ(1)]

= 0.7734− [1− 0.8413] = 0.6147. 2

ISYE 6739 — Goldsman 8/5/20 68 / 108

P (19 < X < 22.5)

(19− µσ

<X − µσ

<22.5− µ

(19− 21

2< Z <

22.5− 21

)= P (−1 < Z < 0.75)

= Φ(0.75)− Φ(−1)

= Φ(0.75)− [1− Φ(1)]

= 0.7734− [1− 0.8413] = 0.6147. 2

ISYE 6739 — Goldsman 8/5/20 68 / 108

P (19 < X < 22.5)

(19− µσ

<X − µσ

<22.5− µ

(19− 21

2< Z <

22.5− 21

)= P (−1 < Z < 0.75)

= Φ(0.75)− Φ(−1)

= Φ(0.75)− [1− Φ(1)]

= 0.7734− [1− 0.8413] = 0.6147. 2

ISYE 6739 — Goldsman 8/5/20 68 / 108

P (19 < X < 22.5)

(19− µσ

<X − µσ

<22.5− µ

(19− 21

2< Z <

22.5− 21

)= P (−1 < Z < 0.75)

= Φ(0.75)− Φ(−1)

= Φ(0.75)− [1− Φ(1)]

= 0.7734− [1− 0.8413] = 0.6147. 2

ISYE 6739 — Goldsman 8/5/20 68 / 108

Example: Suppose that heights of men are M ∼ Nor(68, 4) and heights ofwomen are W ∼ Nor(65, 1).

Select a man and woman independently at random.

Find the probability that the woman is taller than the man.

Answer: Note that

W −M ∼ Nor(E[W −M ],Var(W −M))

∼ Nor(65− 68, 1 + 4) ∼ Nor(−3, 5).

P (W > M) = P (W −M > 0)

0 + 3√5

)= 1− Φ(3/

.= 1− 0.910 = 0.090. 2

ISYE 6739 — Goldsman 8/5/20 69 / 108

Answer: Note that

∼ Nor(65− 68, 1 + 4) ∼ Nor(−3, 5).

P (W > M) = P (W −M > 0)

0 + 3√5

)= 1− Φ(3/

.= 1− 0.910 = 0.090. 2

ISYE 6739 — Goldsman 8/5/20 69 / 108

Answer: Note that

∼ Nor(65− 68, 1 + 4) ∼ Nor(−3, 5).

P (W > M) = P (W −M > 0)

0 + 3√5

)= 1− Φ(3/

.= 1− 0.910 = 0.090. 2

ISYE 6739 — Goldsman 8/5/20 69 / 108

Answer: Note that

∼ Nor(65− 68, 1 + 4) ∼ Nor(−3, 5).

P (W > M) = P (W −M > 0)

0 + 3√5

)= 1− Φ(3/

.= 1− 0.910 = 0.090. 2

ISYE 6739 — Goldsman 8/5/20 69 / 108

Answer: Note that

∼ Nor(65− 68, 1 + 4) ∼ Nor(−3, 5).

P (W > M) = P (W −M > 0)

0 + 3√5

)= 1− Φ(3/

.= 1− 0.910 = 0.090. 2

ISYE 6739 — Goldsman 8/5/20 69 / 108

Answer: Note that

∼ Nor(65− 68, 1 + 4) ∼ Nor(−3, 5).

P (W > M) = P (W −M > 0)

0 + 3√5

)= 1− Φ(3/

.= 1− 0.910 = 0.090. 2

ISYE 6739 — Goldsman 8/5/20 69 / 108

Answer: Note that

∼ Nor(65− 68, 1 + 4) ∼ Nor(−3, 5).

P (W > M) = P (W −M > 0)

0 + 3√5

= 1− Φ(3/√

5).= 1− 0.910 = 0.090. 2

ISYE 6739 — Goldsman 8/5/20 69 / 108

Answer: Note that

∼ Nor(65− 68, 1 + 4) ∼ Nor(−3, 5).

P (W > M) = P (W −M > 0)

0 + 3√5

)= 1− Φ(3/

.= 1− 0.910 = 0.090. 2

ISYE 6739 — Goldsman 8/5/20 69 / 108

Answer: Note that

∼ Nor(65− 68, 1 + 4) ∼ Nor(−3, 5).

P (W > M) = P (W −M > 0)

0 + 3√5

)= 1− Φ(3/

.= 1− 0.910 = 0.090. 2

ISYE 6739 — Goldsman 8/5/20 69 / 108

Sample Mean of Normals

14 Computer Stuff

ISYE 6739 — Goldsman 8/5/20 70 / 108

Lesson 4.9 — Sample Mean of Normals

The sample mean of X1, . . . , Xn is X̄ ≡∑n

i=1Xi/n.

Corollary (of old theorem):X1, . . . , Xn

iid∼ Nor(µ, σ2)⇒ X̄ ∼ Nor(µ, σ2/n).

Proof: By previous work, as long as X1, . . . , Xn are iid something, we haveE[X̄] = µ and Var(X̄) = σ2/n. Since X̄ is a linear combination ofindependent normals, it’s also normal. Done. 2

Remark: This result is very significant! As the number of observationsincreases, Var(X̄) gets smaller (while E[X̄] remains constant). In fact, it’scalled the Law of Large Numbers.

In the upcoming statistics portion of the course, we’ll learn that this makes X̄an excellent estimator for the mean µ, which is typically unknown inpractice.

ISYE 6739 — Goldsman 8/5/20 71 / 108

i=1Xi/n.

ISYE 6739 — Goldsman 8/5/20 71 / 108

i=1Xi/n.

ISYE 6739 — Goldsman 8/5/20 71 / 108

i=1Xi/n.

Proof: By previous work, as long as X1, . . . , Xn are iid something, we have

E[X̄] = µ and Var(X̄) = σ2/n. Since X̄ is a linear combination ofindependent normals, it’s also normal. Done. 2

ISYE 6739 — Goldsman 8/5/20 71 / 108

i=1Xi/n.

Proof: By previous work, as long as X1, . . . , Xn are iid something, we haveE[X̄] = µ and Var(X̄) = σ2/n.

Since X̄ is a linear combination ofindependent normals, it’s also normal. Done. 2

ISYE 6739 — Goldsman 8/5/20 71 / 108

i=1Xi/n.

ISYE 6739 — Goldsman 8/5/20 71 / 108

i=1Xi/n.

ISYE 6739 — Goldsman 8/5/20 71 / 108

i=1Xi/n.

ISYE 6739 — Goldsman 8/5/20 71 / 108

Example: Suppose that X1, . . . , Xniid∼ Nor(µ, 16). Find the sample size n

such thatP (|X̄ − µ| ≤ 1) ≥ 0.95.

How many observations should you take so that X̄ will have a good chance ofbeing close to µ?

Solution: Note that X̄ ∼ Nor(µ, 16/n). Then

P (|X̄ − µ| ≤ 1) = P (−1 ≤ X̄ − µ ≤ 1)

4/√n≤ X̄ − µ

4/√n≤ 1

4/√n

(−√n

4≤ Z ≤

)= 2Φ(

√n/4)− 1.

ISYE 6739 — Goldsman 8/5/20 72 / 108

such thatP (|X̄ − µ| ≤ 1) ≥ 0.95.

P (|X̄ − µ| ≤ 1) = P (−1 ≤ X̄ − µ ≤ 1)

4/√n≤ X̄ − µ

4/√n≤ 1

4/√n

(−√n

4≤ Z ≤

)= 2Φ(

√n/4)− 1.

ISYE 6739 — Goldsman 8/5/20 72 / 108

such thatP (|X̄ − µ| ≤ 1) ≥ 0.95.

P (|X̄ − µ| ≤ 1) = P (−1 ≤ X̄ − µ ≤ 1)

4/√n≤ X̄ − µ

4/√n≤ 1

4/√n

(−√n

4≤ Z ≤

)= 2Φ(

√n/4)− 1.

ISYE 6739 — Goldsman 8/5/20 72 / 108

such thatP (|X̄ − µ| ≤ 1) ≥ 0.95.

P (|X̄ − µ| ≤ 1) = P (−1 ≤ X̄ − µ ≤ 1)

4/√n≤ X̄ − µ

4/√n≤ 1

4/√n

(−√n

4≤ Z ≤

)= 2Φ(

√n/4)− 1.

ISYE 6739 — Goldsman 8/5/20 72 / 108

such thatP (|X̄ − µ| ≤ 1) ≥ 0.95.

P (|X̄ − µ| ≤ 1) = P (−1 ≤ X̄ − µ ≤ 1)

4/√n≤ X̄ − µ

4/√n≤ 1

4/√n

(−√n

4≤ Z ≤

)= 2Φ(

√n/4)− 1.

ISYE 6739 — Goldsman 8/5/20 72 / 108

such thatP (|X̄ − µ| ≤ 1) ≥ 0.95.

P (|X̄ − µ| ≤ 1) = P (−1 ≤ X̄ − µ ≤ 1)

4/√n≤ X̄ − µ

4/√n≤ 1

4/√n

(−√n

4≤ Z ≤

= 2Φ(√n/4)− 1.

ISYE 6739 — Goldsman 8/5/20 72 / 108

such thatP (|X̄ − µ| ≤ 1) ≥ 0.95.

P (|X̄ − µ| ≤ 1) = P (−1 ≤ X̄ − µ ≤ 1)

4/√n≤ X̄ − µ

4/√n≤ 1

4/√n

(−√n

4≤ Z ≤

)= 2Φ(

√n/4)− 1.

ISYE 6739 — Goldsman 8/5/20 72 / 108

Now we have to find n such that this probability is at least 0.95. . . .

2Φ(√n/4)− 1 ≥ 0.95 iff

Φ(√n/4) ≥ 0.975 iff

4≥ Φ−1(0.975) = 1.96

iff n ≥ 61.47 or 62.

So if you take the average of 62 observations, then X̄ has a 95% chance ofbeing within 1 of µ. 2

ISYE 6739 — Goldsman 8/5/20 73 / 108

2Φ(√n/4)− 1 ≥ 0.95 iff

Φ(√n/4) ≥ 0.975 iff

4≥ Φ−1(0.975) = 1.96

iff n ≥ 61.47 or 62.

ISYE 6739 — Goldsman 8/5/20 73 / 108

2Φ(√n/4)− 1 ≥ 0.95 iff

Φ(√n/4) ≥ 0.975 iff

4≥ Φ−1(0.975) = 1.96

iff n ≥ 61.47 or 62.

ISYE 6739 — Goldsman 8/5/20 73 / 108

2Φ(√n/4)− 1 ≥ 0.95 iff

Φ(√n/4) ≥ 0.975 iff

4≥ Φ−1(0.975) = 1.96

iff n ≥ 61.47 or 62.

ISYE 6739 — Goldsman 8/5/20 73 / 108

2Φ(√n/4)− 1 ≥ 0.95 iff

Φ(√n/4) ≥ 0.975 iff

4≥ Φ−1(0.975) = 1.96

iff n ≥ 61.47 or 62.

ISYE 6739 — Goldsman 8/5/20 73 / 108

2Φ(√n/4)− 1 ≥ 0.95 iff

Φ(√n/4) ≥ 0.975 iff

4≥ Φ−1(0.975) = 1.96

iff n ≥ 61.47 or 62.

ISYE 6739 — Goldsman 8/5/20 73 / 108

The Central Limit Theorem + Proof

14 Computer Stuff

ISYE 6739 — Goldsman 8/5/20 74 / 108

Lesson 4.10 — The Central Limit Theorem + Proof

The Central Limit Theorem is the most-important theorem in probability andstatistics!

CLT: Suppose X1, . . . , Xn are iid with E[Xi] = µ and Var(Xi) = σ2. Thenas n→∞,

∑ni=1Xi − nµσ√n

=X̄ − µσ/√n

=X̄ − E[X̄]√

Var(X̄)

d→ Nor(0, 1),

where “ d→ ” means that the cdf of Zn→ the Nor(0, 1) cdf.

ISYE 6739 — Goldsman 8/5/20 75 / 108

=X̄ − µσ/√n

=X̄ − E[X̄]√

Var(X̄)

d→ Nor(0, 1),

ISYE 6739 — Goldsman 8/5/20 75 / 108

=X̄ − µσ/√n

=X̄ − E[X̄]√

Var(X̄)

d→ Nor(0, 1),

ISYE 6739 — Goldsman 8/5/20 75 / 108

=X̄ − µσ/√n

=X̄ − E[X̄]√

Var(X̄)

d→ Nor(0, 1),

ISYE 6739 — Goldsman 8/5/20 75 / 108

=X̄ − µσ/√n

=X̄ − E[X̄]√

Var(X̄)

d→ Nor(0, 1),

ISYE 6739 — Goldsman 8/5/20 75 / 108

=X̄ − µσ/√n

=X̄ − E[X̄]√

Var(X̄)

d→ Nor(0, 1),

ISYE 6739 — Goldsman 8/5/20 75 / 108

=X̄ − µσ/√n

=X̄ − E[X̄]√

Var(X̄)

d→ Nor(0, 1),

ISYE 6739 — Goldsman 8/5/20 75 / 108

Slightly Honors Informal Proof: Suppose that the mgf MX(t) of the Xi’sexists and satisfies certain technical conditions that you don’t need to knowabout. (OK, MX(t) has to exist around t = 0, among other things.)

Moreover, without loss of generality (since we’re standardizing anyway) andfor notational convenience, we’ll assume that µ = 0 and σ2 = 1.

We will be done if we can show that the mgf of Zn converges to the mgf ofZ ∼ Nor(0, 1), i.e., we need to show that MZn(t)→ et

2/2 as n→∞.

To get things going, the mgf of Zn is

MZn(t) = M∑ni=1Xi/

√n (t)

= M∑ni=1Xi

(t/√n) (mgf of a linear function of a RV)

=[MX(t/

√n)]n (Xi’s are iid).

ISYE 6739 — Goldsman 8/5/20 76 / 108

2/2 as n→∞.

√n (t)

= M∑ni=1Xi

=[MX(t/

ISYE 6739 — Goldsman 8/5/20 76 / 108

2/2 as n→∞.

√n (t)

= M∑ni=1Xi

=[MX(t/

ISYE 6739 — Goldsman 8/5/20 76 / 108

2/2 as n→∞.

√n (t)

= M∑ni=1Xi

=[MX(t/

ISYE 6739 — Goldsman 8/5/20 76 / 108

2/2 as n→∞.

√n (t)

= M∑ni=1Xi

=[MX(t/

ISYE 6739 — Goldsman 8/5/20 76 / 108

2/2 as n→∞.

√n (t)

= M∑ni=1Xi

=[MX(t/

ISYE 6739 — Goldsman 8/5/20 76 / 108

Thus, taking logs, our goal is to show that

limn→∞

n `n(MX(t/

√n))

= t2/2.

If we let y = 1/√n, our revised goal is to show that

limy→0

`n(MX(ty)

= t2/2.

Before proceeding further, note that

limy→0

`n(MX(ty)

(MX(0)

)= `n(1) = 0 (3)

andlimy→0

M ′X(ty) = M ′X(0) = E[X] = µ = 0, (4)

where the last equality is from our standardization assumption.

ISYE 6739 — Goldsman 8/5/20 77 / 108

limn→∞

n `n(MX(t/

√n))

= t2/2.

limy→0

`n(MX(ty)

= t2/2.

limy→0

`n(MX(ty)

(MX(0)

)= `n(1) = 0 (3)

andlimy→0

M ′X(ty) = M ′X(0) = E[X] = µ = 0, (4)

ISYE 6739 — Goldsman 8/5/20 77 / 108

limn→∞

n `n(MX(t/

√n))

= t2/2.

limy→0

`n(MX(ty)

= t2/2.

limy→0

`n(MX(ty)

(MX(0)

)= `n(1) = 0 (3)

andlimy→0

M ′X(ty) = M ′X(0) = E[X] = µ = 0, (4)

ISYE 6739 — Goldsman 8/5/20 77 / 108

limn→∞

n `n(MX(t/

√n))

= t2/2.

limy→0

`n(MX(ty)

= t2/2.

limy→0

`n(MX(ty)

(MX(0)

)= `n(1) = 0 (3)

andlimy→0

M ′X(ty) = M ′X(0) = E[X] = µ = 0, (4)

ISYE 6739 — Goldsman 8/5/20 77 / 108

limn→∞

n `n(MX(t/

√n))

= t2/2.

limy→0

`n(MX(ty)

= t2/2.

limy→0

`n(MX(ty)

(MX(0)

)= `n(1) = 0 (3)

andlimy→0

M ′X(ty) = M ′X(0) = E[X] = µ = 0, (4)

ISYE 6739 — Goldsman 8/5/20 77 / 108

limn→∞

n `n(MX(t/

√n))

= t2/2.

limy→0

`n(MX(ty)

= t2/2.

limy→0

`n(MX(ty)

(MX(0)

)= `n(1) = 0 (3)

andlimy→0

M ′X(ty) = M ′X(0) = E[X] = µ = 0, (4)

ISYE 6739 — Goldsman 8/5/20 77 / 108

So after all of this build-up, we have

limy→0

`n(MX(ty)

= limy→0

tM ′X(ty)

2yMX(ty)(by (3) et L’Hôspital to deal with 0 / 0)

= limy→0

t2M ′′X(ty)

2MX(ty) + 2ytM ′X(ty)(by (4) et L’Hôspital encore)

=t2M ′′X(0)

2MX(0) + 0=

t2 E[X2]

2(E[X2] = σ2 − µ2 = 1).

ISYE 6739 — Goldsman 8/5/20 78 / 108

limy→0

`n(MX(ty)

= limy→0

tM ′X(ty)

= limy→0

t2M ′′X(ty)

=t2M ′′X(0)

2MX(0) + 0=

t2 E[X2]

2(E[X2] = σ2 − µ2 = 1).

ISYE 6739 — Goldsman 8/5/20 78 / 108

limy→0

`n(MX(ty)

= limy→0

tM ′X(ty)

= limy→0

t2M ′′X(ty)

=t2M ′′X(0)

2MX(0) + 0

=t2 E[X2]

2(E[X2] = σ2 − µ2 = 1).

ISYE 6739 — Goldsman 8/5/20 78 / 108

limy→0

`n(MX(ty)

= limy→0

tM ′X(ty)

= limy→0

t2M ′′X(ty)

=t2M ′′X(0)

2MX(0) + 0=

t2 E[X2]

2(E[X2] = σ2 − µ2 = 1).

ISYE 6739 — Goldsman 8/5/20 78 / 108

limy→0

`n(MX(ty)

= limy→0

tM ′X(ty)

= limy→0

t2M ′′X(ty)

=t2M ′′X(0)

2MX(0) + 0=

t2 E[X2]

2(E[X2] = σ2 − µ2 = 1).

ISYE 6739 — Goldsman 8/5/20 78 / 108

Remarks: We have a lot to say about such an important theorem.

(1) If n is large, then X̄ ≈ Nor(µ, σ2/n).

(2) The Xi’s don’t have to be normal for the CLT to work! It even workson discrete distributions!

(3) You usually need n ≥ 30 observations for the approximation to work well.(Need fewer observations if the Xi’s come from a symmetric distribution.)

(4) You can almost always use the CLT if the observations are iid.

(5) In fact, there are versions of the CLT that are actually a lot more generalthan the theorem presented here!

ISYE 6739 — Goldsman 8/5/20 79 / 108

Central Limit Theorem Examples

14 Computer Stuff

ISYE 6739 — Goldsman 8/5/20 80 / 108

Lesson 4.11 — Central Limit Theorem Examples

Example: To show that the Central Limit Theory really works, let’s add upjust n = 12 iid Unif(0,1)’s, U1, . . . , Un. Let Sn =

∑ni=1 Ui. Note that

E[Sn] = nE[Ui] = n/2, and Var(Sn) = nVar(Ui) = n/12. Therefore,

Zn ≡Sn − n/2√

n/12= Sn − 6 ≈ Nor(0, 1).

The histogram was compiled using 100,000 simulations of Z12. It works! 2

ISYE 6739 — Goldsman 8/5/20 81 / 108

∑ni=1 Ui.

Note thatE[Sn] = nE[Ui] = n/2, and Var(Sn) = nVar(Ui) = n/12. Therefore,

Zn ≡Sn − n/2√

n/12= Sn − 6 ≈ Nor(0, 1).

ISYE 6739 — Goldsman 8/5/20 81 / 108

Zn ≡Sn − n/2√

n/12= Sn − 6 ≈ Nor(0, 1).

ISYE 6739 — Goldsman 8/5/20 81 / 108

Zn ≡Sn − n/2√

n/12= Sn − 6 ≈ Nor(0, 1).

ISYE 6739 — Goldsman 8/5/20 81 / 108

Zn ≡Sn − n/2√

n/12= Sn − 6 ≈ Nor(0, 1).

ISYE 6739 — Goldsman 8/5/20 81 / 108

Example: Suppose X1, . . . , X100iid∼ Exp(1/1000).

Find P (950 ≤ X̄ ≤ 1050).

Solution: Recall that if Xi ∼ Exp(λ), then E[Xi] = 1/λ andVar(Xi) = 1/λ2.

Further, if X̄ is the sample mean based on n observations, then

E[X̄] = E[Xi] = 1/λ and

Var(X̄) = Var(Xi)/n = 1/(nλ2).

For our problem, λ = 1/1000 and n = 100, so that E[X̄] = 1000 andVar(X̄) = 10000.

ISYE 6739 — Goldsman 8/5/20 82 / 108

Find P (950 ≤ X̄ ≤ 1050).

E[X̄] = E[Xi] = 1/λ and

ISYE 6739 — Goldsman 8/5/20 82 / 108

Find P (950 ≤ X̄ ≤ 1050).

E[X̄] = E[Xi] = 1/λ and

ISYE 6739 — Goldsman 8/5/20 82 / 108

Find P (950 ≤ X̄ ≤ 1050).

E[X̄] = E[Xi] = 1/λ and

ISYE 6739 — Goldsman 8/5/20 82 / 108

Find P (950 ≤ X̄ ≤ 1050).

E[X̄] = E[Xi] = 1/λ and

ISYE 6739 — Goldsman 8/5/20 82 / 108

Find P (950 ≤ X̄ ≤ 1050).

E[X̄] = E[Xi] = 1/λ and

ISYE 6739 — Goldsman 8/5/20 82 / 108

Find P (950 ≤ X̄ ≤ 1050).

E[X̄] = E[Xi] = 1/λ and

ISYE 6739 — Goldsman 8/5/20 82 / 108

So by the CLT,

P (950 ≤ X̄ ≤ 1050)

(950− E[X̄]√

Var(X̄)≤ X̄ − E[X̄]√

Var(X̄)≤ 1050− E[X̄]√

Var(X̄)

(950− 1000

100≤ Z ≤ 1050− 1000

2≤ Z ≤ 1

)= 2Φ(1/2)− 1 = 0.383. 2

Remark: This problem can be solved exactly if we have access to the ExcelErlang cdf function GAMMADIST. And what do you know, you end up withexactly the same answer of 0.383!

ISYE 6739 — Goldsman 8/5/20 83 / 108

So by the CLT,

P (950 ≤ X̄ ≤ 1050)

(950− E[X̄]√

Var(X̄)≤ X̄ − E[X̄]√

Var(X̄)≤ 1050− E[X̄]√

Var(X̄)

(950− 1000

100≤ Z ≤ 1050− 1000

2≤ Z ≤ 1

)= 2Φ(1/2)− 1 = 0.383. 2

ISYE 6739 — Goldsman 8/5/20 83 / 108

So by the CLT,

P (950 ≤ X̄ ≤ 1050)

(950− E[X̄]√

Var(X̄)≤ X̄ − E[X̄]√

Var(X̄)≤ 1050− E[X̄]√

Var(X̄)

(950− 1000

100≤ Z ≤ 1050− 1000

2≤ Z ≤ 1

)= 2Φ(1/2)− 1 = 0.383. 2

ISYE 6739 — Goldsman 8/5/20 83 / 108

So by the CLT,

P (950 ≤ X̄ ≤ 1050)

(950− E[X̄]√

Var(X̄)≤ X̄ − E[X̄]√

Var(X̄)≤ 1050− E[X̄]√

Var(X̄)

(950− 1000

100≤ Z ≤ 1050− 1000

2≤ Z ≤ 1

)= 2Φ(1/2)− 1 = 0.383. 2

ISYE 6739 — Goldsman 8/5/20 83 / 108

So by the CLT,

P (950 ≤ X̄ ≤ 1050)

(950− E[X̄]√

Var(X̄)≤ X̄ − E[X̄]√

Var(X̄)≤ 1050− E[X̄]√

Var(X̄)

(950− 1000

100≤ Z ≤ 1050− 1000

2≤ Z ≤ 1

)= 2Φ(1/2)− 1 = 0.383. 2

ISYE 6739 — Goldsman 8/5/20 83 / 108

Example: Suppose X1, . . . , X100 are iid from some distribution with mean1000 and standard deviation 1000.

Find P (950 ≤ X̄ ≤ 1050).

Solution: By exactly the same manipulations as in the previous example, theanswer .= 0.383.

Notice that we didn’t care whether or not the data came from an exponentialdistribution. We just needed the mean and variance. 2

ISYE 6739 — Goldsman 8/5/20 84 / 108

Example: Suppose X1, . . . , X100 are iid from some distribution with mean1000 and standard deviation 1000. Find P (950 ≤ X̄ ≤ 1050).

ISYE 6739 — Goldsman 8/5/20 84 / 108

Normal Approximation to the Binomial

Suppose Y ∼ Bin(n, p), where n is very large. In such cases, we usuallyapproximate the Binomial via an appropriate Normal distribution.

The CLT applies since Y =∑n

i=1Xi, where the Xi’s are iid Bern(p).

ThenY − E[Y ]√

Var(Y )=

Y − np√npq

≈ Nor(0, 1).

The usual rule of thumb for the Normal approximation to the Binomial is thatit works pretty well as long as np ≥ 5 and nq ≥ 5.

ISYE 6739 — Goldsman 8/5/20 85 / 108

ThenY − E[Y ]√

Var(Y )=

Y − np√npq

≈ Nor(0, 1).

ISYE 6739 — Goldsman 8/5/20 85 / 108

ThenY − E[Y ]√

Var(Y )=

Y − np√npq

≈ Nor(0, 1).

ISYE 6739 — Goldsman 8/5/20 85 / 108

ThenY − E[Y ]√

Var(Y )=

Y − np√npq

≈ Nor(0, 1).

ISYE 6739 — Goldsman 8/5/20 85 / 108

ThenY − E[Y ]√

Var(Y )=

Y − np√npq

≈ Nor(0, 1).

ISYE 6739 — Goldsman 8/5/20 85 / 108

Why do we need such an approximation?

Example: Suppose Y ∼ Bin(100, 0.8) and we want

P (Y ≥ 84) =100∑i=84

)(0.8)i(0.2)100−i.

Good luck with the binomial coefficients (they’re too big) and the number ofterms to sum up (it’s going to get tedious). I’ll come back to visit you in anhour.

The next example shows how to use the approximation.

Note that it incorporates a “continuity correction” to account for the factthat the Binomial is discrete while the Normal is continuous. If you don’twant to use it, don’t worry too much.

ISYE 6739 — Goldsman 8/5/20 86 / 108

P (Y ≥ 84) =

100∑i=84

)(0.8)i(0.2)100−i.

ISYE 6739 — Goldsman 8/5/20 86 / 108

P (Y ≥ 84) =

100∑i=84

)(0.8)i(0.2)100−i.

ISYE 6739 — Goldsman 8/5/20 86 / 108

P (Y ≥ 84) =

100∑i=84

)(0.8)i(0.2)100−i.

ISYE 6739 — Goldsman 8/5/20 86 / 108

P (Y ≥ 84) =

100∑i=84

)(0.8)i(0.2)100−i.

ISYE 6739 — Goldsman 8/5/20 86 / 108

Example: The Braves play 100 independent baseball games, each of whichthey have probability 0.8 of winning. What’s the probability that they win ≥84?

Y ∼ Bin(100, 0.8) and we want P (Y ≥ 84) (as in the last example). . .

P (Y ≥ 84) = P (Y ≥ 83.5) (“continuity correction”)

(Z ≥ 83.5− np

√npq

)(CLT)

(Z ≥ 83.5− 80√

)= P (Z ≥ 0.875) = 0.1908.

The actual answer (using the true Bin(100,0.8) distribution) turns out to be0.1923 — pretty close! 2

ISYE 6739 — Goldsman 8/5/20 87 / 108

(Z ≥ 83.5− np

√npq

)(CLT)

(Z ≥ 83.5− 80√

)= P (Z ≥ 0.875) = 0.1908.

ISYE 6739 — Goldsman 8/5/20 87 / 108

(Z ≥ 83.5− np

√npq

)(CLT)

(Z ≥ 83.5− 80√

)= P (Z ≥ 0.875) = 0.1908.

ISYE 6739 — Goldsman 8/5/20 87 / 108

(Z ≥ 83.5− np

√npq

)(CLT)

(Z ≥ 83.5− 80√

)= P (Z ≥ 0.875) = 0.1908.

ISYE 6739 — Goldsman 8/5/20 87 / 108

(Z ≥ 83.5− np

√npq

)(CLT)

(Z ≥ 83.5− 80√

)= P (Z ≥ 0.875) = 0.1908.

ISYE 6739 — Goldsman 8/5/20 87 / 108

(Z ≥ 83.5− np

√npq

)(CLT)

(Z ≥ 83.5− 80√

)= P (Z ≥ 0.875) = 0.1908.

ISYE 6739 — Goldsman 8/5/20 87 / 108

Extensions — Multivariate Normal Distribution

14 Computer Stuff

ISYE 6739 — Goldsman 8/5/20 88 / 108

Lesson 4.12 — Extensions — Multivariate Normal Distribution

Definition: (X,Y ) has the Bivariate Normal distribution if it has pdf

f(x, y) = C exp

−[z2X(x)− 2ρzX(x)zY (y) + z2Y (y)

]2(1− ρ2)

whereρ ≡ Corr(X,Y ), C ≡ 1

2πσXσY√

1− ρ2,

zX(x) ≡ x− µXσX

, and zY (y) ≡ y − µYσY

ISYE 6739 — Goldsman 8/5/20 89 / 108

f(x, y) = C exp

−[z2X(x)− 2ρzX(x)zY (y) + z2Y (y)

]2(1− ρ2)

2πσXσY√

1− ρ2,

ISYE 6739 — Goldsman 8/5/20 89 / 108

f(x, y) = C exp

−[z2X(x)− 2ρzX(x)zY (y) + z2Y (y)

]2(1− ρ2)

2πσXσY√

1− ρ2,

ISYE 6739 — Goldsman 8/5/20 89 / 108

f(x, y) = C exp

−[z2X(x)− 2ρzX(x)zY (y) + z2Y (y)

]2(1− ρ2)

2πσXσY√

1− ρ2,

ISYE 6739 — Goldsman 8/5/20 89 / 108

Pretty nasty joint pdf, eh?

In fact, X ∼ Nor(µX , σ2X) and Y ∼ Nor(µY , σ2Y ).

Example: (X,Y ) could be a person’s (height,weight). The two quantitiesare marginally normal, but positively correlated.

If you want to calculate bivariate normal probabilities, you’ll need to evaluatequantities like

P (a < X < b, c < Y < d) =

af(x, y) dx dy,

which will probably require numerical integration techniques.

ISYE 6739 — Goldsman 8/5/20 90 / 108

P (a < X < b, c < Y < d) =

af(x, y) dx dy,

ISYE 6739 — Goldsman 8/5/20 90 / 108

P (a < X < b, c < Y < d) =

af(x, y) dx dy,

ISYE 6739 — Goldsman 8/5/20 90 / 108

P (a < X < b, c < Y < d) =

af(x, y) dx dy,

ISYE 6739 — Goldsman 8/5/20 90 / 108

P (a < X < b, c < Y < d) =

af(x, y) dx dy,

ISYE 6739 — Goldsman 8/5/20 90 / 108

Fun Fact (which will come up later when we discuss regression): Theconditional distribution of Y given that X = x is also normal. In particular,

Y |X = x ∼ Nor(µY + ρ(σY /σX)(x− µX), σ2Y (1− ρ2)

Information about X helps to update the distribution of Y .

Example: Consider students at a university. Let X be their combined SATscores (Math and Verbal), and Y their freshman GPA (out of 4).Suppose a study reveals that

µX = 1300, µY = 2.3,

σ2X = 6400, σ2Y = 0.25, ρ = 0.6.

Find P (Y ≥ 2|X = 900).

ISYE 6739 — Goldsman 8/5/20 91 / 108

µX = 1300, µY = 2.3,

σ2X = 6400, σ2Y = 0.25, ρ = 0.6.

Find P (Y ≥ 2|X = 900).

ISYE 6739 — Goldsman 8/5/20 91 / 108

µX = 1300, µY = 2.3,

σ2X = 6400, σ2Y = 0.25, ρ = 0.6.

Find P (Y ≥ 2|X = 900).

ISYE 6739 — Goldsman 8/5/20 91 / 108

Example: Consider students at a university. Let X be their combined SATscores (Math and Verbal), and Y their freshman GPA (out of 4).

Suppose a study reveals that

µX = 1300, µY = 2.3,

σ2X = 6400, σ2Y = 0.25, ρ = 0.6.

Find P (Y ≥ 2|X = 900).

ISYE 6739 — Goldsman 8/5/20 91 / 108

µX = 1300, µY = 2.3,

σ2X = 6400, σ2Y = 0.25, ρ = 0.6.

Find P (Y ≥ 2|X = 900).

ISYE 6739 — Goldsman 8/5/20 91 / 108

µX = 1300, µY = 2.3,

σ2X = 6400, σ2Y = 0.25, ρ = 0.6.

Find P (Y ≥ 2|X = 900).

ISYE 6739 — Goldsman 8/5/20 91 / 108

µX = 1300, µY = 2.3,

σ2X = 6400, σ2Y = 0.25, ρ = 0.6.

Find P (Y ≥ 2|X = 900).

ISYE 6739 — Goldsman 8/5/20 91 / 108

First,

E[Y |X = 900] = µY + ρ(σY /σX)(x− µX)

= 2.3 + (0.6)(√

0.25/6400 )(900− 1300) = 0.8,

indicating that the expected GPA of a kid with 900 SAT’s will be 0.8.

Second,Var(Y |X = 900) = σ2Y (1− ρ2) = 0.16.

Thus,Y |X = 900 ∼ Nor(0.8, 0.16).

Now we can calculate

P (Y ≥ 2|X = 900) = P(Z ≥ 2− 0.8√

)= 1− Φ(3) = 0.0013.

This guy doesn’t have much chance of having a good GPA. 2

ISYE 6739 — Goldsman 8/5/20 92 / 108

First,

E[Y |X = 900] = µY + ρ(σY /σX)(x− µX)

= 2.3 + (0.6)(√

0.25/6400 )(900− 1300) = 0.8,

Second,Var(Y |X = 900) = σ2Y (1− ρ2) = 0.16.

Thus,Y |X = 900 ∼ Nor(0.8, 0.16).

P (Y ≥ 2|X = 900) = P(Z ≥ 2− 0.8√

)= 1− Φ(3) = 0.0013.

ISYE 6739 — Goldsman 8/5/20 92 / 108

First,

E[Y |X = 900] = µY + ρ(σY /σX)(x− µX)

= 2.3 + (0.6)(√

0.25/6400 )(900− 1300) = 0.8,

Second,Var(Y |X = 900) = σ2Y (1− ρ2) = 0.16.

Thus,Y |X = 900 ∼ Nor(0.8, 0.16).

P (Y ≥ 2|X = 900) = P(Z ≥ 2− 0.8√

)= 1− Φ(3) = 0.0013.

ISYE 6739 — Goldsman 8/5/20 92 / 108

First,

E[Y |X = 900] = µY + ρ(σY /σX)(x− µX)

= 2.3 + (0.6)(√

0.25/6400 )(900− 1300) = 0.8,

Second,Var(Y |X = 900) = σ2Y (1− ρ2) = 0.16.

Thus,Y |X = 900 ∼ Nor(0.8, 0.16).

P (Y ≥ 2|X = 900) = P(Z ≥ 2− 0.8√

)= 1− Φ(3) = 0.0013.

ISYE 6739 — Goldsman 8/5/20 92 / 108

First,

E[Y |X = 900] = µY + ρ(σY /σX)(x− µX)

= 2.3 + (0.6)(√

0.25/6400 )(900− 1300) = 0.8,

Second,Var(Y |X = 900) = σ2Y (1− ρ2) = 0.16.

Thus,Y |X = 900 ∼ Nor(0.8, 0.16).

P (Y ≥ 2|X = 900) = P(Z ≥ 2− 0.8√

)= 1− Φ(3) = 0.0013.

ISYE 6739 — Goldsman 8/5/20 92 / 108

First,

E[Y |X = 900] = µY + ρ(σY /σX)(x− µX)

= 2.3 + (0.6)(√

0.25/6400 )(900− 1300) = 0.8,

Second,Var(Y |X = 900) = σ2Y (1− ρ2) = 0.16.

Thus,Y |X = 900 ∼ Nor(0.8, 0.16).

P (Y ≥ 2|X = 900) = P(Z ≥ 2− 0.8√

= 1− Φ(3) = 0.0013.

ISYE 6739 — Goldsman 8/5/20 92 / 108

First,

E[Y |X = 900] = µY + ρ(σY /σX)(x− µX)

= 2.3 + (0.6)(√

0.25/6400 )(900− 1300) = 0.8,

Second,Var(Y |X = 900) = σ2Y (1− ρ2) = 0.16.

Thus,Y |X = 900 ∼ Nor(0.8, 0.16).

P (Y ≥ 2|X = 900) = P(Z ≥ 2− 0.8√

)= 1− Φ(3) = 0.0013.

ISYE 6739 — Goldsman 8/5/20 92 / 108

First,

E[Y |X = 900] = µY + ρ(σY /σX)(x− µX)

= 2.3 + (0.6)(√

0.25/6400 )(900− 1300) = 0.8,

Second,Var(Y |X = 900) = σ2Y (1− ρ2) = 0.16.

Thus,Y |X = 900 ∼ Nor(0.8, 0.16).

P (Y ≥ 2|X = 900) = P(Z ≥ 2− 0.8√

)= 1− Φ(3) = 0.0013.

ISYE 6739 — Goldsman 8/5/20 92 / 108

The bivariate normal distribution is easily generalized to the multivariate case.

Honors Definition: The random vectorX = (X1, . . . , Xk)T has the

multivariate normal distribution with mean vector µ = (µ1, . . . , µk)T

and k × k covariance matrix Σ = (σij) if it has multivariate pdf

f(x) =1

(2π)k/2|Σ|1/2exp

{−(x− µ)TΣ−1(x− µ)

}, x ∈ Rk,

where |Σ| and Σ−1 are the determinant and inverse of Σ, respectively.

It turns out that

E[Xi] = µi, Var(Xi) = σii, Cov(Xi, Xj) = σij .

Notation: X ∼ Nork(µ, Σ).

ISYE 6739 — Goldsman 8/5/20 93 / 108

f(x) =1

(2π)k/2|Σ|1/2exp

{−(x− µ)TΣ−1(x− µ)

}, x ∈ Rk,

It turns out that

ISYE 6739 — Goldsman 8/5/20 93 / 108

f(x) =1

(2π)k/2|Σ|1/2exp

{−(x− µ)TΣ−1(x− µ)

}, x ∈ Rk,

It turns out that

ISYE 6739 — Goldsman 8/5/20 93 / 108

f(x) =1

(2π)k/2|Σ|1/2exp

{−(x− µ)TΣ−1(x− µ)

}, x ∈ Rk,

It turns out that

ISYE 6739 — Goldsman 8/5/20 93 / 108

f(x) =1

(2π)k/2|Σ|1/2exp

{−(x− µ)TΣ−1(x− µ)

}, x ∈ Rk,

It turns out that

ISYE 6739 — Goldsman 8/5/20 93 / 108

f(x) =1

(2π)k/2|Σ|1/2exp

{−(x− µ)TΣ−1(x− µ)

}, x ∈ Rk,

It turns out that

ISYE 6739 — Goldsman 8/5/20 93 / 108

Extensions — Lognormal Distribution

14 Computer Stuff

ISYE 6739 — Goldsman 8/5/20 94 / 108

Lesson 4.13 — Extensions — Lognormal Distribution

Definition: If Y ∼ Nor(µY , σ2Y ), then X ≡ eY has the Lognormaldistribution with parameters (µY , σ

2Y ). This distribution has tremendous

uses, e.g., in the pricing of certain stock options.

Turns Out: The pdf and moments of the lognormal are

f(x) =1

xσY√

2πexp

{− 1

2σ2Y[`n(x)− µY ]2

}, x > 0,

E[X] = exp

{µY +

Var(X) = exp{

2µY + σ2Y}(

exp{σ2Y}− 1).

ISYE 6739 — Goldsman 8/5/20 95 / 108

f(x) =1

xσY√

2πexp

{− 1

2σ2Y[`n(x)− µY ]2

}, x > 0,

E[X] = exp

{µY +

Var(X) = exp{

2µY + σ2Y}(

exp{σ2Y}− 1).

ISYE 6739 — Goldsman 8/5/20 95 / 108

f(x) =1

xσY√

2πexp

{− 1

2σ2Y[`n(x)− µY ]2

}, x > 0,

E[X] = exp

{µY +

Var(X) = exp{

2µY + σ2Y}(

exp{σ2Y}− 1).

ISYE 6739 — Goldsman 8/5/20 95 / 108

f(x) =1

xσY√

2πexp

{− 1

2σ2Y[`n(x)− µY ]2

}, x > 0,

E[X] = exp

{µY +

Var(X) = exp{

2µY + σ2Y}(

exp{σ2Y}− 1).

ISYE 6739 — Goldsman 8/5/20 95 / 108

f(x) =1

xσY√

2πexp

{− 1

2σ2Y[`n(x)− µY ]2

}, x > 0,

E[X] = exp

{µY +

Var(X) = exp{

2µY + σ2Y}(

exp{σ2Y}− 1).

ISYE 6739 — Goldsman 8/5/20 95 / 108

Example: Suppose Y ∼ Nor(10, 4) and let X = eY . Then

P (X ≤ 1000) = P(Y ≤ `n(1000)

(Z ≤ `n(1000)− 10

)= Φ(−1.55) = 0.061. 2

Honors Example (How to Win a Nobel Prize:) It is well-known thatstock prices are closely related to the lognormal distribution. In fact, it’scommon to use the following model for a stock price at a fixed time t,

S(t) = S(0) exp

{(µ− σ2

)t+ σ

}, t ≥ 0,

where µ is related to the “drift” of the stock price (i.e., the natural rate ofincrease), σ is its “volatility” (how much the stock bounces around), S(0) isthe initial price, and Z is a standard normal RV.

ISYE 6739 — Goldsman 8/5/20 96 / 108

P (X ≤ 1000) = P(Y ≤ `n(1000)

= P(Z ≤ `n(1000)− 10

)= Φ(−1.55) = 0.061. 2

S(t) = S(0) exp

{(µ− σ2

)t+ σ

}, t ≥ 0,

ISYE 6739 — Goldsman 8/5/20 96 / 108

P (X ≤ 1000) = P(Y ≤ `n(1000)

(Z ≤ `n(1000)− 10

= Φ(−1.55) = 0.061. 2

S(t) = S(0) exp

{(µ− σ2

)t+ σ

}, t ≥ 0,

ISYE 6739 — Goldsman 8/5/20 96 / 108

P (X ≤ 1000) = P(Y ≤ `n(1000)

(Z ≤ `n(1000)− 10

)= Φ(−1.55) = 0.061. 2

S(t) = S(0) exp

{(µ− σ2

)t+ σ

}, t ≥ 0,

ISYE 6739 — Goldsman 8/5/20 96 / 108

P (X ≤ 1000) = P(Y ≤ `n(1000)

(Z ≤ `n(1000)− 10

)= Φ(−1.55) = 0.061. 2

S(t) = S(0) exp

{(µ− σ2

)t+ σ

}, t ≥ 0,

ISYE 6739 — Goldsman 8/5/20 96 / 108

P (X ≤ 1000) = P(Y ≤ `n(1000)

(Z ≤ `n(1000)− 10

)= Φ(−1.55) = 0.061. 2

S(t) = S(0) exp

{(µ− σ2

)t+ σ

}, t ≥ 0,

ISYE 6739 — Goldsman 8/5/20 96 / 108

P (X ≤ 1000) = P(Y ≤ `n(1000)

(Z ≤ `n(1000)− 10

)= Φ(−1.55) = 0.061. 2

S(t) = S(0) exp

{(µ− σ2

)t+ σ

}, t ≥ 0,

ISYE 6739 — Goldsman 8/5/20 96 / 108

An active area of finance is to estimate option prices.

For example, aso-called European call option C permits its owner, who pays an up-frontfee for the privilege, to purchase the stock at a pre-agreed strike price k, at apre-determined expiry date T .

For instance, suppose IBM is currently selling for $100 a share. If I think thatthe stock will go up in value, I may want to pay $3/share now for the right tobuy IBM at $105 three months from now.

If IBM is worth $120 three months from now, I’ll be able to buy it foronly $105, and will have made a profit of $120− $105− $3 = $12.

If IBM is selling for $107 three months hence, I can still buy it for $105,and will lose $1 (recouping $2 from my original option purchase).

If IBM is selling for $95, then I won’t exercise my option, and will walkaway with my tail between my legs having lost my original $3.

ISYE 6739 — Goldsman 8/5/20 97 / 108

An active area of finance is to estimate option prices. For example, aso-called European call option C permits its owner, who pays an up-frontfee for the privilege, to purchase the stock at a pre-agreed strike price k, at apre-determined expiry date T .

ISYE 6739 — Goldsman 8/5/20 97 / 108

So what is the option worth (and what should I pay for it)? Its expected valueis given by

E[C] = e−rTE[(S(T )− k

x+ ≡ max{0, x}.r, the “risk-free” interest rate (e.g., what you can get from a U.S.Treasury bond). This is used instead of the drift µ.

The term e−rT denotes the time-value of money, i.e., a depreciation termcorresponding to the interest I could’ve made had I used my money tobuy a Treasury note.

Black and Scholes won a Nobel Prize for calculating E[C]. We’ll get thesame answer via a different method — but, alas, no Nobel Prize.

ISYE 6739 — Goldsman 8/5/20 98 / 108

x+ ≡ max{0, x}.

r, the “risk-free” interest rate (e.g., what you can get from a U.S.Treasury bond). This is used instead of the drift µ.

ISYE 6739 — Goldsman 8/5/20 98 / 108

E[C] = e−rTE

[S(0) exp

{(r − σ2

)T + σ

}− k]+

= e−rT∫ ∞−∞

[S(0) exp

{(r − σ2

)T + σ

}− k]+

φ(z) dz

(via the standard conditioning argument)

= S(0)Φ(b+ σ√T )− ke−rTΦ(b) (after lots of algebra),

where φ(·) and Φ(·) are the Nor(0,1) pdf and cdf, respectively, and

b ≡rT − σ2T

2 − `n(k/S(0))

σ√T

There are many generalizations of this problem that are used in practicalfinance problems, but this is the starting point. Meanwhile, get your tickets toNorway or Sweden or wherever they give out the Nobel!

ISYE 6739 — Goldsman 8/5/20 99 / 108

E[C] = e−rTE

[S(0) exp

{(r − σ2

)T + σ

}− k]+

= e−rT∫ ∞−∞

[S(0) exp

{(r − σ2

)T + σ

}− k]+

φ(z) dz

b ≡rT − σ2T

2 − `n(k/S(0))

σ√T

ISYE 6739 — Goldsman 8/5/20 99 / 108

E[C] = e−rTE

[S(0) exp

{(r − σ2

)T + σ

}− k]+

= e−rT∫ ∞−∞

[S(0) exp

{(r − σ2

)T + σ

}− k]+

φ(z) dz

b ≡rT − σ2T

2 − `n(k/S(0))

σ√T

ISYE 6739 — Goldsman 8/5/20 99 / 108

E[C] = e−rTE

[S(0) exp

{(r − σ2

)T + σ

}− k]+

= e−rT∫ ∞−∞

[S(0) exp

{(r − σ2

)T + σ

}− k]+

φ(z) dz

b ≡rT − σ2T

2 − `n(k/S(0))

σ√T

ISYE 6739 — Goldsman 8/5/20 99 / 108

E[C] = e−rTE

[S(0) exp

{(r − σ2

)T + σ

}− k]+

= e−rT∫ ∞−∞

[S(0) exp

{(r − σ2

)T + σ

}− k]+

φ(z) dz

b ≡rT − σ2T

2 − `n(k/S(0))

σ√T

ISYE 6739 — Goldsman 8/5/20 99 / 108

E[C] = e−rTE

[S(0) exp

{(r − σ2

)T + σ

}− k]+

= e−rT∫ ∞−∞

[S(0) exp

{(r − σ2

)T + σ

}− k]+

φ(z) dz

b ≡rT − σ2T

2 − `n(k/S(0))

σ√T

ISYE 6739 — Goldsman 8/5/20 99 / 108

Computer Stuff

14 Computer Stuff

ISYE 6739 — Goldsman 8/5/20 100 / 108

Computer Stuff

Lesson 4.14 — Computer Stuff

Evaluating pmf’s / pdf’s and cdf’s

We can use various computer packages such as Excel, Minitab, R, SAS, etc.,to calculate pmf’s / pdf’s and cdf’s for a variety of common distributions. Forinstance, in Excel, we find the functions:

BINOMDIST = Binomial distributionEXPONDIST = ExponentialNEGBINOMDIST = Negative BinomialNORMDIST and NORMSDIST = Normal and Standard NormalPOISSON = Poisson

Functions such as NORMSINV and TINV can calculate the inverses of thestandard normal and t distributions, respectively.

ISYE 6739 — Goldsman 8/5/20 101 / 108

Computer Stuff

ISYE 6739 — Goldsman 8/5/20 101 / 108

Computer Stuff

ISYE 6739 — Goldsman 8/5/20 101 / 108

Computer Stuff

BINOMDIST = Binomial distribution

EXPONDIST = ExponentialNEGBINOMDIST = Negative BinomialNORMDIST and NORMSDIST = Normal and Standard NormalPOISSON = Poisson

ISYE 6739 — Goldsman 8/5/20 101 / 108

Computer Stuff

BINOMDIST = Binomial distributionEXPONDIST = Exponential

NEGBINOMDIST = Negative BinomialNORMDIST and NORMSDIST = Normal and Standard NormalPOISSON = Poisson

ISYE 6739 — Goldsman 8/5/20 101 / 108

Computer Stuff

BINOMDIST = Binomial distributionEXPONDIST = ExponentialNEGBINOMDIST = Negative Binomial

NORMDIST and NORMSDIST = Normal and Standard NormalPOISSON = Poisson

ISYE 6739 — Goldsman 8/5/20 101 / 108

Computer Stuff

BINOMDIST = Binomial distributionEXPONDIST = ExponentialNEGBINOMDIST = Negative BinomialNORMDIST and NORMSDIST = Normal and Standard Normal

POISSON = Poisson

ISYE 6739 — Goldsman 8/5/20 101 / 108

Computer Stuff

ISYE 6739 — Goldsman 8/5/20 101 / 108

Computer Stuff

ISYE 6739 — Goldsman 8/5/20 101 / 108

Computer Stuff

Simulating Random Variables

Motivation: Simulations are used to evaluate a variety of real-worldprocesses that contain inherent randomness, e.g., queueing systems, inventorysystems, manufacturing systems, etc. In order to run simulations, you need togenerate various RVs, e.g., arrival times, service times, failure times, etc.

Examples: There are numerous ways to simulate RVs.

The Excel function RAND simulates a Unif(0,1) RV.

If U1, U2iid∼ Unif(0,1), then U1 + U2 is Triangular(0,1,2). This is simply

RAND()+RAND() in Excel.

The Inverse Transform Theorem gives (−1/λ)`n(U) ∼ Exp(λ).

If U1, U2, . . . , Ukiid∼ Unif(0,1), then

∑ki=1(−1/λ)`n(Ui) is Erlangk(λ)

because it is the sum of iid Exp(λ)’s.

It can be shown that X = d`n(U)/`n(1− p)e ∼ Geom(p), where d·e isthe “ceiling” (integer round-up) function.

ISYE 6739 — Goldsman 8/5/20 102 / 108

Computer Stuff

ISYE 6739 — Goldsman 8/5/20 102 / 108

Computer Stuff

ISYE 6739 — Goldsman 8/5/20 102 / 108

Computer Stuff

ISYE 6739 — Goldsman 8/5/20 102 / 108

Computer Stuff

ISYE 6739 — Goldsman 8/5/20 102 / 108

Computer Stuff

ISYE 6739 — Goldsman 8/5/20 102 / 108

Computer Stuff

ISYE 6739 — Goldsman 8/5/20 102 / 108

Computer Stuff

ISYE 6739 — Goldsman 8/5/20 102 / 108

Computer Stuff

The simulation of RVs is actually the topic of another course, but we will endthis module with a remarkable method for generating normal RVs.

Theorem (Box and Muller): If U1, U2iid∼ Unif(0, 1), then

Z1 =√−2`n(U1) cos(2πU2) and Z2 =

√−2`n(U1) sin(2πU2)

are iid Nor(0,1).

Example: Suppose that U1 = 0.3 and U2 = 0.8 are realizations of two iidUnif(0,1)’s. Box–Muller gives the following two iid standard normals.

Z1 =√−2`n(U1) cos(2πU2) = 0.480

Z2 =√−2`n(U1) sin(2πU2) = − 1.476. 2

ISYE 6739 — Goldsman 8/5/20 103 / 108

Computer Stuff

Z1 =√−2`n(U1) cos(2πU2) and Z2 =

√−2`n(U1) sin(2πU2)

are iid Nor(0,1).

Z1 =√−2`n(U1) cos(2πU2) = 0.480

Z2 =√−2`n(U1) sin(2πU2) = − 1.476. 2

ISYE 6739 — Goldsman 8/5/20 103 / 108

Computer Stuff

Z1 =√−2`n(U1) cos(2πU2) and

Z2 =√−2`n(U1) sin(2πU2)

are iid Nor(0,1).

Z1 =√−2`n(U1) cos(2πU2) = 0.480

Z2 =√−2`n(U1) sin(2πU2) = − 1.476. 2

ISYE 6739 — Goldsman 8/5/20 103 / 108

Computer Stuff

Z1 =√−2`n(U1) cos(2πU2) and Z2 =

√−2`n(U1) sin(2πU2)

are iid Nor(0,1).

Z1 =√−2`n(U1) cos(2πU2) = 0.480

Z2 =√−2`n(U1) sin(2πU2) = − 1.476. 2

ISYE 6739 — Goldsman 8/5/20 103 / 108

Computer Stuff

Z1 =√−2`n(U1) cos(2πU2) and Z2 =

√−2`n(U1) sin(2πU2)

are iid Nor(0,1).

Z1 =√−2`n(U1) cos(2πU2) = 0.480

Z2 =√−2`n(U1) sin(2πU2) = − 1.476. 2

ISYE 6739 — Goldsman 8/5/20 103 / 108

Computer Stuff

Z1 =√−2`n(U1) cos(2πU2) and Z2 =

√−2`n(U1) sin(2πU2)

are iid Nor(0,1).

Z1 =√−2`n(U1) cos(2πU2) = 0.480

Z2 =√−2`n(U1) sin(2πU2) = − 1.476. 2

ISYE 6739 — Goldsman 8/5/20 103 / 108

Computer Stuff

Z1 =√−2`n(U1) cos(2πU2) and Z2 =

√−2`n(U1) sin(2πU2)

are iid Nor(0,1).

Z1 =√−2`n(U1) cos(2πU2) = 0.480

Z2 =√−2`n(U1) sin(2πU2) = − 1.476. 2

ISYE 6739 — Goldsman 8/5/20 103 / 108

Computer Stuff

Z1 =√−2`n(U1) cos(2πU2) and Z2 =

√−2`n(U1) sin(2πU2)

are iid Nor(0,1).

Z1 =√−2`n(U1) cos(2πU2) = 0.480

Z2 =√−2`n(U1) sin(2πU2) = − 1.476. 2

ISYE 6739 — Goldsman 8/5/20 103 / 108

Computer Stuff

Remarks:

There are many other ways to generate Nor(0,1)’s, but this is the perhapsthe easiest.

It is essential that the cosine and sine must be calculated in radians, notdegrees.

To get X ∼ Nor(µ, σ2) from Z ∼ Nor(0, 1), just take X = µ+ σZ.

Amazingly, it’s “Muller”, not “Müller”.See https://www.youtube.com/watch?v=nntGTK2Fhb0

ISYE 6739 — Goldsman 8/5/20 104 / 108

Computer Stuff

Remarks:

ISYE 6739 — Goldsman 8/5/20 104 / 108

Computer Stuff

Remarks:

ISYE 6739 — Goldsman 8/5/20 104 / 108

Computer Stuff

Remarks:

ISYE 6739 — Goldsman 8/5/20 104 / 108

Computer Stuff

Honors Proof: We follow the method from Module 3 to calculate the jointpdf of a function of two RVs. Namely, if we can express U1 = k1(Z1, Z2) andU2 = k2(Z1, Z2) for some functions k1(·, ·) and k2(·, ·), then the joint pdf of(Z1, Z2) is given by

g(z1, z2) = fU1

(k1(z1, z2)

(k2(z1, z2)

) ∣∣∣∣∂u1∂z1

∂u2∂z2− ∂u2∂z1

∂u1∂z2

∣∣∣∣=

∣∣∣∣∂u1∂z1

∂u2∂z2− ∂u2∂z1

∂u1∂z2

∣∣∣∣ (U1 and U2 are iid Unif(0,1)

In order to obtain the functions k1(Z1, Z2) and k2(Z1, Z2), note that

Z21 + Z2

2 = − 2 `n(U1)[

cos2(2πU2) + sin2(2πU2)]

= − 2 `n(U1),

so thatU1 = e−(Z

22 )/2.

ISYE 6739 — Goldsman 8/5/20 105 / 108

Computer Stuff

g(z1, z2) = fU1

(k1(z1, z2)

(k2(z1, z2)

) ∣∣∣∣∂u1∂z1

∂u2∂z2− ∂u2∂z1

∂u1∂z2

∣∣∣∣

∣∣∣∣∂u1∂z1

∂u2∂z2− ∂u2∂z1

∂u1∂z2

Z21 + Z2

2 = − 2 `n(U1)[

= − 2 `n(U1),

so thatU1 = e−(Z

22 )/2.

ISYE 6739 — Goldsman 8/5/20 105 / 108

Computer Stuff

g(z1, z2) = fU1

(k1(z1, z2)

(k2(z1, z2)

) ∣∣∣∣∂u1∂z1

∂u2∂z2− ∂u2∂z1

∂u1∂z2

∣∣∣∣=

∣∣∣∣∂u1∂z1

∂u2∂z2− ∂u2∂z1

∂u1∂z2

Z21 + Z2

2 = − 2 `n(U1)[

= − 2 `n(U1),

so thatU1 = e−(Z

22 )/2.

ISYE 6739 — Goldsman 8/5/20 105 / 108

Computer Stuff

g(z1, z2) = fU1

(k1(z1, z2)

(k2(z1, z2)

) ∣∣∣∣∂u1∂z1

∂u2∂z2− ∂u2∂z1

∂u1∂z2

∣∣∣∣=

∣∣∣∣∂u1∂z1

∂u2∂z2− ∂u2∂z1

∂u1∂z2

Z21 + Z2

2 = − 2 `n(U1)[

= − 2 `n(U1),

so thatU1 = e−(Z

22 )/2.

ISYE 6739 — Goldsman 8/5/20 105 / 108

Computer Stuff

g(z1, z2) = fU1

(k1(z1, z2)

(k2(z1, z2)

) ∣∣∣∣∂u1∂z1

∂u2∂z2− ∂u2∂z1

∂u1∂z2

∣∣∣∣=

∣∣∣∣∂u1∂z1

∂u2∂z2− ∂u2∂z1

∂u1∂z2

Z21 + Z2

2 = − 2 `n(U1)[

= − 2 `n(U1),

so thatU1 = e−(Z

22 )/2.

ISYE 6739 — Goldsman 8/5/20 105 / 108

Computer Stuff

g(z1, z2) = fU1

(k1(z1, z2)

(k2(z1, z2)

) ∣∣∣∣∂u1∂z1

∂u2∂z2− ∂u2∂z1

∂u1∂z2

∣∣∣∣=

∣∣∣∣∂u1∂z1

∂u2∂z2− ∂u2∂z1

∂u1∂z2

Z21 + Z2

2 = − 2 `n(U1)[

= − 2 `n(U1),

so thatU1 = e−(Z

22 )/2.

ISYE 6739 — Goldsman 8/5/20 105 / 108

Computer Stuff

This immediately implies that

Z21 = −2 `n(U1) cos2(2πU2)

= −2 `n(e−(Z

22 )/2)

cos2(2πU2)

= (Z21 + Z2

2 ) cos2(2πU2),

so that

2πarccos

√Z21

Z21 + Z2

2πarccos

(√Z21

Z21 + Z2

where we (non-rigorously) get rid of the “±” to balance off the fact that therange of y = arccos(x) is only regarded to be 0 ≤ y ≤ π (not 0 ≤ y ≤ 2π).

ISYE 6739 — Goldsman 8/5/20 106 / 108

Computer Stuff

Z21 = −2 `n(U1) cos2(2πU2)

= −2 `n(e−(Z

22 )/2)

cos2(2πU2)

= (Z21 + Z2

2 ) cos2(2πU2),

so that

2πarccos

√Z21

Z21 + Z2

2πarccos

(√Z21

Z21 + Z2

ISYE 6739 — Goldsman 8/5/20 106 / 108

Computer Stuff

Z21 = −2 `n(U1) cos2(2πU2)

= −2 `n(e−(Z

22 )/2)

cos2(2πU2)

= (Z21 + Z2

2 ) cos2(2πU2),

so that

2πarccos

√Z21

Z21 + Z2

2πarccos

(√Z21

Z21 + Z2

ISYE 6739 — Goldsman 8/5/20 106 / 108

Computer Stuff

Z21 = −2 `n(U1) cos2(2πU2)

= −2 `n(e−(Z

22 )/2)

cos2(2πU2)

= (Z21 + Z2

2 ) cos2(2πU2),

so that

2πarccos

√Z21

Z21 + Z2

2πarccos

(√Z21

Z21 + Z2

ISYE 6739 — Goldsman 8/5/20 106 / 108

Computer Stuff

Z21 = −2 `n(U1) cos2(2πU2)

= −2 `n(e−(Z

22 )/2)

cos2(2πU2)

= (Z21 + Z2

2 ) cos2(2πU2),

so that

2πarccos

√Z21

Z21 + Z2

2πarccos

(√Z21

Z21 + Z2

ISYE 6739 — Goldsman 8/5/20 106 / 108

Computer Stuff

Now some derivative fun:

∂u1∂zi

∂zie−(z

22)/2 = − zi e−(z

22)/2, i = 1, 2,

∂u2∂z1

2πarccos

(√z21

z21 + z22

−1√1− z21

z21+z22

√z21

z21 + z22(chain rule)

−1√z22

z21+z22

z21 + z22

)−1/2 ∂

z21z21 + z22

(chain rule again)

=−(z21 + z22)

4πz1z2

2z1z22

(z21 + z22)2=

−z22π(z21 + z22)

∂u2∂z2

2π(z21 + z22)(after similar algebra).

ISYE 6739 — Goldsman 8/5/20 107 / 108

Computer Stuff

Now some derivative fun:∂u1∂zi

∂zie−(z

= − zi e−(z21+z

22)/2, i = 1, 2,

∂u2∂z1

2πarccos

(√z21

z21 + z22

−1√1− z21

z21+z22

√z21

−1√z22

z21+z22

z21 + z22

)−1/2 ∂

z21z21 + z22

(chain rule again)

=−(z21 + z22)

4πz1z2

2z1z22

(z21 + z22)2=

−z22π(z21 + z22)

∂u2∂z2

ISYE 6739 — Goldsman 8/5/20 107 / 108

Computer Stuff

∂zie−(z

22)/2 = − zi e−(z

22)/2, i = 1, 2,

∂u2∂z1

2πarccos

(√z21

z21 + z22

−1√1− z21

z21+z22

√z21

−1√z22

z21+z22

z21 + z22

)−1/2 ∂

z21z21 + z22

(chain rule again)

=−(z21 + z22)

4πz1z2

2z1z22

(z21 + z22)2=

−z22π(z21 + z22)

∂u2∂z2

ISYE 6739 — Goldsman 8/5/20 107 / 108

Computer Stuff

∂zie−(z

22)/2 = − zi e−(z

22)/2, i = 1, 2,

∂u2∂z1

2πarccos

(√z21

z21 + z22

−1√1− z21

z21+z22

√z21

−1√z22

z21+z22

z21 + z22

)−1/2 ∂

z21z21 + z22

(chain rule again)

=−(z21 + z22)

4πz1z2

2z1z22

(z21 + z22)2=

−z22π(z21 + z22)

∂u2∂z2

ISYE 6739 — Goldsman 8/5/20 107 / 108

Computer Stuff

∂zie−(z

22)/2 = − zi e−(z

22)/2, i = 1, 2,

∂u2∂z1

2πarccos

(√z21

z21 + z22

−1√1− z21

z21+z22

√z21

−1√z22

z21+z22

z21 + z22

)−1/2 ∂

z21z21 + z22

(chain rule again)

=−(z21 + z22)

4πz1z2

2z1z22

(z21 + z22)2=

−z22π(z21 + z22)

∂u2∂z2

ISYE 6739 — Goldsman 8/5/20 107 / 108

Computer Stuff

∂zie−(z

22)/2 = − zi e−(z

22)/2, i = 1, 2,

∂u2∂z1

2πarccos

(√z21

z21 + z22

−1√1− z21

z21+z22

√z21

−1√z22

z21+z22

z21 + z22

)−1/2 ∂

z21z21 + z22

(chain rule again)

=−(z21 + z22)

4πz1z2

2z1z22

(z21 + z22)2=

−z22π(z21 + z22)

∂u2∂z2

ISYE 6739 — Goldsman 8/5/20 107 / 108

Computer Stuff

∂zie−(z

22)/2 = − zi e−(z

22)/2, i = 1, 2,

∂u2∂z1

2πarccos

(√z21

z21 + z22

−1√1− z21

z21+z22

√z21

−1√z22

z21+z22

z21 + z22

)−1/2 ∂

z21z21 + z22

(chain rule again)

=−(z21 + z22)

4πz1z2

2z1z22

(z21 + z22)2

=−z2

2π(z21 + z22), and

∂u2∂z2

ISYE 6739 — Goldsman 8/5/20 107 / 108

Computer Stuff

∂zie−(z

22)/2 = − zi e−(z

22)/2, i = 1, 2,

∂u2∂z1

2πarccos

(√z21

z21 + z22

−1√1− z21

z21+z22

√z21

−1√z22

z21+z22

z21 + z22

)−1/2 ∂

z21z21 + z22

(chain rule again)

=−(z21 + z22)

4πz1z2

2z1z22

(z21 + z22)2=

−z22π(z21 + z22)

∂u2∂z2

ISYE 6739 — Goldsman 8/5/20 107 / 108

Computer Stuff

∂zie−(z

22)/2 = − zi e−(z

22)/2, i = 1, 2,

∂u2∂z1

2πarccos

(√z21

z21 + z22

−1√1− z21

z21+z22

√z21

−1√z22

z21+z22

z21 + z22

)−1/2 ∂

z21z21 + z22

(chain rule again)

=−(z21 + z22)

4πz1z2

2z1z22

(z21 + z22)2=

−z22π(z21 + z22)

∂u2∂z2

ISYE 6739 — Goldsman 8/5/20 107 / 108

Computer Stuff

Then we finally have

g(z1, z2) =

∣∣∣∣∂u1∂z1

∂u2∂z2− ∂u2∂z1

∂u1∂z2

∣∣∣∣=

∣∣∣∣−z1 e−(z21+z22)/2 z12π(z21 + z22)

− z22π(z21 + z22)

z2 e−(z21+z22)/2

∣∣∣∣=

2πe−(z

22)/2 =

(1√2π

e−z21/2

)(1√2π

e−z22/2

which is the product of two iid Nor(0,1) pdf’s, so we are done!

ISYE 6739 — Goldsman 8/5/20 108 / 108

Computer Stuff

g(z1, z2) =

∣∣∣∣∂u1∂z1

∂u2∂z2− ∂u2∂z1

∂u1∂z2

∣∣∣∣

∣∣∣∣−z1 e−(z21+z22)/2 z12π(z21 + z22)

− z22π(z21 + z22)

z2 e−(z21+z22)/2

∣∣∣∣=

2πe−(z

22)/2 =

(1√2π

e−z21/2

)(1√2π

e−z22/2

ISYE 6739 — Goldsman 8/5/20 108 / 108

Computer Stuff

g(z1, z2) =

∣∣∣∣∂u1∂z1

∂u2∂z2− ∂u2∂z1

∂u1∂z2

∣∣∣∣=

∣∣∣∣−z1 e−(z21+z22)/2 z12π(z21 + z22)

− z22π(z21 + z22)

z2 e−(z21+z22)/2

∣∣∣∣

2πe−(z

22)/2 =

(1√2π

e−z21/2

)(1√2π

e−z22/2

ISYE 6739 — Goldsman 8/5/20 108 / 108

Computer Stuff

g(z1, z2) =

∣∣∣∣∂u1∂z1

∂u2∂z2− ∂u2∂z1

∂u1∂z2

∣∣∣∣=

∣∣∣∣−z1 e−(z21+z22)/2 z12π(z21 + z22)

− z22π(z21 + z22)

z2 e−(z21+z22)/2

∣∣∣∣=

2πe−(z

(1√2π

e−z21/2

)(1√2π

e−z22/2

ISYE 6739 — Goldsman 8/5/20 108 / 108

Computer Stuff

g(z1, z2) =

∣∣∣∣∂u1∂z1

∂u2∂z2− ∂u2∂z1

∂u1∂z2

∣∣∣∣=

∣∣∣∣−z1 e−(z21+z22)/2 z12π(z21 + z22)

− z22π(z21 + z22)

z2 e−(z21+z22)/2

∣∣∣∣=

2πe−(z

22)/2 =

(1√2π

e−z21/2

)(1√2π

e−z22/2

ISYE 6739 — Goldsman 8/5/20 108 / 108

Computer Stuff

g(z1, z2) =

∣∣∣∣∂u1∂z1

∂u2∂z2− ∂u2∂z1

∂u1∂z2

∣∣∣∣=

∣∣∣∣−z1 e−(z21+z22)/2 z12π(z21 + z22)

− z22π(z21 + z22)

z2 e−(z21+z22)/2

∣∣∣∣=

2πe−(z

22)/2 =

(1√2π

e−z21/2

)(1√2π

e−z22/2

ISYE 6739 — Goldsman 8/5/20 108 / 108

dave goldsman - gatech.edu

Documents