10 s241 moment generating functions

7/24/2019 10 S241 Moment Generating Functions

1/51

Moment Generating Functions


2/51

0

0.1

0.2

0.3

0.4

0 5 10 15

1

b a

a b

( )f x

x

0

0.1

0.2

0.3

0.4

0 5 10 15

1

b a

a b

( )f x

x

1

b a

a b

( )f x

x

Continuous Distributions

The Uniform distribution from a to b

( )1

0 otherwise

a x bf x b a

=


3/51

The Norma distribution

!mean, standard de"iation #

( )( ) 2

221

2

x

f x e

=


4/51

0

0.1

0.2

-2 0 2 4 6 8 10

The $%&onentia distribution

( )0

0 0

xe xf x

x

=


5/51

Weibull distributionwith &arameters and.

( )Thus 1x

F x e

=

( ) ( ) 1and 0x

f x F x x e x

= =


6/51

The 'eibu densit()f!x#

0

0.1

0.2

0.3

0.4

0.5

0.*

0.+

0 1 2 3 4 5

!, 0.5) , 2#

!, 0.+) , 2#

!, 0.-) , 2#


7/51

The Gamma distribution

et the continuous random "ariabeX ha"e

densit( function/

( ) ( )1

0

0 0

x

x e xf x

x

=


8/51

$%&ectation of functions of

andom ariabes


9/51

X is discrete

( ) ( ) ( ) ( ) ( )i ix i

E g X g x p x g x p x = =

( ) ( ) ( )E g X g x f x dx

=

X is continuous


10/51

Moments of andom ariabes


11/51

( )k

k E X =

( )

( )

if is discrete

if is continuous

k

x

k

x p x X

x f x dx X

=

The kthmoment ofX.


12/51

the kthcentralmomentof X

( )0k

k E X =

( ) ( )

( ) ( )

if is discrete

if is continuous

k

x

k

x p x X

x f x dx X

=

where=1

= E!X# , the first moment ofX .


13/51

ues for e%&ectation


14/51

Rules/

[ ]1. where is a constantE c c c=[ ] [ ]2. where ) are constantsE aX b aE X b a b+ = +

( ) ( )20

23. "ar X E X = = ( ) ( )

22 2

2 1E X E X = =

( ) ( )24. "ar "ar aX b a X + =


15/51

Moment generating functions


16/51

Moment Generating function of a R.V.X

( )

( )

( )

if is discrete

if is continuous

tx

xtX

Xtx

e p x X

m t E ee f x dx X

= =


17/51

Examples

1. The inomia distribution !&arametersp, n#

( ) ( )tX txXx

m t E e e p x = =

( )0

1n

n xtx x

x

ne p p

x

=

=

( ) ( )0 0

1n n

x n xt x n x

x x

n ne p p a b

x x

= =

= =

( ) ( )1nn ta b e p p= + = +


18/51

( ) 0)1)2)

x

p x e xx

= = K

The moment generating function ofX , mX!t# is/

2. The oisson distribution !&arameter #

( ) ( )tX tx

X

xm t E e e p x = = 0

xn

tx

xe ex

== ( )

0 0

using

t

xt x

e u

x x

e ue e e e

x x

= =

= = =

( )1te

e

=


19/51

( )

0

0 0

xe x

f x x

=


20/51

( )

2

21

2

x

f x e

=

The moment generating function ofX , mX!t# is/

4. The 6tandard Norma distribution !, 0) , 1#

( ) ( )tX tx

Xm t E e e f x dx

= =

2 22

1

2

x tx

e dx

=

2

21

2

xtxe e dx

=


21/51

( )2 2 2 22 2

2 2 21 12 2

x tx t x tx t

Xm t e dx e e dx

+

= =

'e wi now use the fact that( ) 2

221

1 for a 0)

2

x b

ae dx a b

a

= >'e ha"e

com&eted

the s7uare

( ) 22 2

2 2 21

2

x tt t

e e dx e

= =

This is 1


22/51

( ) ( )1

0

0 0

x

x e xf x

x

=


23/51

'e use the fact

( )1

0

1 for a 0) 0

a

a bx

bx e dx a ba

= > >

( ) ( )

( )1

0

t x

Xm t x e dx

=

( )

( )

( )( )1

0

t xtx e dx

tt

= =

$7ua to 1


24/51

ro&erties of

Moment Generating Functions


25/51

1. mX!0# , 1

( ) ( ) ( ) ( ) ( )0

) hence 0 1 1

tX X

X Xm t E e m E e E

= = = =

( )"# Gamma 8ist9n Xm tt

=

( )2

2i"# 6td Norma 8ist9nt

Xm t e=

( )iii# $%&onentia 8ist9n Xm tt

=

( ) ( )1

ii# oisson 8ist9nte

Xm t e

=( ) ( )i# inomia 8ist9n 1

nt

Xm t e p p= +

Note: the moment generating functions of the foowing

distributions satisf( the &ro&ert( mX!0# , 1


26/51

( ) 2 33212. 12 3

kkXm t t t t t

k

= + + + + + +K K

'e use the e%&ansion of the e%&onentia function/

2 3

12 3

ku u u ue u

k= + + + + + +K K

( ) ( )tXXm t E e=2 3

2 312 3

kkt t tE tX X X X

k

= + + + + + +

K K

( ) ( ) ( ) ( )2 32 31

2 3

k kt t ttE X E X E X E X k

= + + + + + +K K

2 3

1 2 31

2 3

k

k

t t tt

k

= + + + + + +K K


27/51

( ) ( ) ( )0

3. 0k

k

X X kk

t

dm m t

dt

=

= =

Now( ) 2 33211

2 3

kkXm t t t t t

k

= + + + + + +K K

( ) 2 1321 2 32 3

kkXm t t t kt

k

= + + + + +K K

( )2 13

1 22 1

kkt t tk

= + + + + +

K K

( ) 1and 0Xm =

( )( )

242 3

2 2

kkXm t t t t

k

= + + + + +

K K

( ) 2and 0Xm =( )

( )continuing we find 0k

X km =


28/51

( ) ( )i# inomia 8ist9n 1n

t

Xm t e p p= +

ro&ert( 3 is "er( usefu in determining the moments of a

random "ariabeX.

Examples

( ) ( ) ( )1

1n

t t

Xm t n e p p pe

= +

( ) ( ) ( )1

0 0 10 1n

Xm n e p p pe np

= + = = =

( ) ( )( ) ( ) ( )2 1

1 1 1n n

t t t t t

Xm t np n e p p e p e e p p e

= + + +

( ) ( )( ) ( )( )

2

2

1 1 1

1 1

nt t t t

nt t t

npe e p p n e p e p p

npe e p p ne p p

= + + + = + +

[ ] [ ] 2 2 21np np p np np q n p npq = + = + = + =


29/51

( ) ( )1

ii# oisson 8ist9nte

Xm t e

=

( ) ( ) ( )1 1t te e tt

Xm t e e e

+ = =

( ) ( ) ( ) ( )1 1 2 121

t t te t e t e tt

Xm t e e e e

+ + + = + = +

( ) ( ) ( )1 2 12 2 1

t te t e t t t

Xm t e e e e

+ + = + + +

( ) ( )1 2 12 3t te t e t te e e

+ + = + +

( ) ( ) ( )1 3 1 2 13 23

t t te t e t e t e e e

+ + += + +


30/51

( )

( )0 1 0

1

0 e

X

m e

+

= = =( )

( ) ( )0 01 0 1 02 22 0

e e

Xm e e

+ += = + = +

( )

3 0 2 0 0 3 2

3 0 3 3

t

Xm e e e = = + + = + +

To find the moments we set t , 0.


31/51

( )iii# $%&onentia 8ist9n Xm tt

=

( ) ( ) 1

X

d tdm t

dt t dt

= =

( ) ( ) ( ) ( )2 2

1 1t t = =

( ) ( ) ( ) ( ) ( )3 3

2 1 2Xm t t t = =

( ) ( ) ( ) ( ) ( ) ( )4 4

2 3 1 2 3Xm t t t = =

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )5 54

2 3 4 1 4Xm t t t = =

( )

( ) ( ) ( )

1

kk

Xm t k t

=


32/51

Thus

( ) ( )

2

1

1

0Xm

= = = =

( ) ( )3

2 2

20 2Xm

= = =

( ) ( ) ( ) ( )1

0 kk

k X k

km k

= = =


33/51

( ) 2 332112 3

kkXm t t t t t

k

= + + + + + +K K

The moments for the e%&onentia distribution can be cacuated in

an aternati"e wa(. This is note b( e%&anding mX!t# in &owers of t

and e7uating the coefficients of t

k

to the coefficients in/

( ) 2 31 1 111Xm t u u utt u

= = = = + + + +

L

2 3

2 31

t t t

= + + + +L

$7uating the coefficients of tk we get/

1 or

kkk k

k

k

= =


34/51

( )2

2t

Xm t e=

Te moments for te standard normal distribution

'e use the e%&ansion of eu.2 3

0

1 2 3

k ku

k

u u u ue u

k k

=

= = + + + + + + L L

( ) ( ) ( ) ( ) ( )

2 2 2

22

2

2 3

2 2 2

21

2 3

t

kt t t

tXm t e

k= = + + + + + +L L

2 4 * 212 2 3

1 1 11

2 2 2 3 2

k

kt t t t

k

= + + + + + +L L

'e now e7uate the coefficients tkin/

( )

( )

2 22211

2 2

k kk kXm t t t t t

k k

= + + + + + + +K K K


35/51

:f k is odd/ k, 0.

( )2

1

2 2 k kk k

=For e"en 2k/

( )2

2 or

2 k k

k

k

=

( )1 2 3 4 2

2 4Thus 0) 1) 0) 3

2 2 2 = = = = = =


36/51

!ummar"

Moments

Moment generating functions


37/51

Moments of Random Variables

( )kk E X =

( )

( )

if is discrete

if is continuous

k

x

k

x p x X

x f x dx X

=

Te moment generating function

( )( )

( )

if is discrete

if is continuous

tx

xtX

Xtx

e p x X m t E e

e f x dx X

= =


38/51

Examples

( ) ( )1 0)1) 2) )n xxnp x p p x n

x

= =

K


( ) ( ) ( )1n n

t t

Xm t e p p e p q= + = +

( ) 0)1)2)

x

p x e x

x

= = K


( ) ( )1te

Xm t e

=


39/51

( )

0

0 0

xe xf x

x

=


40/51

( ) ( )

1 0

0 0

xx e xf x

x

=


41/51

1. mX!0# , 1

( ) 2 33212. 12 3

kkXm t t t t t

k

= + + + + + +K K

( ) ( ) ( )0

3. 0k

kX X kk

t

dm m tdt

=

= =

#roperties of Moment Generating $unctions

( ) 1i.e. 0Xm =

( ) 20Xm =

( ) 30 ) etcXm =


42/51

et lX !t# , n mX!t# , the og of the moment generating

function

( ) ( )Then 0 n 0 n1 0X Xl m= = =

Te log of Moment Generating $unctions

( ) ( ) ( ) ( )( )1 XX X

X X

m tl t m t

m t m t

= =

( ) ( ) ( ) ( )

( )

2

2

X X X

X

X

m t m t m t l t

m t

=

( ) ( )( )1

0 0

0X

X

X

ml

m

= = =

( ) ( ) ( ) ( )

( )[ ]

2

2 2

2 12

0 0 0 0

0

X X X

X

X

m m ml

m

= = =


43/51

Thus lX !t# , n mX!t# is "er( usefu for cacuating the

mean and "ariance of a random "ariabe

( )1. 0Xl =

( ) 22. 0Xl =


44/51

Examples


( ) ( ) ( )1n n

t t

Xm t e p p e p q= + = +

( ) ( ) ( )n n tX Xl t m t n e p q= = +

( ) 1 tX tl t n e pe p q = + ( )1

0Xl n p npp q = = =+

( ) ( ) ( )

( )2

t t t t

Xt

e p e p q e p e pl t n

e p q

+ =

+

( ) ( ) ( )

( )

2

20X

p p q p pl n npq

p q

+ = = =

+


45/51


( ) ( )1te

Xm t e

=

( ) ( ) ( )n 1tX Xl t m t e= =

( ) tXl t e =

( ) tXl t e =

( )0Xl = =

( )2 0Xl = =


46/51

3. The $%&onentia distribution !&arameter #

( )undefined

X tm t t

t


47/51

4. The 6tandard Norma distribution !, 0) , 1#

( )

2

2t

X

m t e

=( ) ( )

2

2n tX Xl t m t = =

( ) ( )) 1

X X

l t t l t = =

( ) ( )2Thus 0 0 and 0 1X Xl l = = = =


48/51

5. The Gamma distribution !&arameters ) #

( )Xm tt

=

( ) ( ) ( )n n nX Xl t m t t = =

( ) 1Xl tt t

= =

( ) ( ) ( ) ( )

( )

2

21 1Xl t t

t

= =

( ) ( )2 2


49/51

*. The ;his7uare distribution !degrees of freedom #

( ) ( )2

1 2Xm t t

=

( ) ( ) ( )n n 1 22

X Xl t m t t

= =

( ) ( )1

22 1 2 1 2Xl t t t

= =

( ) ( ) ( ) ( )( )

2

2

21 1 2 2

1 2X

l t tt

= =

( ) ( )2


50/51

Summary of Discrete Distributions

Name probability function p(x) Mean Variance

Momentgenerating

function MX(t)

DiscreteUniform

p(x) =1N

x=1,2,...,NN+1

2

N2-112

et

NetN-1et-1

Bernoullip(x) =

p x=1

q x=0

p pq q + pet

Binomialp(x) =

N

xpxqN-x

Np Npq (q + pet)N

Geometric p(x) =pqx-1 x=1,2,... 1p q

p2 pe

t

1-qet

NegativeBinomial p(x) =

x-1

k-1pkqx-k

x=k,k+1,...

k

p

kq

p2

pet

1-qetk

Poissonp(x) =

x

x!e- x=1,2,...

e(et-1)

Hypergeometric

p(x) =

A

x

N-A

n-x

N

n

n

A

N n

A

N

1-A

N

N-n

N-1

not useful


51/51

Summary of Continuous Distributions

Nameprobability

density function f(x) Mean arianceMoment generating

function MX(t)

ContinuousUniform

=otherwise

bxaabxf

0

1#!

a+b

2 (b-a)2

12 ebt-eat

[b-a]t

Exponential

10 s241 moment generating functions

Documents