moment generating function

EDWIN OKOAMPA BOADU

DEFINITIONDefinition 2.3.3. Let X be a random variable

with cdf FX. The moment generating function (mgf) of X (or FX), denoted MX(t), is

provided that the expectation exists for t in some neighborhood of 0. That is, there is an h>0 such that, for all t in –h<t<h, E[etX] exists.

tXX eEtM

If the expectation does not exist in a neighborhood of 0, we say that the moment generating function does not exist.

More explicitly, the moment generating function can be defined as:

variablesrandom discretefor

and , variablesrandom continuousfor

x

txX

txX

xXPetM

dxxfetM

Theorem 2.3.2: If X has mgf MX(t), then

where we define

0

)( 0

t

Xn

nnX tM

dt

dM

0nnXE X M

First note that etx can be approximated around zero using a Taylor series expansion:

2 30 0 2 0 3 0

2 32 3

1 10 0 0

2 6

12 6

tx t t tXM t E e E e te x t e x t e x

t tE x t E x E x

Note for any moment n:

Thus, as t0

1 2 2n

n n n nX Xn

dM M t E x E x t E x t

dt

nnX xEM 0

Leibnitz’s Rule: If f(x,θ), a(θ), and b(θ) are differentiable with respect to θ, then

b

a

b

a

dxxf

bd

dafa

d

dbfdxxf

d

d

,

,,,

Berger and Casella proof: Assume that we can differentiate under the integral using Leibnitz’s rule, we have

dxxfxe

dxxfedt

d

dxxfedt

dtM

dt

d

tx

tx

txX

Letting t->0, this integral simply becomes

This proof can be extended for any moment of the distribution function.

xf x dx E x

Moment Generating Functions for Specific DistributionsApplication to the Uniform Distribution:

abt

eee

tabdxab

etM

atbtb

a

txb

a

tx

X

11

Following the expansion developed earlier, we have:

222

3222

333222

333222

6

1

2

11

6

1

2

11

621

61

21

11

tbabatba

t

t

ab

aabbab

t

t

ab

abab

tab

tab

tab

tab

tab

tabtabtabtM X

Letting b=1 and a=0, the last expression becomes:

The first three moments of the uniform distribution are then:

32

24

1

6

1

2

11 ttttM X

4

16

24

10

3

12

6

10

2

10

3

2

1

X

X

X

M

M

M

Application to the Univariate Normal Distribution

dxx

tx

dxeetMx

txX

2

2

2

1

2

1exp

2

1

2

1 2

2

Focusing on the term in the exponent, we have

2

222

2

222

2

222

2

22

2

2

2

2

1

2

2

1

22

2

1

2

2

1

2

1

txx

txxx

txxx

txxxtx

The next state is to complete the square in the numerator.

422

42222

22

222

2

022

0

02

ttc

tttxx

tx

ctxx

The complete expression then becomes:

2 2 2 4 2

2 2

2

2 22

21 1

2 2

1 1

2 2

x t t txtx

x tt t

The moment generating function then becomes:

22

2

222

2

1exp

2

1exp

2

1

2

1exp

tt

dxtx

tttM X

Taking the first derivative with respect to t, we get:

Letting t->0, this becomes:

2221

2

1exp ttttM X

01XM

The second derivative of the moment generating function with respect to t yields:

Again, letting t->0 yields

2222

2222

2

1exp

2

1exp

tttt

tttM X

222 0 XM

Let X and Y be independent random variables with moment generating functions MX(t) and MY(t). Consider their sum Z=X+Y and its moment generating function:

t x ytz tx tyZ

tx tyX Y

M t E e E e E e e

E e E e M t M t

We conclude that the moment generating function for two independent random variables is equal to the product of the moment generating functions of each variable.

Skipping ahead slightly, the multivariate normal distribution function can be written as:

where Σ is the variance matrix and μ is a vector of means.

xxxf 1'

2

1exp

2

1

In order to derive the moment generating function, we now need a vector t. The moment generating function can then be defined as:

1exp ' '

2XM t t t t

Normal variables are independent if the variance matrix is a diagonal matrix.

Note that if the variance matrix is diagonal, the moment generating function for the normal can be written as:

1 2 3

21

22

23

2 2 2 2 2 21 1 2 2 3 3 1 1 2 2 3 3

2 2 2 21 1 1 1 2 2 2 3 3 3

0 01

exp ' ' 0 02

0 0

1exp

2

1 1 1exp

2 2 2

X

X X X

M t t t t

t t t t t t

t t t t

M t M t M t