maths probability expectation and conditional expectation lec6/8.pdf

7/27/2019 Maths Probability Expectation and Conditional Expectation lec6/8.pdf

http://slidepdf.com/reader/full/maths-probability-expectation-and-conditional-expectation-lec68pdf 1/12

Chapter 6

Expectation and Conditional Expectation

ectures 24 - 30

n this chapter, we introduce expected value or the mean of a random variable. First we define expectation forscrete random variables and then for general random variable. Finally we introduce the notion of conditionalxpectations using conditional probabilities.

Definition 6.1. Two random variables defined on a probability space are said to be

qual almost surely (in short a.s.) if

ow we give a useful characterization of discrete random variables.

heorem 6.0.23 Let be a discrete random variable defined on a probability space . Then

here exists a partition of and such that

here may be .

roof. Let be the distribution function of . Let be the set of all discontinuities of

. Here may be . Since is discrete, we have

et

hen is pairwise disjoint and

ow define

hen is a partition of and



Remark 6.0.6 If is a discrete random variable on a probability space , then the 'effective'

ange of is at the most countable. Here 'effective' range means those values taken by which hasositive probability. This leads to the name 'discrete' random variable.

Remark 6.0.7 If is a discrete random variable, then one can assume without the loss of generality

hat

nce if , then set and for .

heorem 6.0.24 Let be such that is a countable partition of .

hen if

hen

roof. For each , set

hen clearly

so if then . Therefore

his completes the proof.

Definition 6.2. Let be a discrete random variable represented by . Then

xpectation of denoted by is defined as



rovided the right hand side series converges absolutely.

Remark 6.0.8 In view of Remark 6.0.5., if has range , then

xample 6.0.37 Let be a Bernoulli( ) random variable. Then

ence

xample 6.0.38 Let be a Binomial random variable. Then

ence

ere we used the identity

xample 6.0.39 Let be a Poisson ( ) random variable. Then

ence

xample 6.0.40 Let be a Geometric ( ) random variable. Then

ence



heorem 6.0.25 (Properties of expectation) Let and be discrete random variables with finiteeans. Then

) If , then .

i) For

roof. (i) Let be a representation of . Then implies for all

. Hence

i) Let has a representation . Now by setting

ne can use same partition for and . Therefore

ence

Definition 6.3. (Simple random variable) A random variable is said to be simple if it is discrete and thestribution function has only finitely many discontinuities.

heorem 6.0.26Let be random variable in such that , then there exists a sequence

f simple random variables satisfying

) For each , .



i) For each as .

roof. For , define simple random variable as follows:

hen 's satisfies the following:

emma 6.0.3 Let be a non negative random variable and be a sequence of simple random

ariables satisfying (i) and (ii) of Theorem 6.0.25. Then exists and is given by

roof. Since , we have (see exercise). Hence

xists. Also since 's are simple, clearly,

herefore

ence to complete the proof, it suffices to show that for simple and ,

et

here is a partition of . Fix , set for and ,

nce , we have for each .

so

ence



rom the definition of we have

Hence

(6.0.1)

sing continuity property of probability, we have

ow let, in (6.0.1), we get

nce, is arbitrary, we get

his completes the proof.

Definition 6.4. The expectation of a non negative random variable is defined as

(6.0.2)

here is a sequence of simple random variables as in Theorem 6.0.25.

emark 6.0.9 One can define expectation of , non negative random variable, as

ut we use Definition 6.4., since it is more handy.

heorem 6.0.27 Let be a continuous non negative random variable with pdf . Then

roof. By using the simple functions given in the proof of Theorem 6.0.25, we get

(6.0.3)



here is the point given by the mean value theorem.

ence

Definition 6.5. Let be a random variable on . The mean or expectation of is said to

xists if either or is finite. In this case is defined as

here

ote that is the positive part and is the negetive part of

heorem 6.0.28 Let be a continuous random variable with finite mean and pdf . Then

roof. Set

ence



hen is a sequence of simple random variables such that

milarly, set

ence

hen

ow

(6.0.4)

nd

(6.0.5)

he last equality follows by the arguments from the proof of Theorem 6.0.26. Combining (6.0.4) and (6.0.4),e get

ow as in the proof of Theorem 6.0.26, we complete the proof.

We state the following useful properties of expectation. The proof follows by approximation argument usinghe corresponding properties of simple random variables

heorem 6.0.29 Let be random variables with finite mean. Then

) If , then .



i) For ,

ii) Let be a random variable such that . Then has finite mean and .

n the context of Riemann integration, one can recall the following convergence theorem.

` If is a sequence of continuous functions defined on the such that uniformly in

, then

e., to take limit inside the integral, one need uniform convergence of functions. In many situations in it isghly unlikely to get uniform convergence.

n fact uniform convergence is not required to take limit inside an integral. This is illustrated in the followingouple of theorem. The proof of them are beyond the scope of this course.

heorem 6.0.30 (Monotone convergence theorem) Let be an increasing sequence of nonnegative

andom variables such that . Then

Here means .]

heorem 6.0.31 (Dominated Convergence Theorem) Let be random variables such that

) has finite mean.

i)

ii)

hen

efinition 6.6. (Higher Order Moments) Let be a random variable. Then is called the th

oment of and is called the th central moment of . The second central moment is

alled the variance. Now we state the following theorem whose proof is beyond the scope of this course.

heorem 6.0.32 Let be a continuous random variable with pdf and be a continuous

unction such that the integral is finite. Then

he above theorem is generally referred as the ``Law of unconscious statistician'' since often users treat the



bove as a definition itself.ow we define conditional expectation denoted by E[Y|X] of the random variable Y given the information abouthe random variable X. If Y is a Bernoulli (p) random variable and X any discrete random variable, then wexpect E[Y|X = x] to be P{Y = 1|X = x}, since we know that EY = p = P{Y = 1}. i.e.,

Where is the conditional pmf of Y given X. Now since we expect conditional expectation to be

ner and any discrete random variable can be written as a liner combination of Bernoulli random variable weet the following definition.

Definition 6.7. Let are discrete random variable with conditional pmf . Then

onditional expectation of given is defined as

xample 6.0.41 Let be independent random variables with geometric distribution of parameter

. Calculate , where

et

or

ow

herefore

e.,

ow



When X and Y are discrete random variable. E[Y|X] is defined using conditional pmf of Y given X. Hence weefine E[Y|X] when X and Y are continuous random variable with joint pdf f in a similar way as follows.

Definition 6.8. Let be continuous random variable with conditional pdf . Then conditional

xpectation of given is defined as

Remark 6.0.10 One can extend the defination of E[Y|X] when X is any random variable (discrete,ontinuous or mixed) and Y is a any random variable with finite mean. But it is beyound the scope of thisourse.

heorem 6.0.33 (i) Let be discrete random variables with joint pmf , marginal pmfs and

respectively. Then if has finite mean, then

i) Let be continuous random variables with joint pdf , marginal pdfs and respectively.

hen if has finite mean, then

roof. We only prove (ii).

xample 6.0.42 Let be continuous random variables with joint pdf given by

nd and hence calculate . Note that

nd elsewhere. Hence for ,



so elsewhere. Therefore

maths probability expectation and conditional expectation lec6/8.pdf

Documents