maths probability expectation and conditional expectation lec6/8.pdf
Post on 14-Apr-2018
241 Views
Preview:
TRANSCRIPT
7/27/2019 Maths Probability Expectation and Conditional Expectation lec6/8.pdf
http://slidepdf.com/reader/full/maths-probability-expectation-and-conditional-expectation-lec68pdf 1/12
Chapter 6
Expectation and Conditional Expectation
ectures 24 - 30
n this chapter, we introduce expected value or the mean of a random variable. First we define expectation forscrete random variables and then for general random variable. Finally we introduce the notion of conditionalxpectations using conditional probabilities.
Definition 6.1. Two random variables defined on a probability space are said to be
qual almost surely (in short a.s.) if
ow we give a useful characterization of discrete random variables.
heorem 6.0.23 Let be a discrete random variable defined on a probability space . Then
here exists a partition of and such that
here may be .
roof. Let be the distribution function of . Let be the set of all discontinuities of
. Here may be . Since is discrete, we have
et
hen is pairwise disjoint and
ow define
hen is a partition of and
7/27/2019 Maths Probability Expectation and Conditional Expectation lec6/8.pdf
http://slidepdf.com/reader/full/maths-probability-expectation-and-conditional-expectation-lec68pdf 2/12
Remark 6.0.6 If is a discrete random variable on a probability space , then the 'effective'
ange of is at the most countable. Here 'effective' range means those values taken by which hasositive probability. This leads to the name 'discrete' random variable.
Remark 6.0.7 If is a discrete random variable, then one can assume without the loss of generality
hat
nce if , then set and for .
heorem 6.0.24 Let be such that is a countable partition of .
hen if
hen
roof. For each , set
hen clearly
so if then . Therefore
his completes the proof.
Definition 6.2. Let be a discrete random variable represented by . Then
xpectation of denoted by is defined as
7/27/2019 Maths Probability Expectation and Conditional Expectation lec6/8.pdf
http://slidepdf.com/reader/full/maths-probability-expectation-and-conditional-expectation-lec68pdf 3/12
rovided the right hand side series converges absolutely.
Remark 6.0.8 In view of Remark 6.0.5., if has range , then
xample 6.0.37 Let be a Bernoulli( ) random variable. Then
ence
xample 6.0.38 Let be a Binomial random variable. Then
ence
ere we used the identity
xample 6.0.39 Let be a Poisson ( ) random variable. Then
ence
xample 6.0.40 Let be a Geometric ( ) random variable. Then
ence
7/27/2019 Maths Probability Expectation and Conditional Expectation lec6/8.pdf
http://slidepdf.com/reader/full/maths-probability-expectation-and-conditional-expectation-lec68pdf 4/12
heorem 6.0.25 (Properties of expectation) Let and be discrete random variables with finiteeans. Then
) If , then .
i) For
roof. (i) Let be a representation of . Then implies for all
. Hence
i) Let has a representation . Now by setting
ne can use same partition for and . Therefore
ence
Definition 6.3. (Simple random variable) A random variable is said to be simple if it is discrete and thestribution function has only finitely many discontinuities.
heorem 6.0.26Let be random variable in such that , then there exists a sequence
f simple random variables satisfying
) For each , .
7/27/2019 Maths Probability Expectation and Conditional Expectation lec6/8.pdf
http://slidepdf.com/reader/full/maths-probability-expectation-and-conditional-expectation-lec68pdf 5/12
i) For each as .
roof. For , define simple random variable as follows:
hen 's satisfies the following:
emma 6.0.3 Let be a non negative random variable and be a sequence of simple random
ariables satisfying (i) and (ii) of Theorem 6.0.25. Then exists and is given by
roof. Since , we have (see exercise). Hence
xists. Also since 's are simple, clearly,
herefore
ence to complete the proof, it suffices to show that for simple and ,
et
here is a partition of . Fix , set for and ,
nce , we have for each .
so
ence
7/27/2019 Maths Probability Expectation and Conditional Expectation lec6/8.pdf
http://slidepdf.com/reader/full/maths-probability-expectation-and-conditional-expectation-lec68pdf 6/12
rom the definition of we have
Hence
(6.0.1)
sing continuity property of probability, we have
ow let, in (6.0.1), we get
nce, is arbitrary, we get
his completes the proof.
Definition 6.4. The expectation of a non negative random variable is defined as
(6.0.2)
here is a sequence of simple random variables as in Theorem 6.0.25.
emark 6.0.9 One can define expectation of , non negative random variable, as
ut we use Definition 6.4., since it is more handy.
heorem 6.0.27 Let be a continuous non negative random variable with pdf . Then
roof. By using the simple functions given in the proof of Theorem 6.0.25, we get
(6.0.3)
7/27/2019 Maths Probability Expectation and Conditional Expectation lec6/8.pdf
http://slidepdf.com/reader/full/maths-probability-expectation-and-conditional-expectation-lec68pdf 7/12
here is the point given by the mean value theorem.
ence
Definition 6.5. Let be a random variable on . The mean or expectation of is said to
xists if either or is finite. In this case is defined as
here
ote that is the positive part and is the negetive part of
heorem 6.0.28 Let be a continuous random variable with finite mean and pdf . Then
roof. Set
ence
7/27/2019 Maths Probability Expectation and Conditional Expectation lec6/8.pdf
http://slidepdf.com/reader/full/maths-probability-expectation-and-conditional-expectation-lec68pdf 8/12
hen is a sequence of simple random variables such that
milarly, set
ence
hen
ow
(6.0.4)
nd
(6.0.5)
he last equality follows by the arguments from the proof of Theorem 6.0.26. Combining (6.0.4) and (6.0.4),e get
ow as in the proof of Theorem 6.0.26, we complete the proof.
We state the following useful properties of expectation. The proof follows by approximation argument usinghe corresponding properties of simple random variables
heorem 6.0.29 Let be random variables with finite mean. Then
) If , then .
7/27/2019 Maths Probability Expectation and Conditional Expectation lec6/8.pdf
http://slidepdf.com/reader/full/maths-probability-expectation-and-conditional-expectation-lec68pdf 9/12
i) For ,
ii) Let be a random variable such that . Then has finite mean and .
n the context of Riemann integration, one can recall the following convergence theorem.
` If is a sequence of continuous functions defined on the such that uniformly in
, then
e., to take limit inside the integral, one need uniform convergence of functions. In many situations in it isghly unlikely to get uniform convergence.
n fact uniform convergence is not required to take limit inside an integral. This is illustrated in the followingouple of theorem. The proof of them are beyond the scope of this course.
heorem 6.0.30 (Monotone convergence theorem) Let be an increasing sequence of nonnegative
andom variables such that . Then
Here means .]
heorem 6.0.31 (Dominated Convergence Theorem) Let be random variables such that
) has finite mean.
i)
ii)
hen
efinition 6.6. (Higher Order Moments) Let be a random variable. Then is called the th
oment of and is called the th central moment of . The second central moment is
alled the variance. Now we state the following theorem whose proof is beyond the scope of this course.
heorem 6.0.32 Let be a continuous random variable with pdf and be a continuous
unction such that the integral is finite. Then
he above theorem is generally referred as the ``Law of unconscious statistician'' since often users treat the
7/27/2019 Maths Probability Expectation and Conditional Expectation lec6/8.pdf
http://slidepdf.com/reader/full/maths-probability-expectation-and-conditional-expectation-lec68pdf 10/12
bove as a definition itself.ow we define conditional expectation denoted by E[Y|X] of the random variable Y given the information abouthe random variable X. If Y is a Bernoulli (p) random variable and X any discrete random variable, then wexpect E[Y|X = x] to be P{Y = 1|X = x}, since we know that EY = p = P{Y = 1}. i.e.,
Where is the conditional pmf of Y given X. Now since we expect conditional expectation to be
ner and any discrete random variable can be written as a liner combination of Bernoulli random variable weet the following definition.
Definition 6.7. Let are discrete random variable with conditional pmf . Then
onditional expectation of given is defined as
xample 6.0.41 Let be independent random variables with geometric distribution of parameter
. Calculate , where
et
or
ow
herefore
e.,
ow
7/27/2019 Maths Probability Expectation and Conditional Expectation lec6/8.pdf
http://slidepdf.com/reader/full/maths-probability-expectation-and-conditional-expectation-lec68pdf 11/12
When X and Y are discrete random variable. E[Y|X] is defined using conditional pmf of Y given X. Hence weefine E[Y|X] when X and Y are continuous random variable with joint pdf f in a similar way as follows.
Definition 6.8. Let be continuous random variable with conditional pdf . Then conditional
xpectation of given is defined as
Remark 6.0.10 One can extend the defination of E[Y|X] when X is any random variable (discrete,ontinuous or mixed) and Y is a any random variable with finite mean. But it is beyound the scope of thisourse.
heorem 6.0.33 (i) Let be discrete random variables with joint pmf , marginal pmfs and
respectively. Then if has finite mean, then
i) Let be continuous random variables with joint pdf , marginal pdfs and respectively.
hen if has finite mean, then
roof. We only prove (ii).
xample 6.0.42 Let be continuous random variables with joint pdf given by
nd and hence calculate . Note that
nd elsewhere. Hence for ,
7/27/2019 Maths Probability Expectation and Conditional Expectation lec6/8.pdf
http://slidepdf.com/reader/full/maths-probability-expectation-and-conditional-expectation-lec68pdf 12/12
so elsewhere. Therefore
top related