the random variable - northeastern university

�THE RANDOM VARIABLE

Moral: There is no safety in numbers, or in anything else.

James Thurber, The Fairly Intelligent Fly

Chapter 2 covers probabilistic tools effective primarily for simple events withcountably many or equally probable outcomes. They are ineffective or compli-cated for problems with more involved events.

This chapter deals with the random variable approach, a powerful and effec-tive tool for many random problems. It is based on describing every outcome ofa random experiment as a unique number and thus makes many powerful toolsin calculus applicable.

Main Topics

� Concept of Random Variable� Cumulative Distribution Function� Probability Density Function� Uniform Distribution� Gaussian Distribution� Expectation and Moments� Functions of a Random Variable� Generation of Random Numbers

65

3.1 Concept of Random Variable


Recall that a (dependent) variable � � �� is a function �� of an independentvariable � such that to every value of � there corresponds a value of �. That is,�� is obtained by assigning a (unique) number to every value of �.

A random variable is simply a dependent variable as a function of an in-dependent variable — the outcomes of a random experiment. It is a numericaldescription of the outcomes. Specifically, to every outcome � of a random ex-periment, we assign a unique number ��. The function �� thus defined isthen a random variable (RV) (since its value is uncertain prior to performingthe experiment). The value � � �� for a given outcome �� is called itsrealization or the value on which the RV � takes.

Convention:

Upper case Latin letter = RVLower case Latin letter = the value

RVs may be grouped into three types, depending on the range of the function:

� A discrete RV is one that may take on only discrete values.� A continuous RV is one having a continuous range of values.� A mixed RV is one with both discrete and continuous values.

Only discrete RVs may be defined on a discrete sample space �. However, adiscrete RV may have a sample space consisting of a continuous range of pointsor of a mixture of such regions and isolated points.

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

Figure 3.1: Mapping of a random variable.

66


Example 3.2: Some Random Variables Defined on Die Rolling

Consider rolling a die. Let �� be the event that face shows up. Three RVs �, , and � are defined by the following assignments:

��

After defining RVs, the calculation of event probabilities reduces to that of prob-abilities of the RVs taking on certain values. For example,

��an even number shows up� � �� or 6� � �� or � ��

Example 3.3: Events in the Form of �� This example demonstrates that �� is an event. Consider the RV � ofExample 3.2 of rolling a die, that is, �� . Since � can only take onintegers � � � � � �, we have

��

Thus, �� .

� � � � � �

��

� ��

��

��

Figure 3.2: Illustration of �� as an event for Example 3.3.

68

3.2 Cumulative Distribution Function


3.2.1 Motivation and Definition

�� is an event that corresponds to the set of all outcomes ��’s for which�� . Its probability is of interest in many cases. This leads to definingthe cumulative distribution function (CDF) of a RV � as

� �� (3.2)

i.e., �� is the probability that RV � takes on a value not greater than �.It can be shown that

� A discrete RV is one having a stairway-type (and thus discontinuous) CDF(see Figure 3.3).

� A continuous RV is one having an absolutely continuous CDF (see Figure3.4).

�

� ��

Figure 3.3: The general pattern of the CDF of a discrete RV.

�

� ��

�

.......

......................................................................................

.............................................................................................................................................

..............................................................�

� ��

�

....... .......................................................................................................

....................................................................................................................................................................................................................

Figure 3.4: Some general patterns of the CDF of a continuous RV.

70


3.2.2 Properties of CDF

1. � � �� , since � �� is probability.2. � �� , which follows from (3.1).3. � �� , since �� is a sure event.4. � �� is nondecreasing as � increases:

� �� (3.3)

This follows from the fact that �� . Specifically,

� �� ?� �� ?� �� (3.4)

5. By (3.4),�� (3.5)

A special case is:

�� (3.6)

6. The CDF of a discrete RV � taking on values �� is given by

� �� if �� and ��

� ��

��

��

�� (3.7)

where �� , � � � � � �, are known as point masses and�� is the unit step function with a jump at � � ��, defined by

��

��

� � � �(3.8)

� � ��

� � � � � � �

� ��

72


Example 3.7: Determination of CDF: Power Consumption

Consider 3 independent machines, each having 75% of the time in operation anda power consumption of 1kW when in use. We wish to predict the average totalpower consumption of the 3 machines.

Let � � �total # of machines in operation�. Note that � is discrete since itcan only take on 0, 1, 2, or 3. Given � , the total power consumption is knownand thus the problem becomes how to find the CDF of �.

Note that this is a Bernoulli trial problem with a total of 3 trials. Then, from(2.27) with � � ��,

��

�

��

� ��

�

�

��

� ��

�

�

��

� ��

�

�

��

Check: ��

� (kW) � � �

� ��

��

�

��

��

��

��

��

��

Note that the CDF is of a stairway type since the RV is discrete.

74


Example 3.8: From CDF to Probability — Machine Failure

Let � be the service time (i.e., time before failure) of a machine. Suppose thatit has the following (approximate) CDF

� ��

� � �

� � ��

� � � � � � � �

� ��

��

��

This is a continuous RV since its CDF is continuous, though not smooth.Find the following probabilities:

�� (3.6)� ��

�� (3.5)� � ��

� ��

��

��

��

� � � ��

��

� � ��

��

��

Note that �� makes sense since � � �� will alwaysbe true given that � � �� is true.

75

3.3 Probability Density Function


The probability density function (PDF) of a continuous RV � is defined as

��

��(3.10)

which is a density function indicating where the RV values are more (or less)consolidated.

For a discrete RV � taking on values �� , its probability mass function(PMF) is the sequence of its point masses, defined by, for �� ,

��

��

�� (3.11)

where ��’s are point masses and � is the Kronecker delta function

��

��

�� (3.12)

By introducing the delta function

��

��

��

elsewhere(3.13)

the PDF of a RV � taking on values �� with point masses ��’s is

��

��

��

(3.7)�

��

��

�� (3.14)

which is a probabilistically weighted sum of delta functions.

�

��

��

��

��

��

��

� � �

��

��

�

��

��

�

��

��

��

�

��

��

�

��

� � �

��

��

�

��

��

�

Figure 3.5: PMF and PDF of a discrete RV.

76


Properties of PDF

1. Nonnegativity: �� , since � �� is nondecreasing (i.e., ��

�).2. Relation with CDF: Integrating PDF over �� yields CDF:

� ��(3.10)�

� �

� �� area under �� over interval �� (3.18)

3. Normalization property: By (3.18) and property 3 of CDF,� � �� (3.19)

4. Area under �� over interval �� is equal to �� :

��

�� (3.20)

which follows from (3.18) and (3.5):� ��

��

� ��

� ��

� ��

5. For a continuous RV � and any given value �, although �� is not animpossible event, its probability is zero: �� , �, since

��

��

�

�� ?� (3.21)

where ?� follows from the fact that the PDF of a continuous RV does not

involve a delta function. This is also clear from the fact that the CDF of acontinuous RV is continuous and thus �� .

�

�

��

��

Property 5 can be understood via the following example: Throwing a pointof zero size over an interval �� , the probability that the point is exactly on� � � for any given � in between � and � is zero. This is not true for a discreteor mixed RV because its PDF involves a delta function.

79


Example 3.9: CDF, PDF and PMF of Waveform Sampling

The following waveform �� of a period � � is sampled at a random time � .

� � � � � � �

��

�

�

��

Let � � �sampled value of ��. Note that � is a discrete RV taking onfour possible values: �� . The corresponding point masses are

��

Hence, its PDF, PMF and CDF are, respectively,

��(3.14)� ��

��(3.11)� ��

� ��(3.7)� ��

��

��

��

��

��

��

��

�

��

��

��

� ��

��

��

��

��

��

��

��

��

80


Example 3.10: The following waveform �� of a period � � � is sampledat a random time � .

��

��

�

��

��

��

Let � � �sampled value of ��. It will be clear that � is a mixed RV.

(a) Find PDF and CDF of �: By inspection,

�� is true 40% of the time ��

�� is always in between 0 and 4 ��

points in � �� are equally probable ��

� � � � �

where �� is the probability mass uniformly distributed over � ��.Hence

��

��

� ��

� ��

� � �

� � ��

� � � � � ��

(b) Find �� and �� :

�� (since no delta function at � � �)

��

��

��

��

��

�

��

or ��

��

��

�

� ��

��

��

��

��

��

��

��

� ��

��

��

81

3.4 Uniform Distribution

3.4 Uniform Distribution

A uniform random variable � over �� is one having a nonzero constant PDFover the single interval �� :

��

� � �

�

��

elsewhere(3.22)

for some � and �. In this case, we use notation

� � ��

Note that � � ��

� �

�

�

�� (total area = 1)

� ��

� ��

� � �

� � ��

�

�

��

��

��

� � � �

Clearly, a uniform RV is a continuous one. Thus, �� for any given�� and an evenly distributed discrete RV (e.g., � or of Example 3.2 for dierolling) cannot be a uniform RV.

��

��

��

��

� ��

��

�

Figure 3.7: The PDF and CDF of a uniform random variable.

Example 3.11: A capacitor has capacitance � � �� . What isthe probability that � is between �� and ��?

��

� ��

�

��

83

3.5 Gaussian Distribution and Central Limit Theorem


A Gaussian (or normal) random variable � is one having the PDF:

��

�

�

�!"�

��

�� (3.23)

for � � � � . Its CDF is

� ��

�

�!

� �

� "��

�� (3.24)

A shorthand for such a RV is (read as “� is Gaussian distributed with �� and �”)

� � � �� (3.25)

where �� and � are two parameters, called mean and variance, which will bestudied later. Note that this CDF has no closed-form expression.

Although the expression of the Gaussian PDF is weird, the Gaussian distri-bution is extremely popular and useful. It is the most important distribution andwill be treated extensively later. It arises in so many practical situations that itis honored as the “normal” distribution above all other distributions. It deservesthis honor well in view of the central limit theorem, to be studied later.

Some typical examples of Gaussian RVs are: thermal noise, measurementerror, test score of a large class, and height or weight of a large group of people.

.........

....

.........

....

.........

....

.........

....

.........

....

.........

...

.........

....

.........

....

.........

....

.........

....

.........

....

.........

................ ............. ............. ............. ............. ............. ............. ...

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

................ ............. ............. ............. .......��

��

��

��

.............................................................

.....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

............. ............. ............. ............. .............

............. ............. ............. ............. ............. ............. .............

............. ............. ............. ............. ............. ............. ............. ............. ............. ............. ............. ..........

............. ............. .......

��

� ��

�.................................................................................................................

..........................................................................................................................................................................................................................................................................

......................................................................................................�

��

��

��

Figure 3.8: The PDF and CDF of a Gaussian random variable.

85


.........

....

.........

....

.........

....

.........

....

.........

....

.........

...

.........

....

.........

....

.........

....

.........

....

.........

....

.........

...

��

��

�� area�� area

total area � �

...................................................................................................

..................

.............................................................................................................

........

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

.......................................................................................................

...............................................................................................................................................................................................................................

....................................................................................................................................................................................................................................................................................................................................................................................................

Figure 3.9: The probability areas of a standard Gaussian random variable.

A � � �� RV is called standard Gaussian (or standard normal). Its CDFis tabulated with notation

��

��

��!

� �

� "��

� �� (3.26)

The table of �� is sometimes given only for � � . For � � , by evensymmetry of � �� , we may use

�� (3.27)

How to use the table of �� for a nonstandard Gaussian RV � �� ? Theanswer is simple:

� ��

�

�!

� �

� "��

��

��

�

�!

� ��

� "��

� ��

�let � �

� � ��

then � � � � ��

�

� ��

�(3.28)

�� and in general the CDF � �� of a general Gaussian RV � �� aretabulated in the companion software P&R: You can get the value of � �� byspecify �, �� and � in P&R, as shown in Example 3.12.

A function closely related with �� is the so-called error function:

�� !

� �

�"��

�

��

86


Example 3.13: Common Gaussian Probabilities

Consider a RV � � � �� , where � and �� is a given real number.

(a) Find �� :

�� ?� ��

(3.28)� �

��

��

� �� table� ��

(b) Find �� :

��

� ��

��

��

��

(3.27)� ��

Similarly,

��

These probabilities are commonly used and are better learned by heart.

Example 3.14: Probability of Power Consumption

A voltage # � � � ��V�� is applied across a �� resistor. What is the proba-bility that the power � consumed by the resistor is in [�W, �W]?

�� # �� # � � ��

�� # �

��

�� # � �

��

� �� # � �� # � ��

� ��

�

��

��

�

�

��

��

�

��

��

��

�

��

89


Importance of Gaussian RV — The Central Limit Theorem

The central limit theorem states that, for � independent RVs �� withmean �� and variance �� , respectively,

� �

��

��

�� (3.29)

The central limit theorem has the following interpretation:

The properly normalized sum of many “uniformly” small and neg-ligible independent RVs tends to be a standard Gaussian RV. If arandom phenomenon is the cumulative effect of many “uniformly”small sources of uncertainty, it can be reasonably modeled as a Gaus-sian RV.

Fig. 3.11 illustrates how fast the sum of � independent uniform RVs tends tobe a Gaussian RV: �� has a rectangular PDF; �� has a triangular PDF;�� has a PDF with three parabolic pieces that is already quite closeto a Gaussian PDF.

�� 0.0 0.5 1.0 1.50.0

0.5

1.0

1.5

...............

..............

..............

..............

...............

..............

.................................................................................................................................................................................................................................................................... ............. ..........................

.......................................

.............

............. ..........................

..........................

.......................... ............. ............. �

(a) � �

0 1 20.0

0.5

1.0

................................................................................................................................................................................................................................................................................................................................................................... ............. .............

.............

....................................................

............. ..........................

..........................

.......................... ............. ............. ...... �

(b) � �

0.0 1.5 3.00.0

0.5

1.0

............. ............. .........................

.....................................................

............. ..........................

..........................

.......................... ............. ............. ...................................................................................

..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... �

(c) � �

0 2 40.0

0.5

1.0

............. ............. ..........................

.................................................... .............

..........................

..........................

............. ............. ............. .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. �

(d) � �

Figure 3.11: Convergence of sum of uniform RVs to Gaussian distribution.

90

3.7 Expectation and Moments


3.7.1 Introduction and Motivation

Recall that given mass $ distributed over �� with point masses %� � � � %�,respectively, the center of mass (i.e., centroid) is given by

� �

�� %��%�

�

�� %�

$if � �

��

��%�

If $ is distributed continuously over an interval �� with a mass density func-tion %��, then the center of mass (centroid) is given by

� �

� �� %�� %��

�

� �� %��

$if � ��

� �

��%��

Since probability of a RV � can be viewed as mass with point masses �� that sum up to unity or with a density function �� that integrates upto unity, its “centroid” may be defined as

� �

��

�� discrete� �

� �� continuous(3.40)

Example 3.18: Consider Example 3.2 of die rolling. For the RV � , definedby �� , its average value �� over all possible outcomes ��’s is

��

��

��

��

��

Define another RV & by the following assignments:

& �� & �� & �� & �an even number shows� � �

Its average value �' is given by

�' � � � �

��

��

��

��

��

�

�Consequently, the average of a discrete RV � should be defined by

��

��

�� (3.41)

which is a probabilistically weighted sum over all possible outcomes. Note thatit is identical to (3.40) and thus the average of a RV is the same as the centroidof its probability mass. This introduces the expected value.

100


All children are above average.

Humorist Garrison Keillor, Lake Wokegon

3.7.2 Definitions

The expected value of a RV � with PDF �� or PMF �� is definedas

�� (��

� � �

� � �� continuous��

�� discrete (3.42)

where��

stands for sum over all . Note the following aliases:

expected value � mean � average � expectation � first moment

The expected value of a function �� of a RV � is given by

(��

� � �� continuous��

�� discrete (3.43)

Note that��

�� is the weighted sum (or average) of ��.

Other quantities associated with expectation can then be defined, including:

� The %th moment of a RV � is defined as (��, given by (3.43) with�� .

� The %th central moment of � is defined as (�� , given by (3.43)with �� .

In the above, % is a positive integer.Second moments are of particular interest:

mean square value (�� second moment

variance �� var�� (�� second central moment

standard deviation ��

� ��

101


3.7.3 Interpretations

The expected value of a RV � can be interpreted naturally as

� the common-sense average value of � over all possible outcomes, where theaverage is weighted by the probability of the occurrence of the outcome

� the center of gravity (i.e., the centroid or the balance point).

The standard deviation is a measure of the dispersion of a RV from itsmean: A smaller indicates that the distribution is more concentrated aroundits mean. A RV � with � must have �� , meaning that � isactually not random. Thus, the standard deviation can be thought of as a measureof how random (uncertain) a RV is.

� has a small variance �� concentrates more around its mean

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

.

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

.

.............

.............

.............

.............

.............

.............

.............

.............

.............

.............

.

............. ............. ............. ............. ............. ............. ............. ............. ............. .............

............. ............. ............. ............. ............. ............. ............. ...

........................................................................ ...................................................... .................. ...................................................... ..........................................................................................

�

��

��

�

larger variance

��

.............................................................

.............................................................................................................................................................................................................

...........................................................................................................................................................................................................................................................................................................................................................................................................................

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

..................... ............. ............. ............. ............. ............. ............. .............

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

................. ............. ............. ............. ............. ............. ............. ...

...................................................... .................. ........................................................................��

�

��

��

�

smaller variance

......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

Figure 3.19: The PDF of a Gaussian RV with different variances.

��

�

larger variance��

��

��

�

smaller variance� � �

��

Figure 3.20: The PDF of a uniform RV with different variances.

103


3.7.4 Important Properties of Expectation and Variance

1. Expectation (average) of a constant is equal to the constant itself:

(��

� � ��

This makes sense: a constant will show up the same value in every trial.2. Expectation (average) of a constant times a RV is equal to the constant times

the expectation (average) of the RV:

(��)�� )��

� � )�� (�)�� (3.46)

This makes sense: average of scaled numbers = scaled average.3. Expectation of the sum of RVs is equal to the sum of expectations:

(��

)��

��

��

)��

��

� � )��

��

(�)��

(3.47)This makes sense: average of sets of numbers = sum of averages.

4.variance = mean-square value � (mean)� (3.48)

�� (�� (�� (�� (��

�(�� (��

5. Variance of a nonrandom constant times a RV plus a constant is equal to thefirst constant squared times the variance of the RV:

var�� (�� (�� (��

� ��var�� (3.49)

This makes sense: Adding a constant in every trial does not alter randomness.6. Variance of a constant is zero: var�� because � is not random.

(3.46)–(3.47) implies that expectation ( is a linear operator:

expectation of weighted sum � weighted sum of expectations(��)�� )�� (�)�� (�)��

(3.50)

However, (�)�� )�(��, if )�� is nonlinear. If * � )�� forsome � and �, then )�� is nonlinear, such as )�� and )�� "��.

105


3.7.5 Examples

Example 3.19: Mean and Variance of Uniform Distribution

Find the mean and variance of a uniform RV � � �� :

��(��

� �

��

�

��

�

��

� �(��

��

��

� �

��

�

��

�

��

��

��

��

�

�

��

��

��

�

��

��

�

�

��

��

��

��

�

��

Alternatively, the variance can be calculated using (3.48):

� �(��

��

�

��

��

��

��

��

��

�

��

��

��

�

��

�

��

��

��

��

��

��

��

��

��

In summary, for any �� RV,

mean ��

�� center point (3.51)

variance ��

��

�length��

��(3.52)

Since the PDF of a uniform RV is nonzero only over a finite interval, cautionshould be taken when using standard deviation (or variance). See Example 3.37and problems 3.18 and 3.19.

106


Example 3.20: Mean and Variance of Gaussian Distribution

Find the mean and variance of a Gaussian RV � � � �� :

(��

� ��

� � ��

? �

��

��!

"��

��

Since

��

"��

�� is even about � � �

�� is odd about � � �

� � � ��

�! "�

��

�� is odd about � � �

by odd symmetry, � ��

��!

"��

��

The mean is then given by

(�� peak point (3.53)

In general, a RV � with a symmetric PDF about a point � has (�� .Taking derivative on both sides with respect to of the identity

� �

��!

"��

��

��!

"��

��

yields�

�

� ��

��!

"��

��

where ��"

��

� "��

�� . Thus, the variance isgiven by

var�� (��

��!

"��

�� (3.54)

Consequently, the two parameters � and � of a Gaussian distribution arethe mean and variance, respectively.

107


Example 3.21: Mean and Variance of Exponential Distribution

A RV � has the exponential PDF

�� +"�� + � �� unit step function (3.55)

where the factor + is necessary to guarantee�� .

+

��

�

��

Figure 3.21: The PDF of an exponentially-distributed RV.

(a) Find �� and ��: By integration by parts, we have

��

�+"�� "��

��

"�� +

(��

+��"�� similar to above��

+��

(3.48)�

�

+�

Note that a small + implies a flat PDF and a large mean and variance.(b) Find the mean and variance of � �� (�� are constants):

�* � (�� (3.50)� �� + � ��

��(3.49)� ��

�� +

�

(c) Find the mean and variance of � � "�� (� is a constant):

�, �(�"�� "�� +

� �

"��"�� +

� � +

(�� (��"�� (�"��

� ?�

+

�� +(from the above equation)

�� (�� ,�� +

�� +� +�

�� +��

��+

�� +�� +��

Note the difference:

� �� a linear function� �� * � ��

� � "�� a nonlinear function� �� , � "��

108


Example 3.22: Mean and Variance of Binary Distribution

A binary RV or Bernoulli RV � is one with the PMF �� . Its mean and variance are

(��(3.42)�

��

-�� -� ��

��

-�� -� � �

(��(3.43)�

��

-�� -� ��

��

-�� -� � �

�� (�� (��

Example 3.23: Mean and Variance of Poisson Distribution

A Poisson RV � has the PMF given by (3.38).

(a) Find the mean of �:

(��(3.42)�

��

-�� -� ��

-"��+�

- � +"��

��

+��

�- � ��

� � � �� +"��

+�

� � +"��"� � +

note: "� �

��

+�

�

��

(b) Find the variance of �:

(��(3.43)�

��

-�� -� ��

-�"��+�

-

� +"��

��- � �� +��

�- � ��

� +"��

�- � ��+��

�- � ��

��

+��

�- � ��

�

� +"��+

��

+��

�- � �� "�

�� +"��+"� � "�� +� � +

�� (�� (�� +� � +� +� � +

Thus, both mean and variance of a Poisson RV are equal to the parameter +.

109

3.8 Function of a Random Variable


Given a RV � with the PDF �� and a deterministic function ��, � ��is another RV, what is the PDF � �*� of ?

A general procedure for obtaining � �*� from �� is to express first � �*�in terms of �� and then find � �*� � �

�� *�.

Example 3.24: PDF of a Linear Function of a RV

Find � �*� of � �� , where � is continuous and � �� :

� �*� � �� *� � �� *��

��

� � � ��

�� ?� ��

� � � � � ��

� �*� ��

�*� �*� �

��

��

��

�

��* � �

�

�(3.56)

An alternative and usually better method is to use the following theorem.Theorem: The PDF � �*� of � �� can be determined from the PDF

�� of � by:

� �*� ��

��

��

��

�� (3.57)

where �� and ��’s are the (distinct) real roots of the equation * �

�� in terms of *: * � �� .

Example 3.25: PDF of the Output of a Square-Law Device

The output of a square-law device is related to its input � by � ��. Find� �*� if � � � � ��. Note first �� . The two roots of* � �� are ��

*. Thus, � �*� � for * � . For * � , we have

� �*� ��

��

��

�*�

��*� ��

*�

��*��

��

��*

��!

"��

��!

"��

��

!�

��!*

"��

111


Example 3.28: PDF of a Trigonometric Function with a Random Phase

Given a random phase � � ��! !�, find � �*� of � � !"#� � � :

.........

....

.........

....

.........

....

.........

....

.........

....

.........

................. ............. ............. ............. ............. ............. ............. ............. ............. ............. ............. ............. ............. ............. ........

........ ........

*

� � ��

��

�

�

..........

...........

......................................................................................................................................

............................................................................

...........................................................................................................................................................

..........

...........

......................................................................................................................................

............................................................................

...........................................................................................................................................................

..........

...........

......................................................................................................................................

............................................................................

...........................................................................................................................................................

�� !"#� � * ��

��

�� !"#��*��

��

��

�� #� ��

��

�� !"#� ��

��

�� *�

�

� � ��! !� ��

��

�� ! � � � ! elsewhere

There are two roots �� and �� in ��! !� [�� outside ��! !�]. Thus,

� �*�(3.42)�

��

��

�� *��

��

�� *��

�� *��

��!

�� *��

��!

�� *��

��!

�� *�

(3.58)This is the PDF of the Cauchy distribution.

��

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

��

��

Uniform density

��

�

� � !"#��

��

�� *�

*... ...... ...

�

Cauchy density��

...............................................................................................

...................................................................................................................................................................................................................................................................................................................................................................................................................................

113

3.9 Conditional Distributions


Recall that the probability of event . conditioned on the occurrence of event /is defined by

��.�/� ��. �/��/�

Since �� is an event, let . � �� . Then the conditional CDF givenevent / is defined by

� ��/� � ��/� � �� /� �� /�

��/� (3.59)

The conditional PDF given event / is thus defined by

��/� �� /�

��

�� /�

��/� (3.60)

Delta functions may be introduced at discontinuous points of � ��/�.A conditional CDF possesses all the properties of an unconditional CDF,

including

� ��/� �

� ��/� � �

� � ��/� � �

�� /� � � ��/� � � ��/�

� ��/� � � ��/� �� nondecreasing�

�� /� � � ��/� �continuous from right�

All the properties of a PDF also apply to a conditional PDF, including

��/� � � � ��/��

��/� ��

� ��*�/��*

�� /� ��

��/��

114


Example 3.29: Truncated Gaussian

Let / � �� . The following so-called truncated PDF

��/� ��

��

��

�

��

� � �

� � �!��"��!��"�"��

!��"�"�� !��"�"��!��"�"��

�

��

��!��"�"��

elsewhere(3.61)

is the original PDF restricted to � � � � � and renormalized, where �� acts as a scaling factor to scale up the PDF so that the conditional PDF��/� integrates to unity. For example, if � � � �� , then

��

� � �

� ��

��

elsewhere

(3.62)

which is a truncated (censored) Gaussian.The conditional PDF �� of (3.62) and the unconditional (orig-

inal) PDF � �� are plotted in Fig. 3.22.

��

�

��

��

� �...................................................

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

.........

....

...

............. ............. ............. ............. .......................

................................................................................. ........................................................................................................ ............. ............. ............. ............. ..

.................................................................................................................................................................................................................................................................................................................

Figure 3.22: Gaussian PDF �� and truncated Gaussian PDF �� .

Most Gaussian models of real-world problems are actually truncated Gaus-sian, such as test score, and height and weight of people.

115

3.10 Generation of Random Numbers

3.10 Generation of Random Numbers

Uniform random number generators are available in many software packages.Given a uniform random number generator, there are several approaches to

generate other random numbers. The most popular one is the following. Ran-dom numbers of some popular distributions can be generated by the companionsoftware P&R.

3.10.1 Generation of Continuous Random Numbers

Samples of many continuous RVs can be generated by the following inverse-transform method based on the following theorem:

If 0 � �� , then � � ��0� has the CDF � �� provided � ��is continuous and strictly increasing in �, where �� is the inversefunction of � ��.

Thus, if 0� � � � 0� are �� random numbers, then �� 0��,� � � ��0�� are random numbers with the distribution � .

Example 3.30: Generation of Exponential Random Numbers

An exponential RV has the PDF given by (3.55). The corresponding CDF is

� ��

�� "��

� �

Its inverse function �� can be obtained by expressing � in terms of �:

� � �� "��

+�#��

+�#��

Since � �� is continuous and strictly increasing, by the above theorem, given�� random numbers 0� � � � 0�, the random numbers

��

+�#�� 0��

+�#�� 0��

�

are exponentially distributed with the CDF given above. In fact, ��0�� can bereplaced by 0� since both 0� and �� 0�� are �� random numbers.

116

the random variable - northeastern university

Documents