7/29/2019 lab_note1

1/12

KEEE494: 2nd Semester 2009 Lab I

Estimating PDFs, Means, Variances

1 Exp 1: Estimated PDF/CDF Plots From DataSuppose we have a vector x of observations from a continuous probability distribution with density function (PDF)f(x). This experiment teaches you how we can get an estimated plot of f(x). Lets suppose that x consists of nsamples, n large. (We will typically take n = 100000 in our experiments.) We thus subdivide the range of x valuesinto N bins of equal width. (In our experiments, we typically take N = n/100, so that if the data were uniformlydistributed, we would expect about 100 samples in each bin.) Let be the bin width. Let xi be the midpoint of thei-th bin and let ni be the number of samples in the i-th bin (which can be found with the Matlab function hist). Then

ni nf(xi)

This is because the probability that the distribution will yield a value in the i-th bin is the area under the densityfunction f(x) for this bin, which is about f(xi). Hence, we get the estimated value for f(xi) by dividing ni by n:

f(xi) ni/(n)

We can implement these ideas with the following Matlab code:

n=length(x); % n is the number of samples in x

N=floor(n/100); % N is the number of bins

A=min(x); B=max(x); % [A,B] is the range of x values

Delta=(B-A)/N; % Delta is the bin width

t=A-Delta/2+[1:N]*Delta; % horizontal axis of bin midpoints

f=hist(x,t)/(Delta*n); % vertical axis of density estimates

bar(t,f); % estimated density plot

Example 1. Obtain a plot on your screen of estimated uniform(0,1) density from 10000 rand pseudorandomlygenerated samples by running the following code:

n=100000;

x=rand(1,n);

N=floor(n/100);

A=min(x); B=max(x);

Delta=(B-A)/N;

t=A-Delta/2+[1:N]*Delta; f=hist(x,t)/(Delta*n);

bar(t,f);

title(Estimated Uniform (0,1) PDF)

Does your plot look like a jagged rectangular pulse of amplitude one? The jaggedness is unavoidable (it isdue to variance). One way to reduce the jaggedness in the density estimate is to use a smoothing filter. There is

an entire theory devoted to such smoothing filters, which we do not touch in this lab.

Example 2. Run the following in order to obtain an estimated CDF for uniform (0,1) data:

n=100000;

x=rand(1,n);

N=floor(n/100);

1

7/29/2019 lab_note1

2/12

A=min(x); B=max(x);

Delta=(B-A)/N;

t=A-Delta/2+[1:N]*Delta; p=hist(x,t)/n;

CDF=cumsum(p);

plot(t,CDF)

Does the plot look like the CDF of the uniform (0,1) density? Compare the code above to the code in Example1; see from this comparison if you understand how the code above does its job. The estimated CDF plot looks

smoother that the estimated density plot- Why?

Example 3. Obtain a plot on your screen of estimated standard gaussian density from 100000 randn pseudo-randomly generated samples by running the following code:

n=100000;

x=randn(1,n);

N=floor(n/100);

A=min(x); B=max(x);

Delta=(B-A)/N;

t=A-Delta/2+[1:N]*Delta;

f=hist(x,t)/(Delta*n);

bar(t,f)

title(Estimated Standard Gaussian PDF)

Compute 1/

2. Is this about equal to the peak value of the estimated density curve that you see on yourcomputer screen?

You can check by drawing the theoretical PDF on the same figure by adding three lines to the above m file as

below:

hold on

pdf=1/sqrt(2*pi)*exp(-t.2/2);

plot(t,pdf)

t=A-Delta/2+[1:N]*Delta;

f=hist(x,t)/(Delta*n);

bar(t,f)

title(Estimated Standard Gaussian PDF)

Compute 1/

2. Is this about equal to the peak value of the estimated density curve that you see on yourcomputer screen?

Example 4. Run the following to get an estimate of the plot of the CDF of a standard Gaussian distribution:

n=100000;

x=randn(1,n);

N=floor(n/100);

A=min(x); B=max(x);

Delta=(B-A)/N;

t=A-Delta/2+[1:N]*Delta;

p=hist(x,t)/n;

CDF=cumsum(p);

plot(t,CDF)

title(Estimated Gaussian(0,1) CDF)

2

7/29/2019 lab_note1

3/12

Use your standard Gaussian CDF table on page 142 to see if the actual CDF values at z = 0, 0.5, 1.0, 1.5, 2.0conform to the estimated CDF values you get from the curve on your screen.

Example 5. We know we can simulate an exponential distribution with density aeaxu(x) with parameter awith the transformation X = log(U)/a, where U is uniform in the interval [0, 1]. To test this, run the code:

n=100000;u=randn(1,n);

a=0.5;

x=-log(u)/a;

N=floor(n/100);

A=min(x); B=max(x);

Delta=(B-A)/N;

t=A-Delta/2+[1:N]*Delta;

f=hist(x,t)/(Delta*n);

bar(t,f)

title(Estimated Exponential(a) PDF (a=0.5))

Look at the peak value of your estimated density curve. Is this about what you expected it to be? Try another

value ofa if time permits.

Example 6. Run the following code to get the estimated CDF plot for an exponential distribution:

n=100000;

u=randn(1,n);

a=0.5;

x=-log(u)/a;

N=floor(n/100);

A=min(x); B=max(x);

Delta=(B-A)/N;

t=A-Delta/2+[1:N]*Delta;

p=hist(x,t)/n;CDF=cumsum(p);

plot(t,CDF)

axis([0 5 0 1])

title(Estimated Exponential(a) CDF (a=0.5))

Compute the actual values of the CDF 1 e(0.5)x at x = 1, 1.5, 2, 2.5, 3 and see how these compare to whatthe estimated CDF plot gives.

Example 7. Run the following code, which generates estimated PDF and CDF plots for a Rayleigh distribution:

x=randn(1,100000);

y=randn(1,100000);

r=sqrt(x.2+y.2);N=floor(n/100);

A=min(r); B=max(r);

Delta=(B-A)/N;

t=A-Delta/2+[1:N]*Delta;

f=hist(r,t)/(Delta*n);

subplot(2,1,1)

bar (t,f)

3

7/29/2019 lab_note1

4/12

title(Estimated Rayleigh PDF)

p=Delta*f;

CDF=cumsum(p);

subplot(2,1,2)

plot(t,CDF)

title(Estimated Rayleigh CDF)

Example 8. Run the following code, which generates estimated PDF and CDF for a chi-square distribution:

n=100000;

sigma=1;

x=sigma*randn(1,n);

y=x.2;

N=floor(n/100);

A=min(y); B=max(y);

Delta=(B-A)/N;

t=A-Delta/2+[1:N]*Delta;

f=hist(y,t)/(Delta*n);

subplot(2,1,1)bar(t,f)

title(Estimated chi-square PDF)

p=Delta*f;

CDF=cumsum(p);

subplot(2,1,2);

plot(t,CDF)

title(Estimated chi-square CDF)

Examine the effect of changing to = 0.5 in line two of the program. (The here is the standard deviationof the underlying Gaussian distribution, not the standard deviation of the chi-square distribution.)

2 Exp 2: Estimating Means/Variances from Data

Given a vector of data points x from a probability distribution (discrete or continuous), mean(x) is used to estimate

the mean of the distribution , var(x) is used to estimate the variance 2 of the distribution, and std(x) is used toestimate the standard deviation .

Example 9. What is the mean and variance 2 of the standard uniform distribution? (That is, the uniformdistribution in the interval [0, 1]?) Look these two values up in your textbook if you do not know the answer.Run the following lines of code to estimate , 2, from data.

x=rand(1,50000);

mean(x)var(x)

std(x)

Example 10. What is the mean and variance 2 of the standard Gaussian distribution? Look them up if youdont know them. Run the following lines of code to estimate , 2, from data.

4

7/29/2019 lab_note1

5/12

x=randn(1,50000);

mean(x)

var(x)

std(x)

Example 11. What is the mean and variance 2

of the exponential distribution with parameter a = 1? Lookup these if you dont know them. A data point simulating data from this distribution is -log(rand(1,1)).

Run the following lines of code to estimate , 2, from data.

x=-log(rand(1,50000));

mean(x)

var(x)

std(x)

3 Exp 3: Effect of Linear Transformations on Means/Variance

Let X be a RV with mean X and variance 2X . Let Y be a RV

Y = aX+ b,

where a, b are constants. We will show in class that the mean and variance of Y are related to the mean and varianceof X by the equations

Y = aX + b

2Y = a22X .

This experiment will provide numerical verification of this fact.

Example 12. Let X be uniformly distributed in [0, 1]. Let Y = 3X + 7. What will the mean and variance ofY be? Try to figure this out using paper and pencil. Then, do the following simulation to get the approximateanswers.

x=rand(1,50000);

y=-3*x+7;

mean(y)

var(y)

Example 13. Let X be standard Gaussian. Let Y = 3X + 7. What will the mean and variance ofY be? Tryto figure this out using paper and pencil. Then, do the following simulation to get the approximate answers.

x=randn(1,50000);

y=-3*x+7;

mean(y)

var(y)

Example 14. Let X be exponential with parameter a = 1. Let Y = 3X+ 7. What will the mean and varianceofY be? Try to figure this out using paper and pencil. Then, do the following simulation to get the approximateanswers.

x=-log(rand(1,50000));

y=-3*x+7;

mean(y)

var(y)

5

7/29/2019 lab_note1

6/12

Lab Homework 1

1. Let us denote the chi-square random variable Y, then Y =n

k=1 X2k , where Xk is the independent and

identically distributed Gaussian random variable with zero mean and variance 2. Generate 100,000chi-square samples with degree of 4 (n = 4) and set 2 such that E[Y] = 1.

Estimate the PDF, the mean and the variance. Plot the estimated PDF and the exact PDF curves on thesame figure. Calculate the exact mean and the variance of the chi-square with degree of 4 and compare

the exact values with the estimated values.

2. Let us denote the Ricean random variable Y, then Y is

Y = X21 + X22

where X1 and X2 are Gaussian random variable with means of m1 and m2 and variance of 2, respec-tively. Denote s2 = m21 + m

22. Generate 100,000 Ricean samples with m1 = m2 = 1/

2 and 2 = 1.

Estimate the PDF, the mean and the variance. Plot the estimated PDF and the exact PDF curves on the

same figure. Calculate the exact mean and the variance of the chi-square with degree of 4 and compare

the exact values with the estimated values.

In this lab we will learn some special random variables which are widely used not only in the communication area

but also in many other natural phenomena. Those are (1) Central-Chi square and (2) Non-Central Chi-Square. As

special cased for (1) and (2), we consider the Rayleigh and Ricean random variables, respectively.

Central Chi-Square Density Define the random variable Y as

Y =ni=1

X2i (1)

where the Xi, i = 1, 2, , n are statistically independent and identically distributed (i.i.d) Gaussianrandom variables with zero-mean and variance

2. Then, the characteristic function ofY

is given by

Y(j) E{ejY} = 1(1 j22)n/2 (2)

which is the consequence of the statistical independence of the Xis. The pdf ofY is obtained of theinverse Fourier transform, that is

fY(y) =1

n2n/2(n/2)yn/21ey/2

2

, y 0 (3)

where (q) is the gamma function defined as

(q) =

0

tq1et dt, q > 0

(q) = (q 1)!, q an integer and q > 0(

1

2) =

,

3

2

=

2

This pdf is called the Gamma or Chi-square pdf with n degrees of freedom and is illustrated in (3) forseveral values ofn.

2(n).

6

7/29/2019 lab_note1

7/12

0 2 4 6 8 10 12 14 160

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

f Y(y)

y

n=1

n=2

n=4

n=8

Figure 1: Pdf of a chi-square-distributed random variable for several degrees of freedom(2 = 1)

The first and second moments of Y can be computed to give

E{Y} = n2E{Y} = 2n4 + n24

2Y = 2n4 (4)

The cumulative distribution function (cdf) of Y is obtained by integrating (3), that is

FY(y)

Prob{Y y} = y

0

1

n2n/2(n/2) un/21

eu/22

du (5)

For the special case when n is even (let k=n/2), (5) can be computed in closed form to give

FY(y) = 1 ey/22

k1j=0

1

j!

y

22

j

Noncentral Chi-Square Density when the Gaussian random variables of (1) have nonzero means mi, i = 1, 2, , n but still identical

variances 2, the characteristic function of Y is no longer given by (2) but becomes

Y(j ) =1

(1 j22)n/2 expjn

i=1m2

i1 j22

(6)

As expected, (6) reduces to (2) when all the means are zero. The inverse Fourier transform of this charac-

teristic function yields

fY(y) =1

22

y

s2

(n2)/4exp

s

2 + y

22

In21

y

s

2

, y 0 (7)

7

7/29/2019 lab_note1

8/12

where s2 is the sum of means squared, that is

s2 =ni=1

m2i

and Ik

(x) is the kth-order modified Bessel function of the first kind which can be expressed by the infiniteseries

Ik(x) =j=0

(x/2)k+2j

j!(k +j + 1), x 0

The pdf, as given by (7), is typically referred to as the noncentral Chi-square density with n degreesof freedom. The parameter s2 is called the noncentrality parameter of the pdf. The cdf is obtained byintegration as

FY(y) =

y0

1

22

u

s2

(n2)/4exp

s

2 + u

22

In22

u

s

2

du

Here again, when n is even (k=n/2), the cdf can be expressed in terms of the generalized Q-functionQk(x, y) as

FY(y) = 1 Qk

s

,

y

where

Qk(a, b)

b

x

x

a

k1exp

x

2 + a2

2

Ik1(ax) dx

= Q1(a, b) + exp

x

2 + a2

2

k1j=1

b

a

jIj(ab)

The function Q1(a, b) is typically denoted by Q(a, b) [not to be confused with the Gaussian tail integralQ(x) which is a function of one variable] and is referred to as the Marcum Q-function and is given by

Q(a, b)

b

x exp

x

2 + a2

2

I0(ax) dx

The first and second moments of Y can be computed in closed form to give

E{Y} = n2 + s2E{Y2} = 2n4 + 42s2 + (n2 + s2)2

2y = 2n4 + 42s2

which reduce to (4) when the means are zero(that is, s=0)

8

7/29/2019 lab_note1

9/12

Rayleigh Density Suppose we define a new random variable R which is

R

Y =

n

i=1

X2i (8)

where the Xi, i = 1, 2, , n are as defined earlier and the means are zero. Then, the pdf of R can beeasily obtained from the Chi-square pdf with a simple transformation to give

fR(r) =rn1

2(n2)/2n(n/2)exp

r

2

22

, r 0 (9)

Here again, when n is even (let k = n/2), the cdf ofR can be obtained in closed form as

FR(r) = 1 exp r

2

22

k1j=0

1

j!

r2

22

j, r 0

and the mth moment ofR is given by

E{Rm} = (22)m/2 ((n + m)/2)(n/2)

, m 0

for any integer m. As a special case of (9) when n = 2, we obtain the familiar Rayleigh pdf

fR(r) =r

2exp

r

2

22

, r 0 (10)

with corresponding cdf

FR(r) = 1 expr2

22

, r 0 (11)

9

7/29/2019 lab_note1

10/12

Rician Density When R is still defined by (8) and the Xis are nonzero means (each Xi has mean mi) each with variance

2, then the pdf ofR becomes

fR(r) =rn/2

2s(n

2)/2

expr2 + s2

22 In21

rs

2, r

0 (12)

with corresponding cdf for n even (k = n/2)

FR(r) = 1 Qk

s

,

r

where s2 is defined in (8). The mth moment ofR is

E{Rm} = (22)m/2 exp s

2

22

((n + m)/2)

(n/2)1F1

n+m2 ,

n2 ;

s2

22

where 1F1(,, ; x) is the confluent hypergeometric function given by

1F1(,, ; x) j=0

( +j)()xj

()(+j)j!, = 0,1,2,

when n = 2, (12) reduces to the familiar Rician pdf

fR(r) =r

2exp

s

2 + r2

22

I0

rs

2

, r 0 (13)

where s =

m21 + m22 and corresponding cdf

FR(r) = 1 Q

s

,

r

(14)

when s = 0, the Rician pdf and cdf of (13) and (14) reduce to the Rayleigh pdf and cdf of (10) and (11),respectively.

10

7/29/2019 lab_note1

11/12

On The Joint Density of Amplitude and Phase So far, we have been concentrating on the marginal densities of Y or R, basically ignoring any phase

information. Lets define

R =

X21 + X22 and tan =

X2X1

where X1 and X2 are i.i.d. Gaussian random variables with variance 2 and m1 = m but m2 = 0.

We already know that R is Rician distributed if m = 0 but Rayleigh distributed if m = 0. It wouldbe interesting to derive the joint pdf fR,(r, ) for both cases. Since R and are two functions of tworandom variables, it is easily shown that

fR,(r, ) = rfX1,X2(r cos , r sin ), r 0, =

r

22exp

1

22

(r cos m)2 + (r sin )2

or equivalently

)fR,(r, ) =r

22exp

1

22

r2 2mr cos + m2

, r 0, (15)

To obtain the marginal densities, we integrate the joint density

fR(r) =

fR,(r, ) d =r

22exp

r

2 + m2

22

exp

2mr cos

22

d

=r

2exp

r

2 + m2

22

I0

mr

2

, r 0

which is the Rician pdf as expected with s = |m|. On the other hand, the marginal pdf of becomes

f() =

0

fR,(r, ) dr

=1

22exp

m2

22

0

r expr2 2mr cos

22 dr (16)

completing the square in the integral and then substituting x = r m cos , (??) reduces to

f() =1

22exp

m

2 sin2

22

m cos

(x + m cos )exp

x

2

22

dx

or

f() =m cos

2exp

m

2 sin2

22

1 Q

m cos

+

exp

m2

22

2(17)

note that when m = 0, the joint pdf in (??) becomes

fR,(r, ) = r22

exp r2

22

= fR(r)f()

where fR(r) = (r/2) exp(r2/22), r 0 and f() = 1/2, . Hence, R and

are independent random variables with R Rayleigh distributed and uniformly distributed. On the otherhand, when m = 0, R and are dependent with joint pdf as given by (??). The marginal pdf of R isRician and F() is given by (??) and is plotted in (??) as a function of|m|/. As one would expect, thepdf of becomes quite peaked as |m|/ increases.

11

7/29/2019 lab_note1

12/12

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.00

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

()

f

()

|m|/=4

2

1

0

Figure 2: Pdf for phase angle, Rician channel

12