chapter 13b: random signals and noisehomepages.wmich.edu/~bazuinb/ece3800/b_notes13b.pdf · 2020....

Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and

Random Signals, Oxford University Press, February 2016.

B.J. Bazuin, Fall 2020 1 of 58 ECE 3800

Charles Boncelet, “Probability, Statistics, and Random Signals," Oxford University Press, 2016. ISBN: 978-0-19-020051-0

Chapter 13b: RANDOM SIGNALS AND NOISE

Sections 13.1 Introduction to Random Signals 13.2 A Simple Random Process 13.3 Fourier Transforms 13.4 WSS Random Processes 13.5 WSS Signals and Linear Filters 13.6 Noise

13.6.1 Probabilistic Properties of Noise 13.6.2 Spectral Properties of Noise

13.7 Example: Amplitude Modulation 13.8 Example: Discrete Time Wiener Filter 13.9 The Sampling Theorem for WSS Random Processes

13.9.1 Discussion 13.9.2 Example: Figure 13.4 13.9.3 Proof of the Random Sampling Theorem

Summary Problems




13.3 Fourier Transform

Definition:PSD

Let Rxx(t) be an autocorrelation function for a WSS random process. The power spectral density is defined as the Fourier transform of the autocorrealtion function.

diwRRwS XXXXXX exp

The inverse exists in the form of the inverse transform

dwiwtwStR XXXX exp2

1

Properties:

1. Sxx(w) is purely real as Rxx(t) is conjugate symmetric

2. If X(t) is a real-valued WSS process, then Sxx(w) is an even function, as Rxx(t) is real and even.

3. Sxx(w)>= 0 for all w.

Wiener–Khinchin Theorem For WSS random processes, the autocorrelation function is time based and has a spectral decomposition given by the power spectral density.

Also see:

http://en.wikipedia.org/wiki/Wiener%E2%80%93Khinchin_theorem

Why this is very important … the Fourier Transform of a “single instantiation” of a random process may be meaningless or even impossible to generate. But if the random process can be described in terms of the autocorrelation function (all ergodic, WSS processes), then the power spectral density can be defined.

I can then know what the expected frequency spectrum output looks like and I can design a system to keep the required frequencies and filters out the unneeded frequencies (e.g. noise and interference).




Relation of Spectral Density to the Autocorrelation Function

For “the right” random processes, the power spectral density is the Fourier Transform of the autocorrelation:

diwtXtXERwS XXXX exp

For a real ergodic process, we can use time-based processing to arrive at an equivalent result …

txtxdttxtx

T

T

TT

XX 2

1lim

T

TT

XX dttxtxT

tXtXE 2

1lim

diwdttxtx

TtXtXE

T

TT

XX exp2

1lim

dtdiwtxtxT

T

TT

XX

exp2

1lim

dtdiwttiwtxtxT

T

TT

XX

exp2

1lim

dtdtiwtxiwttxT

T

TT

XX

expexp2

1lim

dtdtiwtxiwttxT

T

TT

XX

expexp2

1lim

If there exists wXX

dtwXiwttxT

T

TT

XX

exp

2

1lim

dttwitxT

wXT

TT

XX

exp

2

1lim

2wXwXwXXX




Properties of the Fourier Transform:

diwxxwX exp

https://en.wikipedia.org/wiki/Fourier_transform

For x(t) purely real

dwiwxxwX sincos

dwxidwxxwX sincos

dwxidwxwOiwEwX XX sincos

dwxwEX cos and

dwxwOX sin

Notice that:

wEdwxdwxwE XX

coscos

wOdwxdwxwO XX

sinsin

Therefore, the real part is symmetric and the imaginary part is anti-symmetric!

Note also, for real signals *wXwXconjwX

X(w) is conjugate symmetric about the zero axis.




Relatingthistoarealautocorrelationfunctionwhere XXXX RR

wOiwER XXXX

dwiwRR XXXX sincos

dtwtiwttRR XXXX sincos

dtwttRidtwttRR XXXXXX sincos

wOiwER XXXX

Since Rxx is symmetric, we must have that

XXXX RR and wOiwEwOiwE XXXX

For this to be true, wOiwOi XX , which can only occur if the odd portion of the

Fourier transform is zero! 0wOX .

This provides information about the power spectral density,

wERwS XXXXX

wEwS XXX

0 wS XX

The power spectral density necessarily contains no phase information!




Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for

Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.

Example9.5‐3S&WandBonceletp.343

Find the psd of the following autocorrelation function … of the random telegraph.

0,exp forRXX

Find a good Fourier Transform Table … otherwise

dwjRwS XXXX exp

dwjwS XX expexp

0

0

expexpexpexp dwjdwjwS XX

0

0

expexp dwjdwjwS XX




0

0

expexp

wj

wj

wj

wjwS XX

wj

wj

wj

wj

wj

wj

wj

wjwS XX

exp0exp

0expexp

wjwj

wjwj

wjwjwS XX

11

2222

22

ww

wS XX

For a=3

Figure 9.5-2 Plot of psd for exponential autocorrelation function.

Laplace vs. Fourier

𝑤 𝑗 ∙ 𝑠

𝑆 𝑠2 ∙ 𝑎𝑠 𝑎

2 ∙ 𝑎𝑠 𝑎 ∙ 𝑠 𝑎




Example9.5‐4

Find the psd of the triangle autocorrelation function … autocorrelation of rect.

TtriRXX

or TT

RXX

,1

T

T

XX dwjT

wS

exp1

T

T

XX dwjT

dwjT

wS0

0

exp1exp1

TT

TT

XX

dwjT

dwj

dwjT

dwjwS

00

00

expexp

expexp

T

T

T

T

XX

wj

wj

wj

wj

T

wj

wj

wj

wj

T

wj

wj

wj

wjwS

0

2

0

2

0

0

expexp1

expexp1

expexp

22

22

1expexp1

expexp11

1expexp1

ww

Twj

wj

TwjT

T

w

Twj

wj

TwjT

wT

wjwj

Twj

wj

Twj

wjwS XX




222

expexp121

expexp1

expexp

w

Twj

w

Twj

TwT

wj

TwjT

wj

TwjT

T

wj

Twj

wj

TwjwS XX

22

cos212sin2sin2

w

Tw

TwTw

Tw

w

TwwS XX

TwwT

wS XX cos112

2

2

2

2

2

2

2sin

2sin2

12

Tw

Tw

TTw

jwT

wS XX

Don’t you love the math ?!

Using a table is much faster and easier ….




Properties of the Fourier Transform

ThediscreteFourierTransformalsoexists(ifx(n)isbounded).Dr.Bazuinmayalsocallthisthediscrete‐time,continuous‐frequencyFourierTransform(somethingfromECE4550)

n

nwjnxnxwX exp , for w

and X(w) periodic in 2 pi outside the range (it repeats in the frequency domain)

The inverse transform is defined as

dwnwjwXwXnx exp

2

11




Deriving the Mean-Square Values from the Power Spectral Density

Using the Fourier transform relation between the Autocorrelation and PSD

diwRwS XXXX exp

dwiwtwStR XXXX exp2

1

The mean squared value of a random process is equal to the 0th lag of the autocorrelation

dwwSdwiwwSRXE XXXXXX 2

10exp

2

102

dffSdwfifSRXE XXXXXX 02exp02

Therefore, to find the second moment, integrate the PSD over all frequencies.

As a note, since the PSD is real and symmetric, the integral can be performed as

0

2

2

120 dwwSRXE XXXX

0

2 20 dffSRXE XXXX




Converting between Autocorrelation and Power Spectral Density

Using the properties of the functions we can actually define variations of Transforms!

The power spectral density as a function is always real, positive, and an even function in w/f.

You can convert between the domains using any of the following …

The Fourier Transform in w

diwRwS XXXX exp

dwiwtwStR XXXX exp2

1

The Fourier Transform in f

dfiRfS XXXX 2exp

dfftifStR XXXX 2exp

The 2-sided Laplace Transform (the jw axis of the s-plane)

dsRsS XXXX exp

j

j

XXXX dsstsSj

tR exp2

1




NotesonusingtheLaplaceTransform

ECE 3100 and ECE 3710 stuff …

(1) When converting from the s-domain to the frequency domain use:

jws or jsw

(2) As an even function, the PSD may be expected to have a polynomial form as: (Hint: no odd powers of w in the numerator or denominator!)

0

22

4242

2222

20

22

4242

2222

2

0bwbwbwbw

awawawawSwS

mm

mm

m

nn

nn

n

XX

This can be factored and expressed as:

sTsT

sdsd

scscwS XX

To compute the autocorrelation function for 0 use a partial fraction expansion such that

sd

sg

sd

sgwS XX

and solve for 0 using the LHP poles and zeros as

0,exp

2

1

tfordsstsd

sgtR

j

j

XX

for determining 0 , use the RHP expansion, replace –s with s, perform the Laplace transform and replace t with –t.

Another hint, once you have 0 , make the image and skip the math tRtR XXXX .

Final Note … for sTsT

sdsd

scscwS XX

If you “define” T(s) as all the LHP poles and zeros and T(-s) as all the RHP poles and zeros, then T(s) will represent a (1) causal, (2) minimum phase, and (3) stable filter (if there are no poles on the jw axis).

You give me a power spectral density and I can design a filter that passes “signal energy” and filters out as much of the rest as possible!




Example:InverseLaplaceTransform.

22

2

22

2

2

1

2

w

A

w

A

wS

X

XXX

Substitute s for w

ss

A

s

AsS XX

2

22

2 22

Partial fraction expansion

ss

A

ss

sksk

s

k

s

ksS XX

21010 2

2

02

10

1010

222

0

AkAkk

kkskk

s

A

s

AsS XX

22

Taking the LHP Laplace Transform

𝐿𝐴𝛽 𝑠

𝐴 ∙ 𝑒𝑥𝑝 𝛽 ∙ 𝑡 , 𝑓𝑜𝑟 𝑡 0

Taking the RHP with –s and then –t.

0expexpexp 2222

tfortAtAtA

s

AL

Combining the positive and negative time “halves” we have

tARXX exp2




7-6.3 A stationary random process has a spectral density of.

else

wwS XX

,0

2010,5

(a) Find the mean-square value of the process.

02

2

2

10 dwwSdwwSR XXXXXX

20

10

10

20

20

10

52

125

2

15

2

10 dwdwdwRXX

50

10202

10

2

100

20

10

wRXX

(b) Find the auto-correlation function the process.

dwtwjwStR XXXX exp2

1

10

20

20

10

expexp2

5dwtwjdwtwjtRXX

10

20

20

10

expexp

2

5

tj

twj

tj

twjtRXX

tj

tj

tj

tj

tj

tj

tj

tjtRXX

20exp10exp10exp20exp

2

5

tj

tj

tj

tj

tj

tj

tj

tjtRXX

10exp10exp20exp20exp

2

5




ttttj

tj

tj

tjtRXX

10sin20sin510sin220sin2

2

5

ttt

ttt

tRXX

15cos5sin

10

2

1020cos

2

1020sin2

5

ttt

t

ttRXX

15cos5

sinc50

15cos5

5sin50

(c) Find the value of the auto-correlation function at t=0..

015cos05

sinc50

015cos05

05sin500

XXR

1150

1150

0 XXR

50

0 XXR

It must produce the same result!




White Noise

Noise is inherently defined as a random process. You may be familiar with “thermal” noise, based on the energy of an atom and the mean-free path that it can travel.

As a random process, whenever “white noise” is measured, the values are uncorrelated with each other, not matter how close together the samples are taken in time.

Further, we envision “white noise” as containing all spectral content, with no explicit peaks or valleys in the power spectral density.

As a result, we define “White Noise” as tSRXX 0

𝑆 𝑓 𝑆𝑁2

This is an approximation or simplification because the area of the power spectral density is infinite!

Nominally, noise is defined within a bandwidth to describe the power. For example,

Thermal noise at the input of a receiver is defined in terms of kT, Boltzmann’s constant times absolute temperature, in terms of Watts/Hz. Thus there is kT Watts of noise power in every Hz of bandwidth. For communications at “room temperature”, this is equivalent to –174 dBm/Hz or –204 dBW/Hz.

For typical applications, we are interested in Band-Limited White Noise where

𝑆 𝑓 𝑆𝑁2

, |𝑓| 𝐵

0, 𝐵 |𝑓|

The equivalent noise power is then:

𝐸 𝑋 𝑅 0 𝑆 ∙ 𝑑𝑓 2 ∙ 𝐵 ∙ 𝑆 2 ∙ 𝐵 ∙𝑁2

𝑁 ∙ 𝐵

For communications, we use kTB where W=B and N0=kT.

How much noise power, in dBm, would I say that there is in a 1 MHz bandwidth?

dBmBdBkTdBkTBdB 11460174

See: https://en.wikipedia.org/wiki/Johnson%E2%80%93Nyquist_noise




Band Limited White Noise

𝑆 𝑓 𝑆𝑁2

, |𝑓| 𝑊

0, 𝐵 |𝑓|

The equivalent noise power is then:

𝐸 𝑋 𝑅 0 𝑆 ∙ 𝑑𝑓 2 ∙ 𝑊 ∙ 𝑆 2 ∙ 𝑊 ∙𝑁2

𝑁 ∙ 𝑊

Butwhatabouttheautocorrelation?

W

W

XX dfftiStR 2exp0

ti

Wti

ti

WtiS

ti

ftiStR

W

WXX

2

2exp

2

2exp

2

2exp00

ti

WtiiStRXX

2

2sin20

For xt

xtxt

sinc

WtSWtRXX 2sinc2 0

Using the concept of correlation, for what values will the autocorrelation be zero? (At these delays in time, sampled data would be uncorrelated with previous samples!)

,2,12

2

kforW

kt

kWt

Sampling at 1/2W seems to be a good idea, but isn’t that the Nyquist rate!!

Also note, noise passed through a filter becomes band-limited, and the narrower the filter the smaller the noise power … but the wider is the sinc autocorrelation function.




Noise and Filtered Noise Matlab Simulation

Based on Cooper and McGillem HW Problem 6-4.6.

x=randn(N,1); % zero mean, unit power random signal [b,a] = butter(4,20/500); y=filter(b,a,x); % applying a digital filter y=y/std(y); % normalizing the output power Rxx=xcorr(x)/(N+1); Ryy=xcorr(y)/(N+1); DFTx = fftshift(fft(x))/N; DFTy = fftshift(fft(y))/N; DFTRxx = fftshift(fft(Rxx,2*N))/N; DFTRyy = fftshift(fft(Ryy,2*N))/N;




Pink_Noise … If we call constant at all frequencies white noise, then noise in a limited low frequency band is sometimes called pink noise.

OK. I’m getting ahead …. we just did this.




Section 9.3 S&W or Boncelet Section 13.5 Continuous-Time Linear Systems with Random Inputs

Linear system requirements:

Definition 9.3-1 Let x1(t) and x2(t) be two deterministic time functions and let a1 and a2 be two scalar constants. Let the linear system be described by the operator equation

txLty

then the system is linear if “linear super-position holds”

txLatxLatxatxaL 22112211

for all admissible functions x1 and x2 and all scalars a1 and a2.

For x(t), a random process, y(t) will also be a random process.

Linear transformation of signals: convolution in the time domain

txthty

th ty

System

tx

Linear transformation of signals: multiplication in the Laplace domain

sXsHsY

sX sH sY

The convolution Integrals (applying a causal filter)

0

dhtxty

or

t

dxthty

Where for physical realize-ability, causality, and stability constraints we require

00 tforth and

dtth




Example: Applying a linear filter to a random process

03exp5 tfortth

tMtX 2cos4

where M and are independent random variables, uniformly distributed [0,2].

What can the linear filter do to the input signal?

(1) If a DC term passes through a linear filter, what can happen? The magnitude can change.

(2) If a cos or sin signal passes through a linear filter, what can happen? The magnitude and phase can change!

What do we expect at the output … a signal with a DC offset plus a cos where the magnitude and phase has changed! Now determine the value of the change!!

We can perform the filter function since an explicit formula for the random process is known.

t

dxthty

t

dMtty 2cos43exp5

tt

dtdtMty 2cos3exp203exp5

t

t

diiiit

tMty

2exp2exp3exp10

3

3exp5

t

i

iit

i

iitMty

23

2exp3exp

23

2exp3exp10

3

5

23

2exp

23

2exp10

3

5

i

iti

i

itiMty

49

2exp232exp2310

3

5 itiiitiiMty

ttM

ty 2sin22cos313

20

3

5




Linear filtering will change the magnitude and phase of a sinusoidal signals (DC too!).

tMtX 2cos4

Using trig identities on the previous solution the result is …

69.33,2cos413

5

3

5 tMty

Expectedvalueoperatorwithlinearsystems

For a causal linear system we would have

0

dhtxty

and taking the expected value of the output

0

dhtxEtyE

0

dhtxEtyE

0

dhttyE

For x(t) WSS

00

dhdhtyE

Notice the condition for a physically realizable system!

The coherent gain of a filter is defined as:

00

Hdtthhgain

Therefore, 0HXEhXEtYE gain

The coherent gain of the filter is the sum/integral of the filter impulse respose!




The Fourier Transform of the filter can help us – magnitude and phase at frequencies!

Note that:

dttfithfH 2exp

For a causal filter

0

2exp dttfithfH

At f=0

0

0 dtthH

And 0HtyE

Filters change magnitude and phase for deterministic signals and random signals the same amount at each defined frequency.




Whataboutacross‐correlation?(Convertinganauto‐correlationtocross‐correlation)

For a linear system we would have

dhtxty

And performing a cross-correlation (assuming real R.V. and processing)

dhtxtxEtytxE 2121

dhtxtxEtytxE 2121

dhtxtxEtytxE 2121

dhttRtytxE XX 2121 ,

For x(t) WSS

dhRRtytxE XXXY

hRRtytxE XXXY

What about the other way … YX instead of XY

And performing a cross-correlation (assuming real R.V. and processing)

2121 txdhtxEtxtyE

dhtxtxEtxtyE 2121

dhtxtxEtxtyE 2121

dhttRtxtyE XX 2121 ,

For x(t) WSS … see the next page




For x(t) WSS

dhttRRtxtyE XXYX

dhRRtxtyE XXYX

Perform a change of variable for lamba to “-kappa” (assuming h(t) is real, see text for complex)

dhRRtxtyE XXYX

Therefore

dhRRtxtyE XXYX

hRRtxtyE XXYX

Whatabouttheauto‐correlationofy(t)?

And performing an auto-correlation (assuming real R.V. and processing)

222211112121 , dhtxdhtxEttRtytyE YY

112222112121 , dhdhtxtxEttRtytyE YY

112222112121 , dhdhtxtxEttRtytyE YY

112222112121 ,, dhdhttRttRtytyE XXYY

For x(t) WSS

122112 ddhhRRtytyE XXYY

112221 dhdhRRtytyE XXYY




The output autocorrelation can also be defined in terms of the cross-correlation as

111 dhRRtytyE XYYY

hRRtytyE XYYY

The cross-correlation can be used to determine the output auto-correlation!

Continue in this concept, the cross correlation is also a convolution. Therefore,

hhRRtytyE XXYY

If h(t) is complex, the term in h(-t) must be a conjugate.

Looking forward … convolution in the time domain is multiplication in the frequency domain …

TheMeanSquareValueataSystemOutput

Based on the output autocorrelation formula

1221122 0 ddhhRRtyE XXYY

211122

2 0 ddhRhRtyE XXYY

dhRhRtyE XXYY 02

Based on the input to output cross-correlation formula

dhRRtytyE XYYY




Example:WhiteNoiseInputstoacausalfilter

Let tNtRXX

20

0

122

0

2112 0 ddhRhRtYE XXYY

0

122

0

210

12

20 ddh

NhRtYE YY

0

11102

20 dhh

NRtYE YY

0

12

102

20 dh

NRtYE YY

For a white noise process, the mean squared (or 2nd moment) is proportional to the filter power.




Example:RCfilter

The RC low-pass filter

CRs

CRCRs

CsR

CssH

1

1

1

11

1

Inverse Laplace Transform

tuCR

t

CRth

exp1

Coherent Gain of the RC Filter

00

Hdtthhgain

0

exp1

dtCR

t

CRhgain

CR

CR

t

CRCR

CR

t

CRhgain

1

exp1

1

exp1 0

10

expexp1

CRCR

hgain

If driven by a white noise process, what is the output power?

0

202

2 dh

NtYE

0

2

02 exp1

2

dCRCR

NtYE




0

2

02 2exp

1

2

dCRCR

NtYE

CR

CR

CR

NtYE

2

2exp

1

20

2

02

CR

NCR

NtYE

4

11

2

1

2 002

ComparingNoisePowerinthefilterbandwidth(equivalentnoisebandwidth)

Power in “rectangular” band-limited noise

𝐸 𝑁𝑁2∙

12𝜋

∙ 1 ∙ 𝑑𝑤𝑁2∙ 1 ∙ 𝑑𝑓

𝐸 𝑁𝑁2∙

12𝜋

∙ 2 ∙ 𝑊𝑁2∙ 2 ∙ 𝐵

𝐸 𝑁 𝑁 ∙𝑊2𝜋

𝑁 ∙ 𝐵

where WEQ is in rad/sec and BEQ in Hz

The noise power in an RC filter 𝐸 𝑌 𝑁 ∙∙ ∙

For an equivalent band-limited noise process to have the same power (assume a brick wall filter)

𝐸 𝑁 𝑁 ∙𝑊2𝜋

𝑁 ∙1

4 ∙ 𝑅 ∙ 𝐶 𝐸 𝑌

Solving for the equivalent noise bandwidth

𝑊2𝜋

14 ∙ 𝑅 ∙ 𝐶

Therefore in rad/sec 𝑊∙ ∙

of in Hz 𝐵∙ ∙




Note that the nominal -3dB band (½ power) of an RC network is

RCW dB

13 or

RCB dB

2

13

Comparing these two, the equivalent noise bandwidth is greater than the –3dB bandwidth by

𝑊 ∙𝑊 or 𝐵 ∙ 𝐵

Note: B in Hz and W in rad/sec.

The equivalent noise bandwidth must take into account all the noise that would pass through the filter, not just the -3dB cut-off frequency value. Therefore, for low pass filters, we expect the equivalent noise bandwidth to be greater than the desired filter passband bandwidth.

0 0.5 1 1.5 2 2.5 3 3.5 4-0.2

0

0.2

0.4

0.6

0.8

1




The power spectral density output of linear systems

The first cross-spectral density

hRR XXXY

diwRwS XYXY exp

diwhRwS XXXY exp

Using convolution identities of the Fourier Transform (if you want the proof it isn’t bad, just tedious)

wHwSwS XXXY

The second cross-spectral density

hRR XXYX

diwRwS YXYX exp

diwhRwS XXYX exp*

Using convolution identities of the Fourier Transform (if you want the proof it isn’t bad, just tedious)

*wHwSwS XXYX

The output power spectral density becomes

hhRR XXYY

diwRwS YYYY exp

diwhhRwS XXYY exp

Using convolution identities of the Fourier Transform

*wHwHwSwS XXYY

2wHwSwS XXYY

This is a very significant result that provides a similar advantage for the power spectral density computation as the Fourier transform does for the convolution.

This leads to the following table.




Example 13.1

X(n) is a discrete WSS random sequence with a defined PSD.

𝑌 𝑛 𝑋 𝑛 𝛼 ∙ 𝑋 𝑛 1

This defines a causal digital filter.

ℎ 𝑛 𝛿 𝑛 𝛼 ∙ 𝛿 𝑛 1

Determine the discrete-time Fourier transform

𝐻 𝑤 ℎ 𝑛 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤 ∙ 𝑛

𝐻 𝑤 𝛿 𝑛 𝛼 ∙ 𝛿 𝑛 1 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤 ∙ 𝑛

𝐻 𝑤 1 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤 ∙ 0 𝛼 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤 ∙ 1

𝐻 𝑤 1 𝛼 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤

Finding the PSD of Y

𝑆 𝑤 𝑆 𝑤 ∙ |𝐻 𝑤 |

for

|𝐻 𝑤 | 𝐻 𝑤 ∙ 𝑐𝑜𝑛𝑗 𝐻 𝑤

|𝐻 𝑤 | 1 𝛼 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤 ∙ 𝑐𝑜𝑛𝑗 1 𝛼 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤

|𝐻 𝑤 | 1 𝛼 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤 ∙ 1 𝛼 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤

|𝐻 𝑤 | 1 𝛼 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤 𝛼 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤 𝑎

|𝐻 𝑤 | 1 2 ∙ 𝛼 ∙ 𝑐𝑜𝑠 𝑤 𝑎

𝑆 𝑤 𝑆 𝑤 ∙ 1 2 ∙ 𝛼 ∙ 𝑐𝑜𝑠 𝑤 𝑎




Signal plus noise processing

System analysis with a noise input …

tx

tn

th ty tr

Where the signal of interest is x(t), n(t) is a noise or interfering process. The signal plus noise is r(t) and the received system output is y(t) which has been filtered.

We have tntxtr

Assuming WSS with x and n independent and n zero mean

tntxtntxEtrtrERRR

tntntxtntntxtxtxERRR

NNXXRR RtxtnEtntxERR

NNNXXXRR RRR 2

For 0 mean noise …

NNXXRR RRR

And then

* hhRRR NNXXYY

hhRhhRR NNXXYY

Design considerations

the filter should not “modify” the signal of interest (unity gain, no phase)

the filter should remove as much noise as possible.




Signal‐to‐Noise‐(Power)‐RatioSNR(alwaysdoneforpowers)

The signal-to-noise ratio is the power ratio of the signal power to the noise power.

The input SNR is defined as

0

02

2

NN

XX

Noise

Signal

R

R

tNE

tXE

P

P

The output SNR is defined as

0

02

2

hhR

hhR

thtNE

thtXE

P

P

NN

XX

Noise

Signal

For a white noise process and assuming the “filter” does not change the input signal (unity gain), but strictly reduces the noise power by the equivalent noise bandwidth of the filter.

We have

dhN

dwwHwSR XXYY202

22

10

With appropriate filtering with unity gain where the signal exists and bandwidth reduction for the noise

𝑅 01

2𝜋∙ 𝑆 𝑤 ∙ |𝐻 𝑤 | 𝑁 ∙ 𝐵

or 𝑅 0 𝑅 0 𝑁 ∙ 𝐵

The output SNR is defined as

EQ

XX

Noise

Signal

BN

R

P

P

0

0

The narrower the filter applied prior to signal processing, the greater the SNR of the signal! Therefore, always apply an analog filter prior to processing the signal of interest!

From the definition of band-limited noise power, the equation for the equivalent noise bandwidth (performed as a previous example)

12

10

12

12

20 dh

NdhRthtNE NN

dffHdtthBEQ

22

2

1

2

1

Under the unity gain condition

dtth1




Otherwise, the equivalent noise bandwidth can be defined as

dffHfH

BEQ

2

2max

12

For a real, low pass filter this simplifies to

dffHH

BEQ

2

20

12

Using Parseval’s Theorem this can be also defined as

dffHdwwHdtth222

2

1

2

2

2

2

02

dtth

dtth

H

dtth

BEQ




Example Section 13.7 Amplitude Modulation.

𝑋 𝑡 𝐴 ∙ 1 𝛽 ∙ 𝑚 𝑡 ∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜃 Where m(t) is a random message, typically WSS, zero mean and bounded by +/-1. A is a an amplitude, f is the center frequency and there is a random phase angle (uniform distribution around a circle). The R.V. are independent.

𝐸 𝑋 𝑡 𝐸 𝐴 ∙ 1 𝛽 ∙ 𝑚 𝑡 ∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜃

𝐸 𝑋 𝑡 𝐴 ∙ 𝐸 1 𝛽 ∙ 𝑚 𝑡 ∙ 𝐸 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜃

𝐸 𝑋 𝑡 𝐴 ∙ 1 0 ∙ 0 0

Autocorrelation

𝐸 𝑋 𝑡 ∙ 𝑋 𝑡 𝜏 𝑅 𝑡, 𝑡 𝜏

𝑅 𝑡, 𝑡 𝜏 𝐸 𝐴 ∙ 1 𝛽 ∙ 𝑚 𝑡 ∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜃

∙ 𝐴 ∙ 1 𝛽 ∙ 𝑚 𝑡 𝜏 ∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜏 𝜃

𝐸 𝑋 𝑡 ∙ 𝑋 𝑡 𝜏 𝐴 ∙ 𝐸1 𝛽 ∙ 𝑚 𝑡 𝛽 ∙ 𝑚 𝑡 𝜏 𝛽 ∙ 𝑚 𝑡 ∙ 𝑚 𝑡 𝜏∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜃 ∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜏 𝜃

𝑅 𝑡, 𝑡 𝜏 𝐴 ∙ 1 𝛽 ∙ 𝑅 𝜏 ∙12∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝜏

𝑅 𝜏𝐴2∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝜏

𝐴2∙ 𝛽 ∙ 𝑅 𝜏 ∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝜏

Forming the PSD

𝑆 𝑤𝐴4∙ 𝛿 𝑤 2𝜋 ∙ 𝑓 𝛿 𝑤 2𝜋 ∙ 𝑓

𝐴4∙ 𝛽 ∙ 𝑆 𝑤 ∗ 𝛿 𝑤 2𝜋 ∙ 𝑓 𝛿 𝑤 2𝜋 ∙ 𝑓

Which after performing the convolution becomes

𝑆 𝑤𝐴4∙ 𝛿 𝑤 2𝜋 ∙ 𝑓 𝛿 𝑤 2𝜋 ∙ 𝑓

𝐴4∙ 𝛽 ∙ 𝑆 𝑤 2𝜋 ∙ 𝑓 𝑆 𝑤 2𝜋 ∙ 𝑓

Your textbook simplifies the problem a bit … dealing with only the message and not the additional carrier component.




ExamplesofLinearSystemFrequency‐DomainAnalysis

Noise in a linear feedback system loop.

sX sY

1

1 ssA

sN

Linear superposition of X to Y and N to Y.

sNsYsXss

AsY

1

sNsXss

A

ss

AsY

11

1

sNsXss

A

ss

AsssY

11

2

sNAss

sssX

Ass

AsY

2

2

2

There are effectively two filters, one applied to X and a second apply to N.

Ass

AsH X

2 and

Ass

sssH N

2

2

sNsHsXsHsY NX

Generic definition of output Power Spectral Density:

wSwHwSwHwS NNNXXXYY 22




Change in the input to output signal to noise ratio.

dwwS

dwwS

SNR

NN

XX

In

dwwHN

wSH

dwwSwH

dwwSwH

SNR

N

XXX

NNN

XXX

Out20

2

2

2

2

0

EQ

XX

X

N

XX

Out BN

wS

dwH

wHN

wS

SNR

0

2

2

0

02

Where for this special case … (HN not a low pass or band pass filter … bad example)

dwH

wHB

X

NEQ 2

2

0

If the noise is added at the x signal input, the expected definition of noise equivalent bandwidth results.




Systems that Maximize Signal-to-Noise Ratio – Advanced Concept

SNR is defined as

EQNoise

Signal

BN

tsE

P

P

0

2

Define for an input signal tnts

Define for a filtered output signal tnts oo

For a linear system, we have:

0

dtntshtnts oo

The input SNR can be describe as

2

2

tnE

tsE

P

PSNR

Noise

Signalin

The output SNR can be described as

EQo

o

o

o

Noise

Signalout BN

tsE

tnE

tsE

P

PSNR

2

2

2

0

2

2

0

2

1dtthN

dtshE

SNR

o

out

Using Schwartz’s Inequality the numerator becomes

0

2

0

2

2

0

dtsEdhEdtshE




Applying this inequality, an SNR inequality can be defined as

0

2

0

2

0

2

2

1dtthN

dtsEdh

SNR

o

out

Canceling the filter terms, we have

0

22 dtsEN

SNRo

out

To achieve the maximum SNR, the equality condition of Schwartz’s Inequality must hold, or

0

2

0

2

2

0

dtsdhdtsh

This condition can be met for utsKh

where K is an arbitrary gain constant.

The desired impulse response is simply the time inverse of the signal waveform at time t, a fixed moment chosen for optimality. If this is done, the maximum filter power (with K=1) can be computed as

tdsdtsdht

2

0

2

0

2

tN

SNRo

out 2

max

This filter concept is called a matched filter. – Advanced Concept

If you wanted to detect a burst waveform that has been transmitted, to maximize the received SNR in white noise, the receiving filter should be the time inverse of the signal transmitted!

Note and caution: when using such a filter, the received signal maximum SNR will occur when the signal and convolved filter perfectly overlap. This moment in time occurs when the “complete” burst has been received by the system. If measuring the time-of-flight of the burst, the moment is exactly the filter length longer than the time-of-flight. (Think about where the leading edge of the signal-of-interest is when transmitted, when first received, and when fully present in the filter).




TheMatchedFilter–AdvancedConcept

Wikipedia: https://en.wikipedia.org/wiki/Matched_filter

“In signal processing, a matched filter is obtained by correlating a known signal, or template, with an unknown signal to detect the presence of the template in the unknown signal.[1][2] This is equivalent to convolving the unknown signal with a conjugated time-reversed version of the template. The matched filter is the optimal linear filter for maximizing the signal to noise ratio (SNR) in the presence of additive stochastic noise. Matched filters are commonly used in radar, in which a known signal is sent out, and the reflected signal is examined for common elements of the out-going signal. Pulse compression is an example of matched filtering. It is so called because impulse response is matched to input pulse signals. Two-dimensional matched filters are commonly used in image processing, e.g., to improve SNR for X-ray. Matched filtering is a demodulation technique with LTI (linear time invariant) filters to maximize SNR.[3].”

Applications:

Radar Sonar Pulse Compression Digital Communications (Correlation detectors) GPS pseudo-random sequence correlation

If you are looking for a signal, maximize the output of the filter when the signal is input! You will have a matched filter!

See ChirpCorrelationReceiver.m




Cooper & McGillem 9-6 Another optimal solution: Systems that Minimize the Mean-Square Error between the desired output and actual output – Advanced Concept

The error function tYtXtErr

where

0

dtNtXhtY

Performed in the Laplace Domain

sFsFsHsFsFsFsF NXXYXE

sFsHsHsFsFsFsF NXYXE 1

Computing the error power

j

j

NNXX dssHsHsSsHsHsSj

ErrE 112

12

j

j XXXXXX

NNXX dssSsHsSsHsS

sHsHsSsS

jErrE

2

12

Defining the input PSD sSsSsFsF NNXXCC

j

j

CC

NNXX

C

XXC

C

XXC

ds

sFsF

sSsS

sF

sSsHsF

sF

sSsHsF

jErrE

2

12

We can not do much about the last term, but we can minimize the terms containing H(s). Therefore, we focus on making the following happen

0

sF

sSsHsF

C

XXC

Step 1:Note that for this filter 1s- F

1

s F

1

CC sSsS NNXX

This is called a whitening filter as it forces the signal plus noise PSD to unity (white noise).

Step 2: Letting sF

sHsHsHsHC

221




Minimizing the terms containing H2(s), now we must focus on

sF

sSsH

C

XX

2 and

sF

sSsH

C

XX2

Letting H2 be defined for the appropriate Left or Right half-plane poles

Let LHPC

XX

sF

sSsH

2 and RHPC

XX

sF

sSsH

2

The composite filter is then

LHPC

XX

C sF

sS

sFsHsHsH

1

21

This solution is often called a Wiener Filter and is widely applied when the signal and noise statistics are known a-priori!




13.8 Example: Discrete Time Wiener Filter – Advanced Concept

Your textbook works to derive a discrete form of the Wiener Filter.

In it, you must calculate the coefficients of a Finite-Impulse-Response (FIR) digital filter.




Cooper & McGillem - Eigenvalue Based Filters – Advanced Concept We can continue a derivation started in the previous class discussion about time-sample filters, matrices and eigenvalues.

kxkwky

knkskx The expected value

HH kxkwkxkwEkykyE

HXXH kwkRkwkykyE

For a WSS input

HXXH kwRkwkykyE

If the signal and noise are zero mean, this becomes

HNNSSH kwRRkwkykyE

How do we maximize the output SNR

HNN

HSS

Noise

Signal

kwRkw

kwRkw

P

P

If we assume that the noise is white, IR NNN 2

H

HSS

NH

HSS

NNoise

Signal

kwkw

kwRkw

kwIkw

kwRkw

P

P

22

11

Performing a cholesky factorization of the signal autocorrelation matrix generates the following. Here, the numerator should suggest that an eigenvalue computation could provide a degree of simplification.

H

HHSS

NNoise

Signal

kwkw

kwRRkw

P

P

2

1

Once formed, the eigenvalue equation to solve is kwRkw S

which result in solutions for the resulting eigenvalues and eigenvectors of the form

2

2

2

1

NH

H

NNoise

Signal

kwkw

kwkw

P

P

Selecting the maximum eigenvalue and it’s eigenvector for the weight that maximizes the SNR!

2

2max

NNoise

Signal

P

P




13.9 The sampling Theorem for WSS Random Processes – Advanced Concept

If you take ECE 4550, you will be dealing with the sampling theorem.

It involves discrete time sampling of a continuous signal and the conditions under which the continuous signal can be regenerated from the discrete time samples. .




Advanced Topic Adaptive Filter – Advanced Concept

If a desired signal reference is available, we may wish to adapt a system to minimize the difference between the desired signal or signal characteristics and a filter input signal.

The following information is based on:

S. Haykin, Adaptive Filter Theory, 5th ed., Prentice-Hall, 2014.

There are four classes of Adaptive Filter Applications

Identification

Inverse Modeling

Prediction

Interference Cancellation

Identification

The mathematical Model of an “unknown plant”

In state space control system this is an adaptive observer of the Plant

Examples: Seismology predicting earth strata

InverseModeling

Providing an “Inverse Model” of the plant




For a transmission medium, the inverse model corrects non-ideal transmission characteristics.

An adaptive equalizer

Prediction

Based on past values, provide the best prediction possible of the present values.

Positioning/Navigation systems often need to predict where an object will be based on past observations

InterferenceCancellationExample

Cancellation of unknown interference that is present along with a desired signal of interest. Two sensors of signal + interference and just interference Reference signal (interference) is used to cancel the interference in the Primary signal

(noise + interference)




Classic Examples: Fetal heart tone monitors, spatial beamforming, noise cancelling headphones.

From: S. Haykin, Adaptive Filter Theory, 5th ed., Prentice-Hall, 2014

The “Reference signal” contains the unwanted interference. The goal of the adaptive filter is to match the reference signal with the “interference” in the “Primary signal and force the output “difference error” to be minimized in power. Since “interference” is the only thing available to work with, the “power minimum” solution would be one where the interference is completely removed!

These techniques are based on the Weiner filter solution. While the signal and interference statistics are not known a-priori (before the filter gets started), after a number of input samples they can be estimated and used to form the filter coefficients. Then, as time continues, there is a sense that the estimates should improve until the adaptive coefficients are equal to those that would be computed with a-priori information.

The advantage ... adaptive filter can work when the statistics are slowly time varying!

Note: An application using an LMS adaptive filter is not too difficult for a senior project! (noise cancelling headphones, remove 60 cycle hum, etc.)




Textbook:Cancellinganinterferingwaveform–AdvancedConcept

The example in your textbook p. 407-411.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-20

-10

0

10

20or

igin

al

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-10

-5

0

5

10

15

Filt

ered

Time (sec)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-8

-6

-4

-2

0

2

4

6Adaptive Weights in Time

Tim

e (s

ec)

Weights

a1

a2

a3

a4




Matlab % % Text Adaptive Filter Example % clear close all t=0:1/200:1; % Interfereing Signal - 60 Hz n=10*sin(2*pi*60*t+(pi/4)*ones(size(t))); % Signal of interest and S + I x1=1.1*(sin(2*pi*11*t)); x2=x1+n; % Reference signal to excise r=cos(2*pi*60*t); m=0.15; a=zeros(1,4); z=zeros(1,201); z(1:4)=x2(1:4); w(1,:)=a'; w(2,:)=a'; w(3,:)=a'; w(4,:)=a'; % Adaptive weight computation and application for k=4:200 a(1)=a(1)+2*m*z(k)*r(k); a(2)=a(2)+2*m*z(k)*r(k-1); a(3)=a(3)+2*m*z(k)*r(k-2); a(4)=a(4)+2*m*z(k)*r(k-3); z(k+1)=x2(k+1)-a(1)*r(k+1)-a(2)*r(k)-a(3)*r(k-1)-a(4)*r(k-2); w(k+1,:)=a'; end figure(1) subplot(2,1,1); plot(t,x2,'k') ylabel('original') subplot(2,1,2) plot(t,z,'k');grid; ylabel('Filtered'); xlabel('Time (sec)'); figure(2) plot(t,w);grid; title('Adaptive Weights in Time') ylabel('Time (sec)') xlabel('Weights') legend('a1','a2','a3','a4');

Review Skills 7 and Skills 8 from the solution web site. Exam based questions! Hopefully you have looked at them and potentially tried a few.




Point totals beyond 3 for this homework will be considered extra credit …

Highlights from Skills 8 include the following

36.1 Suppose the circuit shown below has input ttx 9sin6 , where is a random

variable uniformly distributed on [0,2pi]. Assuming the R=1 M and C=1uF,

a. If the output signal is y(t), find the transfer function of the circuit. (H(s) possible given)

sX

sCR

RsY

1

sCR

sCRsH

1 and

s

ssH

1

111 2

2

s

s

s

s

s

ssHsH

b. Find the spectral density Syy(w) of the output and simplify. (Need AC. of Rxx and PSD Sxx)

99sin69sin6 ttERXX

99sin9sin36 ttERXX

2918cos9cos2

36 tERXX

9cos18 XXR

dtjwttwS XX exp9cos18

dtjwttjtjwS XX exp9exp9exp9

999 wwwS XX

Now the PSD of the output can be computer

1

9992

2

w

wwwwSYY




9982

81999

19

99

2

2

wwwwwSYY

c. Sketch the spectral density Syy(w).

Syy(w) is almost identical to Sxx(w). The filter used is a high-pass filter with a relative cutoff frequency w0 of 1.

d. What would happen if the input term had a “DC” component? What would the filter output for signals at w=0 be?

36.5 Consider the following linear circuit, where x(t) is the input voltage signal and y(t) is the output voltage signal.

x(t)

C

y(t)

LR

a. Find the transfer function of this system. For R=25 ohms, L=5H and C=0.05F (Note: these are not realistic values!)

sXsLsCR

sCsY

1

1

05.052505.01

1

1

122

ssLCssCR

sH

41

4

125.01

1

25.025.11

12

ssssss

sH

161

16

41

4

41

422

ssssss

sHsH

b. Assume the input signal is 21 5cos42cos312 tttx , where 1 and 2 are independent random variables that are uniform on the interval [0,2pi]. From this compute (1) the autocorrelation of the input signal and (2) the power spectral density of the input signal.

21

21

55cos422cos312

5cos42cos312

tt

ttERXX




22

122

21

111

21

55cos5cos16

22cos5cos125cos48

55cos2cos12

22cos2cos92cos36

55cos4822cos36144

tt

ttt

tt

ttt

tt

ERXX

2

2121

1212

1

2510cos2

165cos

2

16

27cos2

1223cos

2

12

58cos2

1253cos

2

12

224cos2

92cos

2

9144

t

tt

tt

t

ERXX

5cos82cos2

9144 XXR

dtjwtwS XX exp5cos82cos

2

9144

558222

9288 wwwwwwS XX

c. Compute (1) the autocorrelation of the output signal and (2) the spectral density of the output signal. Simplify all answers.

Output Power Spectrum sHsHsSsS XXYY

161

1622

ww

wSwS XXYY

161

16

558222

9288

22

ww

wwwwwwSYY

161

16558

161

1622

2

9

161

16288

22

22

22

wwww

wwww

www

wSYY




16515

16558

16212

1622

2

9

16010

16288

22

22

22

ww

ww

w

wSYY

55533

64

22100

72

288

551066

168

22100

16

2

9

288

ww

ww

w

ww

ww

w

wSYY

dwjwwSR YYYY

exp2

1

dwjw

ww

wwwRYY

exp

55533

64

22100

72288

2

1

5cos533

642cos

100

72144 YYR

d. Compute the ratio of the output to input total average power of the signals.

5cos82cos2

9144

5cos533

642cos

100

72144

0

0

XX

YY

in

out

R

R

P

P

9255.0

5.156

84.144

82

9144

533

64

100

72144

0

0

XX

YY

in

out

R

R

P

P

e. Compute the ratio of the output to input dc average power of the two signals. 1

144

144

XX

YY

in

out

R

R

P

P

f. From the above, comment on the filtering effect of the original LCR circuit. Is it a low pass, band pass, or high pass filter?




This is a low-pass filter where dc is passed from the input to the output. For the selected input signal, with oscillation at w=2 and w=5, the filter first rolls off at w=1 and then continues at w=4.

The input to output power for w=3 is 16.025

4

9

72

100

2

2

9100

72

in

out

P

P

The input to output power for w=5 is 015.0533

8

8533

64

in

out

P

P

chapter 13b: random signals and noisehomepages.wmich.edu/~bazuinb/ece3800/b_notes13b.pdf · 2020....

Documents