correlation and matched filters
TRANSCRIPT
![Page 1: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/1.jpg)
6.003: Signal Processing
Correlation and Matched Filters
November 20, 2018
![Page 2: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/2.jpg)
Last Lab: Effects of Noise
Adding imperceptible amounts of noise → enormous consequences.
In the absence of noise, blurring was removed by inverse filtering.
inversefilter
But imperceptible noise at the input dominated the output image.
inversefilter
+
noise(< 0.002/pixel)
Fixed by making small changes to the inverse filter.
![Page 3: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/3.jpg)
Noise is Everywhere
Sources of noise.
Thermal (Johnson) noise: electronic noise caused by thermal ag-
itations. Manifests as an additive noise with gaussian distribution.
Quantization: technically a distortion (since it’s not random) these
errors result when we represent sample values with elements from a
discrete set (such as 8-bit representations for conventional images).
Shot (Poisson) noise: variations in brightness caused by the parti-
cle nature of photons. If photons arise independently but at a given
rate, then the number of photons collected over some time period
varies in proportion to the square root of the average number.
![Page 4: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/4.jpg)
Communications
Noise considerations are of central importance in communications
applications, where the goal is to move information.
Example: serial transmission of DNA sequence as follows:
1
−1
4
A: x1(t)
t
1
−1
4
G: x2(t)
t
1
−14
C: x3(t)
t
1
−14
T: x4(t)
t
1
−128
sequence: y(t)
t
T G T T C T T
![Page 5: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/5.jpg)
Communications
Given y(t):
1
−128
y(t)
t
What is the best way to decode the message?
decodery(t) T, G, T, T, C, T, T
What is the best way to decode if y(t) is corrupted by noise?
decoder+
noise
y(t) T, G, T, T, C, T, T
![Page 6: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/6.jpg)
Check Yourself
Assume that the decoder can be implemented with an LTI filter.
1
−1
4
x(t)
t h(t) y(t)
Each of the following impulse responses is 0 if t < 0 or
t > 4. Which one maximizes y(4) for x(t) shown above?
1
−1
h1(t)
t
1
−1
h2(t)
t
1
−1
h3(t)
t
1
−1
h4(t)
t
![Page 7: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/7.jpg)
Check Yourself
Assume that the decoder can be implemented with an LTI filter.
1
−1
4
x(t)
t h(t) y(t)
y(4) =∫ ∞−∞
h(τ)x(4− τ)dτ =∫ 4
0h(τ)x(4− τ)dτ
This is maximized by choosing h(t) so the integrand is always 1:
h(t) = 1 if x(4− t) = 1−1 if x(4− t) = −1
Thus h(t) = x(4− t) = h4(t).
1
−1
h1(t)
t
1
−1
h2(t)
t
1
−1
h3(t)
t
1
−1
h4(t)
t
![Page 8: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/8.jpg)
Check Yourself
Assume that the decoder can be implemented with an LTI filter.
1
−1
4
x(t)
t h(t) y(t)
Each of the following impulse responses is 0 if t < 0 or
t > 4. Which one maximizes y(4) for x(t) shown above? h4
1
−1
h1(t)
t
1
−1
h2(t)
t
1
−1
h3(t)
t
1
−1
h4(t)
t
![Page 9: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/9.jpg)
Communications
Today’s lecture provides some new concepts for thinking about com-
munications systems.
decoder+
noise
y(t) T, G, T, T, C, T, T
![Page 10: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/10.jpg)
Correlation
Cross-correlation measures the similarity of two signals.
Cross-Correlation:
(x ? y)(t) ≡∫ ∞−∞
x(τ) y(τ + t) dτ
combines parts of signals with the same time difference:
(τ + t) – τ = t
Similar to Convolution:
(x ∗ y)(t) ≡∫ ∞−∞
x(τ) y(t− τ) dτ
but convolution combines parts of signals with the same total lag:
(t− τ) + τ = t
![Page 11: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/11.jpg)
Applications of Cross-Correlation
Large cross-correlation at t = tp indicates similarity of x(t) and y(t+tp).
Therefore we can find a target signal x(t)
t
x(t)
in a noisy signal y(t)
t
y(t)
by finding the peak in (x ? y)(t):
tpt
(x ? y)(t)
![Page 12: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/12.jpg)
Relation Between Correlation and Convolution
Definitions.
(x ? y)(t) =∫x(τ)y(τ + t)dτ
(x ∗ y)(t) =∫x(τ)y(t− τ)dτ
Let xf (t) = x(−t) represent a flipped version of x(t):
(xf ? y)(t) =∫x(−τ) y(τ + t) dτ =
∫x(λ) y(t− λ) dλ
= (x ∗ y)(t)
Correlating xf (t) with y(t) is the same as convolving x(t) and y(t).
Let yf (t) = y(−t) represent a flipped version of y(t):
(x ? yf )(t) =∫x(τ) y(−τ − t) dτ
= (x ∗ y)(−t)
Different from correlating xf (t) with y(t); different from convolution.
![Page 13: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/13.jpg)
Correlation is NOT Commutative
Commuting the order of the inputs flips the correlation.
(x ? y)(t) =∫x(τ)y(τ + t)dτ
(y ? x)(t) =∫y(τ)x(τ + t)dτ
=∫x(λ)y(λ− t)dλ where λ = τ + t
= (x ? y)(−t)
![Page 14: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/14.jpg)
Radar, Ultrasound, and Optical Tomography
Cross-correlation is widely used in range-finding applications.
Each of these images was created by correlating an input signal
(radio wave, ultra-sound, or light) with its reflection off a structure.
![Page 15: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/15.jpg)
Autocorrelation
Autocorrelation measures patterns within a signal.
Cross-Correlation:
(x ? y)(t) ≡∫ ∞−∞
x(τ) y(τ + t) dτ
Autocorrelation:
(x ? x)(t) ≡∫ ∞−∞
x(τ)x(τ + t) dτ
![Page 16: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/16.jpg)
Finding Pitch with Autocorrelation
Speech utterance ”bat” (left) and autocorrelation (right).
Periodicity is still clear in autocorrelation after adding noise.
![Page 17: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/17.jpg)
Energy
The autocorrelation at t = 0 is equal to the energy in the signal.
(x ? x)(t) =∫x(τ)x(τ + t) dτ
(x ? x)(0) =∫x(τ)x(τ) dτ =
∫x2(t) dt
The total energy of a continuous-time signal x(t) is defined as
E =∫x2(t) dt
and that of a discrete-time signal x[n] is similarly
E =∑
x2[n]
Neither of these definitions matches the conventional notion of en-
ergy in physics, but the concepts are closely related. For example,
if x(t) represents the voltage across a 1Ω resistor, then E equals the
energy dissipated by the resistor.
![Page 18: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/18.jpg)
Parseval’s Relation
Energy can also be calculated from the Fourier transform.
E =∫x2(t) dt
=∫x(t)x(t) dt
=∫x(t)
( 12π
∫X(ω)ejωtdω
)dt
= 12π
∫X(ω)
(∫x(t)ejωtdt
)dω
= 12π
∫X(ω)X∗(ω) dω
= 12π
∫ ∣∣X(ω)∣∣2dω
![Page 19: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/19.jpg)
Check Yourself
Let
f(t) ctft⇐⇒ F (ω)
g(t) ctft⇐⇒ G(ω)
Which of the following represents the CTFT of (f ? g)(t)?
1. F (−ω)G(ω)2. F (ω)G(−ω)3. F (−ω) ∗G(ω)4. F (ω) ∗G(−ω)5. none of the above
![Page 20: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/20.jpg)
Check Yourself
Let
f(t) ctft⇐⇒ F (ω)
g(t) ctft⇐⇒ G(ω)
Let ff (t) = f(−t). Then
(f ? g)(t) = (ff ∗ g)(t) ctft⇐⇒ Ff (ω)G(ω) = F (−ω)G(ω)
![Page 21: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/21.jpg)
Check Yourself
Let
f(t) ctft⇐⇒ F (ω)
g(t) ctft⇐⇒ G(ω)
Which of the following represents the CTFT of (f ? g)(t)?
1. F (−ω)G(ω)2. F (ω)G(−ω)3. F (−ω) ∗G(ω)4. F (ω) ∗G(−ω)5. none of the above
![Page 22: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/22.jpg)
Schwartz Inequality(∫ T
0x(t)h(t)dt
)2
≤
(∫ T
0x2(t)dt
)(∫ T
0h2(t)dt
)
Define the “inner product” of two functions as 〈a, b〉 =∫ T
0 a(t)b(t) dt.
Let λ = 〈x,h〉〈h,h〉 where 〈h, h〉 > 0 if h(t) 6= 0 (if h(t) = 0 the proof is trivial).
Then
0 ≤ 〈x− λh, x− λh〉= 〈x, x〉 − 2λ〈x, h〉+ λ2〈h, h〉
= 〈x, x〉 − 2(〈x, h〉)2
〈h, h〉+ (〈x, h〉)2〈h, h〉
(〈h, h〉)2
= 〈x, x〉 − (〈x, h〉)2
〈h, h〉and therefore
(〈x, h〉)2 ≤ 〈x, x〉〈h, h〉
![Page 23: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/23.jpg)
Optimality of Matched Filtering
We can use the Schwartz inequality to show that a matched filter
optimally detects x(t) from the output y(t) for 0 ≤ t ≤ T , as follows.
If h(t) is the impulse response of the decoding system, then
y(T ) = (x ∗ h)(T ) =∫ T
0x(τ)h(T − τ)dτ =
∫ T
0x(τ)hf (τ)dτ = 〈x, hf 〉
where hf (t) = h(T − t). By the Schwartz inequality,
(〈x, hf 〉)2 ≤ 〈x, x〉〈hf , hf 〉 .
If the energy in h(t) is bounded so that 〈h, h〉 = 〈hf , hf 〉 ≤ 〈x, x〉,
(〈x, hf 〉)2 ≤ 〈x, x〉〈hf , hf 〉 ≤ (〈x, x〉)2.
For a matched filter, h(t) = x(T − t) and
〈x, h〉 = (x ∗ h)(T ) =∫ T
0x(τ)h(T − τ)dτ =
∫ T
0x(τ)x(τ)dτ = 〈x, x〉
Thus the matched filter achieves the Schwartz bound with equality.
This means that no impulse response with energy less than that in
x(t) has output y(T ) that is bigger than that of a matched filter.
![Page 24: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/24.jpg)
Decoding with a Matched Filter (why look at t = T?)
The simplest form of a matched filter has impulse response
h(t) = x(−t) .
Let y(t) = (x ∗ h)(t) =∫x(τ)h(t− τ) dτ =
∫x(τ)x(τ − t) dτ
Then, by the Schwartz inequality,(∫x(τ)x(τ − t)dτ
)2≤(∫
x(τ)x(τ) dτ)(∫
x(τ − t)x(τ − t) dτ)
=(∫
x(τ)x(τ) dτ)(∫
x(λ)x(λ) dλ)
=(∫
x(τ)x(τ) dτ)2
Thus the maximum output of such a filter occurs at t = 0.
Shifting the impulse response by t = T (where T= duration of x(t))
hc(t) = x(T − t)
makes it causal, and shifts the maximum output to t = T .
![Page 25: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/25.jpg)
Complex Signals
Correlation, energy, Parseval’s relation, and matched filters can be
generalized to apply to complex-valued signals.
Correlation
(x ? y)(t) =∫x(τ)y∗(τ + t)dτ
Energy
E =∫x(t)x∗(t) dt =
∫ ∣∣x(t)|2dt
Parseval’s Relation∫ ∣∣x(t)|2dt = 12π
∫ ∣∣F (ω)∣∣2dω
Matched Filter
h(t) = x∗(T − t)
![Page 26: Correlation and Matched Filters](https://reader034.vdocuments.site/reader034/viewer/2022050104/626c67060464d55d2e39e327/html5/thumbnails/26.jpg)
Summary
Correlation quantifies the similarity of signals.
– useful for quantifying time shifts (as in radar and tomography)
Autocorrelation quantifies patterns within a signal.
– useful for finding the period of a quasi-periodic signal
Matched filters are useful for finding a pattern that is specified by
one signal in a second signal.
– useful in decoding information coded in a message