july 30th, 2004comp.dsp conference1 frequency estimation techniques peter j. kootsookos...
TRANSCRIPT
July 30th, 2004 comp.dsp conference 1
Frequency Estimation TechniquesPeter J. Kootsookos
July 30th, 2004 comp.dsp conference 2
Frequency Estimation TechniquesTalk Summary
• Some acknowledgements• What is frequency estimation?
o What other problems are there?
• Some algorithmso Maximum likelihoodo Subspace techniqueso Quinn-Fernandes
• Associated problemso Analytic signal generation
Kay / Lank-Reed-Pollon estimatorso Performance bounds: Cramér-Rao Lower Bound
July 30th, 2004 comp.dsp conference 3
Frequency Estimation TechniquesSome Acknowledgements
• Eric Jacobson – for his presence on comp.dsp and for his work on the topic.
• Andrew Reilly – for his presence on comp.dsp and for analytic signal advice.
• Steven M. Kay – for his books on estimation and detection generally, and published research work on the topic.
• Barry G. Quinn – as a colleague and for his work the topic.
• I. Vaughan L. Clarkson – as a colleague and for his work on the topic.
• CRASys – Now defunct Cooperative Research Centre for Robust & Adaptive Systems.
July 30th, 2004 comp.dsp conference 4
Frequency Estimation TechniquesTalk Summary
• Some acknowledgements• What is frequency estimation?
o What other problems are there?
• Some algorithmso Maximum likelihoodo Subspace techniqueso Quinn-Fernandes
• Associated problemso Analytic signal generation
Kay / Lank-Reed-Pollon estimatorso Performance bounds: Cramér-Rao Lower Bound
July 30th, 2004 comp.dsp conference 5
Frequency Estimation TechniquesWhat is frequency estimation?
Find the parameters A, , , and 2 in
y(t) = A cos [t-) + )] + (t)
where t = 0..T-1, T-1/2 and (t) is a noise with zero mean and variance 2.
is used to denote the vector [A 2 ]T.
July 30th, 2004 comp.dsp conference 6
Frequency Estimation TechniquesWhat other problems are there?
y(t) = A cos [t-) + )] + (t)
• What about A(t) ?o Estimating A(t) is envelope estimation (AM demodulation).o If the variation of A(t) is slow enough, the problem of
estimating and estimating A(t) decouples.
• What about (t)?o This is the frequency tracking problem.
• What’s (t) ?o Usually assumed additive, white, & Gaussian.o Maximum likelihood technique depends on Gaussian
assumption.
July 30th, 2004 comp.dsp conference 7
Frequency Estimation TechniquesWhat other problems are there? [continued]
Amplitude-varying example: condition monitoring in rotating machinery.
July 30th, 2004 comp.dsp conference 8
Frequency Estimation TechniquesWhat other problems are there? [continued]
Frequency tracking example: SONAR
Thanks to Barry Quinn & Ted Hannan for the plot from their book “The Estimation & Tracking of Frequency”.
July 30th, 2004 comp.dsp conference 9
Frequency Estimation TechniquesWhat other problems are there? [continued]
Multi-harmonic frequency estimation
y(t) = Am cos [mt-) + m)] + (t)
• For periodic, but not sinusoidal, signals.
• Each component is harmonically related to the fundamental frequency.
p
m=1
July 30th, 2004 comp.dsp conference 10
Frequency Estimation TechniquesWhat other problems are there? [continued]
Multi-tone frequency estimation
y(t) = Am cos [mt-) + m)] + (t)
• Here, there are multiple frequency components with no relationship between the frequencies.
p
m=1
July 30th, 2004 comp.dsp conference 11
Frequency Estimation TechniquesTalk Summary
• Some acknowledgements• What is frequency estimation?
o What other problems are there?
• Some algorithmso Maximum likelihoodo Subspace techniqueso Quinn-Fernandes
• Associated problemso Analytic signal generation
Kay / Lank-Reed-Pollon estimatorso Performance bounds: Cramér-Rao Lower Bound
July 30th, 2004 comp.dsp conference 12
Frequency Estimation TechniquesThe Maximum Likelihood Approach
The likelihood function for this problem, assuming that (t) is Gaussian is
L() = 1/((2)T/2|R|) exp(–(Y –Ŷ())TR-1(Y –Ŷ())/ 2)
where R= The covariance matrix of the noise
Y = [y(0) y(1) … y(T-1)]T
Ŷ = [A cos() A cos( + ) … A cos((T-1) + )]T
Y is a vector of the date samples, and Ŷ is a vector of the modeled samples.
July 30th, 2004 comp.dsp conference 13
Frequency Estimation TechniquesThe Maximum Likelihood Approach [continued]
Two points to note:
• The functional form of the equation
L() = 1/((2)T/2|R|) exp(–(Y –Ŷ())TR-1(Y –Ŷ())/ 2)
is determined by the Gaussian distribution of the noise.
• If the noise is white, then the covariance matrix R is just 2I – a scaled identity matrix.
July 30th, 2004 comp.dsp conference 14
Frequency Estimation TechniquesThe Maximum Likelihood Approach [continued]
Often, it is easier to deal with the log-likelihood function:
ℓ () = –(Y –Ŷ())TR-1(Y –Ŷ())
where the additive constant, and multiplying constant have been ignored as they do not affect the position of the peak (unless is zero or infinite).
If the noise is also assumed to be white, the maximum likelihood problem looks like a least squares problem as maximizing the expression above is the same as minimizing
(Y –Ŷ())T(Y –Ŷ())
July 30th, 2004 comp.dsp conference 15
Frequency Estimation TechniquesThe Maximum Likelihood Approach [continued]
If the complex-valued signal model is used, then estimating is equivalent to maximizing the periodogram:
P() =| y(t) exp(-i t) |2
For the real-valued signal used here, this equivalence is only true as T tends to infinity.
t=0
T-1
July 30th, 2004 comp.dsp conference 16
Frequency Estimation TechniquesTalk Summary
• Some acknowledgements• What is frequency estimation?
o What other problems are there?
• Some algorithmso Maximum likelihoodo Subspace techniqueso Quinn-Fernandes
• Associated problemso Analytic signal generation
Kay / Lank-Reed-Pollon estimatorso Performance bounds: Cramér-Rao Lower Bound
July 30th, 2004 comp.dsp conference 17
Frequency Estimation TechniquesSubspace Techniques
The peak of the spectrum produced by spectral estimators other than the periodogram can be used for frequency estimation.
Signal subspace estimators use either
PBar() = v*() RBar v()or
PMV() = 1/( v*() RMV-1 v() )
where v() = [ 1 exp(iexp(i2exp(I(T-1)and an estimate of the covariance matrix is used.
^
^
Note: If Ryy is full rank, the PBar is the same as the periodogram.
July 30th, 2004 comp.dsp conference 18
Frequency Estimation TechniquesSubspace Techniques - Signal
Bartlett:
RBar = k e k e*k
Minimum Variance:
RMV -1 = 1/k e k e*k
Assuming there are p frequency components.
^
^
k=1
p
k=1
p
July 30th, 2004 comp.dsp conference 19
Frequency Estimation TechniquesSubspace Techniques - Noise
Pisarenko:
RPis -1 = e p+1 e*p+1
Multiple Signal Classification (MUSIC):
RMUSIC -1 = e k e*k
Assuming there are p frequency components.
Key Idea: The noise subspace is orthogonal to the signal subspace, so zeros of the noise subspace will indicate signal frequencies.
^
^M
k=p+1
While Pisarenko is not statistically efficient, it is very fast to calculate.
July 30th, 2004 comp.dsp conference 20
Frequency Estimation TechniquesQuinn-Fernandes
The technique of Quinn & Fernandes assumes that the data fits the ARMA(2,2) model:
y(t) – y(t-1) + y(t-2) = (t) – (t-1) + (t-2)
1. Set 1 = 2cos().2. Filter the data to form
zj (t) = y(t) + jzj (t-1) – zj(t-2)
3. Form j by regressing ( zj (t) + zj (t-2) ) on zj (t-1)
j = t( zj (t) + zj (t-2) ) zj (t-1) / t zj2
(t-1)
4. If |j - j | is small enough, set = cos-1(j / 2), otherwise set j+1 = j and iterate from 2.
July 30th, 2004 comp.dsp conference 21
Frequency Estimation TechniquesQuinn-Fernandes [continued]
The algorithm can be interpreted as finding the maximum of a smoothed periodogram.
July 30th, 2004 comp.dsp conference 22
Frequency Estimation TechniquesTalk Summary
• Some acknowledgements• What is frequency estimation?
o What other problems are there?
• Some algorithmso Maximum likelihoodo Subspace techniqueso Quinn-Fernandes
• Associated problemso Analytic signal generation
Kay / Lank-Reed-Pollon estimatorso Performance bounds: Cramér-Rao Lower Bound
July 30th, 2004 comp.dsp conference 23
Frequency Estimation TechniquesAssociated Problems
Other questions that need answering are:
• What happens when the signal is real-valued, and my frequency estimation technique requires a complex-valued signal?
o Analytic Signal generation
• How well can I estimate frequency?
o Cramer-Rao Lower Boundo Threshold performance
July 30th, 2004 comp.dsp conference 24
Frequency Estimation TechniquesAssociated Problems: Analytic Signal Generation
Many signal processing problems already use “analytic” signals: communications systems with “in-phase” and “quadrature” components, for example.
An analytic signal, exp(i-blah), can be generated from a real-valued signal, cos(blah) , by use of the Hilbert transform:
z(t) = y(t) + i H[ y(t) ]
where H[.] is the Hilbert transform operation.
Problems occur if the implementation of the Hilbert transform is poor. This can occur if, for example, too short an FIR filter is used.
July 30th, 2004 comp.dsp conference 25
Frequency Estimation TechniquesAssociated Problems: Analytic Signal Generation [continued]
Another approach is to FFT y(t) to obtain Y(k). From Y(k), form
Z(k) = 2Y(k) for k = 1 to T/2 - 1
Y(k) for k = 0
0 for k = T/2 to T
and then inverse FFT Z(k) to find z(t).
Unless Y(k) is interpolated, this can cause problems.
Makes sure the DC term is correct.
July 30th, 2004 comp.dsp conference 26
Frequency Estimation TechniquesAssociated Problems: Analytic Signal Generation [continued]
If you know something about the signal (e.g. frequency range of interest), then use of a band-pass Hilbert transforming filter is a good option.
See the paper by Andrew Reilly, Gordon Fraser & Boualem Boashash, “Analytic Signal Generation : Tips & Traps” IEEE Trans. on ASSP, vol 42(11), pp3241-3245
They suggest designing a real-coefficient low-pass filter with appropriate bandwidth using a good FIR filter algorithm (e.g. Remez). The designed filter is then modulated with a complex exponential of frequency fs/4.
July 30th, 2004 comp.dsp conference 27
Frequency Estimation TechniquesKay’s Estimator and Related Estimators
If an analytic signal, z(t), is obtained, then the simple relation:
arg( z(t+1)z*(t) )
can be used to find an estimate of the frequency at time t.
See this by writing:
z(t+1)z*(t) = exp(i ((t+1) + ) ) exp(-i (t + ) )
= exp(i )
July 30th, 2004 comp.dsp conference 28
Frequency Estimation TechniquesKay’s Estimator and Related Estimators [continued]
What Kay did was to form an estimator
= arg( w(t) z(t+1)z*(t) )
where the weights, w(t), are chosen to minimize the mean square error.
Kay found that, for very small noisew(t) = 6t(T-t) / (T(T2-1))
which is a parabolic window.
T-2
t=0
^
July 30th, 2004 comp.dsp conference 29
Frequency Estimation TechniquesKay’s Estimator and Related Estimators [continued]
If the SNR is known, then it’s possible to choose an optimal set of weights.
For “infinite” noise, the rectangular window is best – this is the Lank-Reed-Pollon estimator.
The figure shows how the weights vary with SNR.
July 30th, 2004 comp.dsp conference 30
Frequency Estimation TechniquesAssociated Problems: Cramer-Rao Lower Bound
The lower bound on the variance of unbiased estimators of the frequency a single tone in noise is
var() >= 122 / (T(T2-1)A2)^
July 30th, 2004 comp.dsp conference 31
Frequency Estimation TechniquesAssociated Problems: Cramer-Rao Lower Bound [continued]
The CRLB for the multi-harmonic case is:
var() >= 122 / (T(T2-1) m2Am2)
So the effective signal energy in this case is influenced by the square of the harmonic order.
p
m=1
^
July 30th, 2004 comp.dsp conference 32
Frequency Estimation TechniquesAssociated Problems: Threshold Performance
Key idea: The performance degrades when peaks in the noise spectrum exceed the peak of the frequency component.
Dotted lines in the figure show the probability of this occurring.
July 30th, 2004 comp.dsp conference 33
Frequency Estimation TechniquesAssociated Problems: Threshold Performance [continued]
For the multi-harmonic case, two threshold mechanisms occur: the noise outlier case and rational harmonic locking.
This means that, sometimes, ½, 1/3, 2/3, 2 or 3 times the true frequency is estimated.
July 30th, 2004 comp.dsp conference 34
Frequency Estimation TechniquesTalk Summary
• Some acknowledgements• What is frequency estimation?
o What other problems are there?
• Some algorithmso Maximum likelihoodo Subspace techniqueso Quinn-Fernandes
• Associated problemso Analytic signal generation
Kay / Lank-Reed-Pollon estimatorso Performance bounds: Cramér-Rao Lower Bound
July 30th, 2004 comp.dsp conference 35
Frequency Estimation TechniquesThanks!
Thanks to Lori Ann, Al and Rick for hosting and/or organizing this get-together.
July 30th, 2004 comp.dsp conference 36
Frequency Estimation TechniquesGood-bye!