Detection techniques for validating Doppler estimates in heterodyne lidar

Download Detection techniques for validating Doppler estimates in heterodyne lidar

Post on 03-Oct-2016




0 download

Embed Size (px)


<ul><li><p>in</p><p>ioaarft,Tiko</p><p>tral or correlation function estimates.1,2 Thesethat degrade estimator performance lies in normalreceiver noise as is usually the case in lidar, consen-estimates are obtained by independent measure-ments of returns from the range gate within a shorttime compared with the period characteristic of grossfluctuations in the wind speed being measured. Weaddress two remaining questions here. First, howdo we know, in practice, when the accumulation hasbeen sufficient to ensure optimal performance?Second, what can be done to evaluate an estimatewhen sufficient accumulation has not or cannot beused?Somewhat similar problems have been addressed</p><p>sus appears to offer no advantage over accumulation,1and that detection ~or discrimination! techniques ofthe type we consider in this paper improve estimateselectivity compared with consensus.5 The differ-ence is that, in radar, spurious estimates are oftenoutliers. That is, they are not consequences of theaerosol backscatter and system statistics but arisefrom clutter, real objects that are not wind borne.For example, the statistical averaging method wasintroduced recently to remove outliers caused bybackscatter from birds, which give rise to signals thatcan be distinguished from atmospheric returns be-cause their statistics are non-Gaussian. In this pa-per we continue to assume that Doppler lidars are notsusceptible to outliers and consider the problem inthe ideal limit that spurious estimates are noise re-lated and can be described by the use of the ~known!statistics of the receiver noise. Two other tech-niques used in Doppler radar remain of interest.The first is context analysis in which an estimate iscompared with others from neighboring range gatesand observation times. Different forms of this tech-nique, some using pattern recognition, are used in</p><p>B. J. Rye is with the Environmental Technology Laboratory,Cooperative Institute for Research in Environmental Science ~Uni-versity of Colorado and National Oceanic and Atmospheric Admin-istration!, 325 Broadway, Boulder, Colorado 80303. R. M.Hardesty is with the National Oceanic and Atmospheric Adminis-tration Environmental Technology Laboratory, 325 Broadway,Boulder, Colorado 80303.Received 3 June 1996; revised manuscript received 23 Septem-</p><p>ber 1996.0003-6935y97y091940-12$10.00y0 1997 Optical Society of AmericaDetection techniques forvalidating Doppler estimates</p><p>Barry J. Rye and R. Michael Hardesty</p><p>We investigate the ability of detectheterodyne lidar Doppler estimatesdistinction between estimates thatRao lower bound. The conditionaltion of a signal when none is presenof selection of a spurious estimate.detection functions other than the lcessing rejected data in the light of c 1997 Optical Society of America</p><p>1. Introduction</p><p>Unbiased Doppler frequency shift estimates can beobtained from pulsed heterodyne lidar systems withperformance limited essentially by a CramerRaolower bound ~CRLB!. But this is achieved only if athreshold, which is characterized mainly by the totalenergy in the signal, is surpassed. At low signallevels, it may be necessary to augment the data sam-ple from a single-range gate by accumulation of spec-1940 APPLIED OPTICS y Vol. 36, No. 9 y 20 March 1997heterodyne lidar</p><p>n techniques based on the likelihood ratio to discriminate betweent low signal levels using examples generated by simulation. Thee regarded as acceptable and as spurious is based on the Crameralse alarm probability, which ordinarily describes recording detec-is then found to be an approximate upper bound on the probabilityhe method is superior theoretically to similar techniques based onelihood ratio. The likelihood ratio also provides a basis for repro-ntextual information provided by those estimates that are accepted.</p><p>in Doppler radar by the introduction of the consen-sus3 and statistical averaging method.4 In eachcase, the time series data are divided into segments,and an estimate is obtained from each segment. Theconsensus technique searches for clustering withinthe set of estimates obtained; if insufficient clusteringis obtained, the data set is rejected. We have arguedearlier that, if the reason for the spurious estimates</p></li><li><p>radar to distinguish outliers, but we comment hereinthat detection leads naturally to a context analysismethod that might be implemented in lidar. Thesecond technique is the coherence field, or normalizedcoherent power diagnostic, in which a function of thefirst lag term in the time series autocorrelation func-tion ~ACF! ~indicating presence of a coherent signal!is recorded, and data are rejected if this function doesnot lie above a threshold value. This approach isimplemented in Doppler processors in both radarsand lidars, and although apparently undocumented,has some similarity with the method described here.Our purpose, then, is to describe and illustrate a</p><p>straightforward method of selecting estimates at lowsignal levels that uses standard Doppler processingand is based closely on standard detection techniquesand to suggest routes whereby it might be extended.Some details of the assumptions and notation that weuse are summarized in Appendix A. We have de-scribed this method earlier in conference5 and work-shop papers.6</p><p>2. Estimate Statistics</p><p>A. Estimate Scatter: Histogram-Based Techniques</p><p>Maximum likelihood ~ML! estimators of the Dopplerlidar frequency shift79 have the properties ofmatched filters in the spectral domain.7,8,10 As-sume, for the present, that the frequency shift F1 isthe only unknown. From the measured data set x,the algorithms generate a log likelihood that is afunction of frequency F, ln@p~x, F!#. The term p~x,F! is the probability of obtaining the data set x if F isthe Doppler shift. According to Bayes theorem, itcan also be interpreted as the probability of F beingequal to the shift, given only the data set ~i.e., withoutany a priori information concerning the value of theshift!. Thus the ML frequency estimate F1* of theDoppler frequency shift F1 is that at which the max-imum in the log likelihood occurs, so that ln@p~x,F!#max 5 ln@p~x, F1*!#.Figure 1 contains a scatterplot characteristic of fre-</p><p>quency estimates F1* and the peak log-likelihood val-ues ln@p~x, F1*!# from which they are derived in a lowsignal regime. It is clear that such estimates, dis-tributed across the entire receiver search band, mustbe evaluated to assess their reliability. The ten-dency of estimates to cluster around the true value isdescribed by the histogram of frequency estimates,which is the expected value ~EV! of the linear densityof the abscissa in Fig. 1. An example of a histogramobtained by using simulations is given in Fig. 2~a!,which is taken from an earlier paper5:</p><p>g~F1*! 5 ~1 2 a! 1a</p><p>2psgexpF2 ~F1* 2 F1!22sg2 G , (1)</p><p>where the first term represents a uniform back-ground distribution, the Gaussian function repre-sents the clustering of estimates around F1, and aand sg are the fitted parameters. Figure 2~b! showsthat the width of this Gaussian peak, sg is no greaterthan sCR, indeed it is rather lower, and that the ratiosgysCR diminishes as the signal level decreases. Ex-tensive studies of the estimate histogram for variousML estimators have been presented by Frehlich andYadlowsky9 since amplified by Frehlich.11</p><p>B. Detection-Based Techniques</p><p>The detection technique is based not on the histogrambut on the probability density function ~PDF! of the loglikelihoods, which is the EV of the linear density of theordinates in Fig. 1. It is apparent that many of theclustered estimates in Fig. 1 are characterized by log-likelihood values in excess of those of the others andcould, in principle, be distinguished from them by theuse of a threshold. In ordinary detection theory, thelog likelihood is used in this way to determine thepresence of a signal in a linear communication chan-nel. Determination of the position ~in time or, forradar, in range! of the return signal from a target is,however, a nonlinear problem12 because the measure-ment equation that relates the measurements to theestimate is nonlinear. This is also true in Dopplerlidar for determination of the frequency shift, which isthe position ~in frequency! of the signal. For thesenonlinear problems, a single test may not be availableto assign a detection probability for F1*. Use of theword detection in this context is perhaps undesirable,especially as in heterodyne lidar it is often alreadyused ambiguously to refer to both the photodetector~really the photomixer! and any subsequent signal rec-tifier in the receiver. We have continued with it be-cause of its links in processing theory.We begin by discussing detection proper, or the de-</p><p>cision on the presence of the signal, that necessitatesour using the complete likelihood function ~generally afunction of signal strength and width as well as of</p><p>Fig. 1. Scatterplot of 180 pairs of values of the maximum loglikelihood ln@p~xuF!#max and the frequency estimate F1*, the latterplotted relative to the receiver bandwidth f1* 5 F1*yFS ~see Ap-pendix A!. The mean spectrum is described by Eqs. ~2! and ~16!,and data samples are processed to obtain F1* by correlation withthe modified Levin filter function @Eq. ~7!#. The sample size isM 5 64, the relative frequency of the signal shift and bandwidthare, respectively, f1 5 0.2 and f2 5 F2yFS 5 0.02, and the widebandsignal-to-noise ratio d 5 210 dB ~a1 ' 1.2, Npc ' 10!.</p><p>20 March 1997 y Vol. 36, No. 9 y APPLIED OPTICS 1941</p></li><li><p>frequency shift!. The likelihood ratio, on which suchdecisions are based, is first derived in an approxima-tion ~a derivation without the approximation is givenin Appendix B!, and its application is illustrated andcompared with some simpler but less powerful alter-natives including the coherence field. Then we takeadvantage of the assumption ~see Section 1! that spu-rious estimates are noise related and that the PDF oftheir likelihoods is characterized mainly by a subset ofthe PDF for estimates obtained when the signal isabsent. When this is valid, a decision about the pres-ence of the signal becomes nearly equivalent to a de-cision as to whether the frequency estimate is one ofthose clustered about the true value.</p><p>3. Log of the Likelihood Ratio or Log-LikelihoodDifference</p><p>In the frequency domain, the signal and noise aredescribed by spectral power densities. The EV of thetotal spectral power in the ith discrete channel is the</p><p>Fig. 2. ~a! Histogram ~squares! of estimates obtained from simu-lations for parametersM 5 64, f1 5 0.0, f2 5 0.02, d 5 210 dB, anda fit ~solid curve! to these values obtained from Eq. ~1!. Thedashed curve is a Gaussian curve with a standard deviation equalto the CramerRao lower bound sCR ~Appendix A!. ~b! Parame-ters of curves fitted to histograms of the type exemplified in ~a!.The parameters are the same as for ~a! except that the effect ofincrease in the sample size is illustrated by an accumulation ofreturns from n pulses at constant M. The squares give the ratiosgysCR and the curve is the fitted value of a @Eq. ~1!#.</p><p>1942 APPLIED OPTICS y Vol. 36, No. 9 y 20 March 1997sum of the EVs of the signal and the noise powers:</p><p>fi 5 fi~S! 1 fi</p><p>~N!. (2)</p><p>Binary decision theory12 is based on the selection orrejection of a hypothesis H0. Here the hypothesis isthat only noise is present, and we write H0:f</p><p>~S! 5 0.Rejection of H0 is regarded as acceptance of its alter-native, which is written H1:f</p><p>~S! . 0. Decisions onwhether to accept H0 or H1 for a given data sampleare based on the value of the likelihood ratio:</p><p>L~x! 5@p1~xuH1!#max@p0~xuH0!#max</p><p>, (3)</p><p>where p1 is the likelihood function obtained by ourprocessing the data set x and assuming that H1 andp0 are the likelihood obtained assuming H0. Un-known parameters within p0 and p1 are assumed tobe set to the values that maximize p0 and p1, i.e., theyare ML estimates. In the frequency domain thedata set x is the sample spectral density.Under H0, the spectrum is that of noise. If the</p><p>noise is white, fi~N! 5 f~N! and fluctuations in the</p><p>spectral components xi are uncorrelated. If we as-sume an accumulation of n spectra and normalize sothat EV~xi! 5 f</p><p>~N!, the statistics of the spectral com-ponents are chi-square1:</p><p>p0~xuH0! 5 )i50</p><p>M21 S nf ~N!Dn xi</p><p>n21 exp@2nxiyf ~N!#G~n!</p><p>, (4)</p><p>where G~n! 5 ~n 2 1!! An M point power spectrum,derived from a data time series containingM complexdata points, is assumed throughout ~Appendix A!.The log likelihood can be written as</p><p>ln@p0~xuH0!# 5 q~x! 1 r@x; f ~N!#,</p><p>q~x! 5 M@n ln~n! 2 lnG~n!#</p><p>1 ~n 2 1! (i50</p><p>M21</p><p>ln~xi!,</p><p>r~x; f ~N!! 5 2n (i50</p><p>M21</p><p>$ln@f ~N!# 1 @xiyf ~N!#%. (5)</p><p>The ML estimate of f~N! could be obtained empiri-cally through iteration by finding the value of f~N!</p><p>that maximizes r~x; f~N!!. However, we can easilyshow by differentiating r~x; f~N!! with respect to f~N!that the ML estimate is simply the average of the xi,as would be expected.For the calculation under H1 with signal present,</p><p>we use Levins approximation13 that the spectralcomponents remain uncorrelated. Use of theBrovkoZrnic likelihood function79 derived by ourassuming realistic statistics for the time domain datais better at high signal levels,9 and a derivation of thelikelihood ratio with thismodel, which is analogous towhat follows here, is given in Appendix B. Levinsapproximation is simpler and leads to an analyticexpression for the CRLB ~Appendix A! and to satis-factory estimates in the weak signal and white noise</p></li><li><p>regime ~where the spectrally uncorrelated noise dom-inates the signal!, which is the regime of prime in-terest. The expression for the likelihood function isidentical with that in Eq. ~4! when f~N! is replaced byfi. It can be written as</p><p>ln@p1~xuH1!# 5 q~x! 2 n (i50</p><p>M21</p><p>@ln~fi! 1 ~xiyfi!#,</p><p>5 q~x! 1 r@x; f ~N!# 1 ln$L@x, f ~N!, f ~S!#%,(6)</p><p>where the log of the likelihood ratio @Eq. ~3!#, or thelog-likelihood difference ~LLD!, which is the functionwe seek, is</p><p>ln$L@x, f ~N!, f ~S!#% 5 n (i50</p><p>M21 F xif ~N! fi~S!fi 2 lnS fif ~N!DG . (7)ML estimates of the parameters of fi can be foundeither from Eq. ~6! by maximizing the expression r@x,f# 5 2n i ln@fi# 2 n i xiyfi, in which the last termis the correlation of the data xi with Levins matchedfilter function 21yfi, or from Eq. ~7!, in which thefirst term on the right-hand side is the correlation ofthe data ~normalized to fN! with the modified Levinfilter function 1fi</p><p>~S!yfi. Eq. ~7! provides the LLDdirectly and was used to construct Fig. 1.Although the spectral density of the noise f~N! can</p><p>usually be assumed to be known, this is not the casefor fi</p><p>~S!. The unknown spectrum is usually replacedwith its ML estimate fi</p><p>~S!* and the LLD with thegeneralized log-likelihood difference ~gLLD! based onthese estimates:</p><p>ln$Lg@x, f ~N!, f ~S!*#% 5 n (i50</p><p>M21 H xif ~N! fi~S!*fi* 2 lnFfi*f ~N!GJ .(8)</p><p>In the examples that follow, we assume the signalbandwidth F2 to be known when we determine fi</p><p>~S!*</p><p>and the gLLD on the basis that the signal bandwidthcan be dominated in lidar by the pulse and not theatmospheric return. This leaves two unknowns, theshift F1 and a...</p></li></ul>


View more >