detection techniques for validating doppler estimates in heterodyne lidar

Download Detection techniques for validating Doppler estimates in heterodyne lidar

Post on 03-Oct-2016




0 download

Embed Size (px)


  • in


    tral or correlation function estimates.1,2 Thesethat degrade estimator performance lies in normalreceiver noise as is usually the case in lidar, consen-estimates are obtained by independent measure-ments of returns from the range gate within a shorttime compared with the period characteristic of grossfluctuations in the wind speed being measured. Weaddress two remaining questions here. First, howdo we know, in practice, when the accumulation hasbeen sufficient to ensure optimal performance?Second, what can be done to evaluate an estimatewhen sufficient accumulation has not or cannot beused?Somewhat similar problems have been addressed

    sus appears to offer no advantage over accumulation,1and that detection ~or discrimination! techniques ofthe type we consider in this paper improve estimateselectivity compared with consensus.5 The differ-ence is that, in radar, spurious estimates are oftenoutliers. That is, they are not consequences of theaerosol backscatter and system statistics but arisefrom clutter, real objects that are not wind borne.For example, the statistical averaging method wasintroduced recently to remove outliers caused bybackscatter from birds, which give rise to signals thatcan be distinguished from atmospheric returns be-cause their statistics are non-Gaussian. In this pa-per we continue to assume that Doppler lidars are notsusceptible to outliers and consider the problem inthe ideal limit that spurious estimates are noise re-lated and can be described by the use of the ~known!statistics of the receiver noise. Two other tech-niques used in Doppler radar remain of interest.The first is context analysis in which an estimate iscompared with others from neighboring range gatesand observation times. Different forms of this tech-nique, some using pattern recognition, are used in

    B. J. Rye is with the Environmental Technology Laboratory,Cooperative Institute for Research in Environmental Science ~Uni-versity of Colorado and National Oceanic and Atmospheric Admin-istration!, 325 Broadway, Boulder, Colorado 80303. R. M.Hardesty is with the National Oceanic and Atmospheric Adminis-tration Environmental Technology Laboratory, 325 Broadway,Boulder, Colorado 80303.Received 3 June 1996; revised manuscript received 23 Septem-

    ber 1996.0003-6935y97y091940-12$10.00y0 1997 Optical Society of AmericaDetection techniques forvalidating Doppler estimates

    Barry J. Rye and R. Michael Hardesty

    We investigate the ability of detectheterodyne lidar Doppler estimatesdistinction between estimates thatRao lower bound. The conditionaltion of a signal when none is presenof selection of a spurious estimate.detection functions other than the lcessing rejected data in the light of c 1997 Optical Society of America

    1. Introduction

    Unbiased Doppler frequency shift estimates can beobtained from pulsed heterodyne lidar systems withperformance limited essentially by a CramerRaolower bound ~CRLB!. But this is achieved only if athreshold, which is characterized mainly by the totalenergy in the signal, is surpassed. At low signallevels, it may be necessary to augment the data sam-ple from a single-range gate by accumulation of spec-1940 APPLIED OPTICS y Vol. 36, No. 9 y 20 March 1997heterodyne lidar

    n techniques based on the likelihood ratio to discriminate betweent low signal levels using examples generated by simulation. Thee regarded as acceptable and as spurious is based on the Crameralse alarm probability, which ordinarily describes recording detec-is then found to be an approximate upper bound on the probabilityhe method is superior theoretically to similar techniques based onelihood ratio. The likelihood ratio also provides a basis for repro-ntextual information provided by those estimates that are accepted.

    in Doppler radar by the introduction of the consen-sus3 and statistical averaging method.4 In eachcase, the time series data are divided into segments,and an estimate is obtained from each segment. Theconsensus technique searches for clustering withinthe set of estimates obtained; if insufficient clusteringis obtained, the data set is rejected. We have arguedearlier that, if the reason for the spurious estimates

  • radar to distinguish outliers, but we comment hereinthat detection leads naturally to a context analysismethod that might be implemented in lidar. Thesecond technique is the coherence field, or normalizedcoherent power diagnostic, in which a function of thefirst lag term in the time series autocorrelation func-tion ~ACF! ~indicating presence of a coherent signal!is recorded, and data are rejected if this function doesnot lie above a threshold value. This approach isimplemented in Doppler processors in both radarsand lidars, and although apparently undocumented,has some similarity with the method described here.Our purpose, then, is to describe and illustrate a

    straightforward method of selecting estimates at lowsignal levels that uses standard Doppler processingand is based closely on standard detection techniquesand to suggest routes whereby it might be extended.Some details of the assumptions and notation that weuse are summarized in Appendix A. We have de-scribed this method earlier in conference5 and work-shop papers.6

    2. Estimate Statistics

    A. Estimate Scatter: Histogram-Based Techniques

    Maximum likelihood ~ML! estimators of the Dopplerlidar frequency shift79 have the properties ofmatched filters in the spectral domain.7,8,10 As-sume, for the present, that the frequency shift F1 isthe only unknown. From the measured data set x,the algorithms generate a log likelihood that is afunction of frequency F, ln@p~x, F!#. The term p~x,F! is the probability of obtaining the data set x if F isthe Doppler shift. According to Bayes theorem, itcan also be interpreted as the probability of F beingequal to the shift, given only the data set ~i.e., withoutany a priori information concerning the value of theshift!. Thus the ML frequency estimate F1* of theDoppler frequency shift F1 is that at which the max-imum in the log likelihood occurs, so that ln@p~x,F!#max 5 ln@p~x, F1*!#.Figure 1 contains a scatterplot characteristic of fre-

    quency estimates F1* and the peak log-likelihood val-ues ln@p~x, F1*!# from which they are derived in a lowsignal regime. It is clear that such estimates, dis-tributed across the entire receiver search band, mustbe evaluated to assess their reliability. The ten-dency of estimates to cluster around the true value isdescribed by the histogram of frequency estimates,which is the expected value ~EV! of the linear densityof the abscissa in Fig. 1. An example of a histogramobtained by using simulations is given in Fig. 2~a!,which is taken from an earlier paper5:

    g~F1*! 5 ~1 2 a! 1a

    2psgexpF2 ~F1* 2 F1!22sg2 G , (1)

    where the first term represents a uniform back-ground distribution, the Gaussian function repre-sents the clustering of estimates around F1, and aand sg are the fitted parameters. Figure 2~b! showsthat the width of this Gaussian peak, sg is no greaterthan sCR, indeed it is rather lower, and that the ratiosgysCR diminishes as the signal level decreases. Ex-tensive studies of the estimate histogram for variousML estimators have been presented by Frehlich andYadlowsky9 since amplified by Frehlich.11

    B. Detection-Based Techniques

    The detection technique is based not on the histogrambut on the probability density function ~PDF! of the loglikelihoods, which is the EV of the linear density of theordinates in Fig. 1. It is apparent that many of theclustered estimates in Fig. 1 are characterized by log-likelihood values in excess of those of the others andcould, in principle, be distinguished from them by theuse of a threshold. In ordinary detection theory, thelog likelihood is used in this way to determine thepresence of a signal in a linear communication chan-nel. Determination of the position ~in time or, forradar, in range! of the return signal from a target is,however, a nonlinear problem12 because the measure-ment equation that relates the measurements to theestimate is nonlinear. This is also true in Dopplerlidar for determination of the frequency shift, which isthe position ~in frequency! of the signal. For thesenonlinear problems, a single test may not be availableto assign a detection probability for F1*. Use of theword detection in this context is perhaps undesirable,especially as in heterodyne lidar it is often alreadyused ambiguously to refer to both the photodetector~really the photomixer! and any subsequent signal rec-tifier in the receiver. We have continued with it be-cause of its links in processing theory.We begin by discussing detection proper, or the de-

    cision on the presence of the signal, that necessitatesour using the complete likelihood function ~generally afunction of signal strength and width as well as of

    Fig. 1. Scatterplot of 180 pairs of values of the maximum loglikelihood ln@p~xuF!#max and the frequency estimate F1*, the latterplotted relative to the receiver bandwidth f1* 5 F1*yFS ~see Ap-pendix A!. The mean spectrum is described by Eqs. ~2! and ~16!,and data samples are processed to obtain F1* by correlation withthe modified Levin filter function @Eq. ~7!#. The sample size isM 5 64, the relative frequency of the signal shift and bandwidthare, respectively, f1 5 0.2 and f2 5 F2yFS 5 0.02, and the widebandsignal-to-noise ratio d 5 210 dB ~a1 ' 1.2, Npc ' 10!.

    20 March 1997 y Vol. 36, No. 9 y APPLIED OPTICS 1941

  • frequency shift!. The likelihood ratio, on which suchdecisions are based, is first derived in an approxima-tion ~a derivation without the approximation is givenin Appendix B!, and its application is illustrated andcompared with some simpler but less powerful alter-natives including the coherence field. Then we takeadvantage of the assumption ~see Section 1! that spu-rious e