how extracting information from data highpass filters its additive noise victor s. reinhardt...

How Extracting Information from How Extracting Information from Data Highpass Filters its Additive Data Highpass Filters its Additive

NoiseNoise

Victor S. ReinhardtVictor S. ReinhardtRaytheon Space and Airborne SystemsRaytheon Space and Airborne Systems

El Segundo, CA, USAEl Segundo, CA, USA

PTTI 2007PTTI 2007

Thirty-Ninth Annual Precise Time and Time Interval (PTTI) Thirty-Ninth Annual Precise Time and Time Interval (PTTI) Systems and Applications Meeting Systems and Applications Meeting

November 26 - 29, 2007November 26 - 29, 2007Long Beach, California Long Beach, California

PTTI 2007 -- V. Reinhardt

Various Measures of Random Error Are Used Across EE Community

• Mean Square (Observable)Mean Square (Observable)Residual ErrorResidual ErrorAfter removing a modelAfter removing a model

function estimate of truefunction estimate of truecausal behavior of datacausal behavior of data

• Jitter (and Wander)Jitter (and Wander)Residual errors + Residual errors +

HP & LP filteringHP & LP filtering

• Difference (Difference () Variances) VariancesAllan Allan Mean sq of Mean sq of (()y(t))y(t)Hadamard or PicinbonoHadamard or Picinbono Mean sq of Mean sq of (())22y(t)y(t)

ffc

JitterHP Filter

WanderLP Filter

t● ● ● ●

()v(tn) = v(tn+) - v(tn)

●

t● ●

●

●●

ResError

●

● ●●

CausalEstimate

v(t)v(t)x(t),y(t),x(t),y(t),(t)(t)


Some Common Wisdoms About These Error Measures

• Residual error diverges in presence of Residual error diverges in presence of negative power law (neg-p) noisenegative power law (neg-p) noisePower law noise Power law noise PSD L PSD Lvv(f) (f) f f p p Neg-p Neg-p p < 0 (-1,-2,-3,-4) p < 0 (-1,-2,-3,-4) White noise White noise p = 0 p = 0Jitter & Jitter & -variance used to correct this problem-variance used to correct this problem

-variance doesn’t measure same thing as -variance doesn’t measure same thing as MS residual errorMS residual error““Need ‘real’ variance not Allan Variance”Need ‘real’ variance not Allan Variance”

• Will show common wisdoms are not trueWill show common wisdoms are not trueAnd all 3 essentially measure same thing for And all 3 essentially measure same thing for

polynomial models of causal behaviorpolynomial models of causal behavior-Variances -Variances Residual Error Residual Error Jitter Jitter-Variances -Variances Residual Error Residual Error Jitter Jitter


Will Demonstrate This by Showing

• The estimation of causal behavior from The estimation of causal behavior from data HP filters noise in residual errordata HP filters noise in residual errorRes error in general converges for neg-p noiseRes error in general converges for neg-p noiseRes error Res error guaranteedguaranteed to converge if free to to converge if free to

choose model for causal behaviorchoose model for causal behavior

• Can define jitter simply as observable Can define jitter simply as observable residual errorresidual error

-variances are measures of residual -variances are measures of residual errorerrorFor any # of samplesFor any # of samplesWhen causal model is a polynomialWhen causal model is a polynomial-Variances -Variances Residual Error Residual Error Jitter Jitter


Residual Error

• Consider N data samples of data over Consider N data samples of data over observation interval Tobservation interval T

• Data v(tData v(tnn) will be modeled as ) will be modeled as True causal function = vTrue causal function = vcc(t(tnn) … plus) … plusTrue noise = vTrue noise = vpp(t(tnn) with L) with Lvv(f) (f) f f p p vvpp(t(tnn) also true residual error ) also true residual error But not measurable from data over TBut not measurable from data over T

t●

●

● ●

●

●

●

●

●

●True Causal Function vc(t)

v(tn)

N Data Samples over T = N∙o

True Noisevp(tn)

Also True Residual

Error


• Can estimate residual error by fitting Can estimate residual error by fitting model function vmodel function vw,Mw,M(t,A) to data(t,A) to dataA = (aA = (aoo,a,a11,…a,…aM-1M-1) ) M adjustable parameters M adjustable parametersA A Information extracted from data Information extracted from data

Class of vClass of vw,Mw,M(t,A) (t,A) (M-1) (M-1)thth order polynomials order polynomials

• Will use Least SQ Fit to estimate vWill use Least SQ Fit to estimate vcc(t(tnn) ) LSQF equivalent to many other methodsLSQF equivalent to many other methods

Residual Error

t●

●

● ●

●

●

●

●

●

●vc(t)

Model Fn Est

vw,M(t,A)

v(tn)



• Observable residual error Observable residual error data – est fn data – est fnvvjj(t(tnn) = v(t) = v(tnn) - v) - vw,Mw,M(t(tnn,A) ,A) Jitter Jitter v vjj(t(tnn) with no ad hoc filtering) with no ad hoc filtering

• True function error True function error est – true functions est – true functionsvvww(t(tnn) = v) = vw,Mw,M(t(tnn,A) – v,A) – vcc(t(tnn))vvww(t(tnn) ) wander (Not observable from data wander (Not observable from data over T alone)over T alone)

Residual Error

t●

●

● ●

●

●

●

●

●

●vc(t) Observable

Res Error vj(tn) True Function

Error vw(t)

Model Fn Est

vw,M(t,A)

v(tn)


v-jv-j22 = MS of v = MS of vjj(t(tnn))

v-wv-w22 = MS of v = MS of vww(t(tnn))


• Directly relates lower level error to primary Directly relates lower level error to primary performance measures in many systemsperformance measures in many systemsSNR SNR BER BER MNR MNR NPR NPR ENOBENOB

• Must use observable residual error for Must use observable residual error for verification with data over Tverification with data over TFor true errors (over T) must measure over >> T For true errors (over T) must measure over >> T

for neg-p noise (exception to be discussed)for neg-p noise (exception to be discussed)

Why Residual Error is Important Measure

t●

●

● ●

●

●

●

●

●

●vc(t) Observable

Res Error vj(tn) True Function

Error vw(t)

v(tn)


Model Fn Est

vw,M(t,A)


HP Filtering of Noise Due toInformation Extraction

• Proved in paperProved in paperWill explain graphicallyWill explain graphically

as followsas follows

• For white noise LSQF For white noise LSQF behaves in classical mannerbehaves in classical mannerv-wv-w 0 as N 0 as N Note T fixed as N variesNote T fixed as N varies

• But for neg-p noise But for neg-p noise v-wv-w notnot 0 as N 0 as N Because fitted vBecause fitted vw,Mw,M(t,A) tracks (t,A) tracks highly correlated LF noise highly correlated LF noise components in datacomponents in data

long term error-1.xls

T

LSQF for Various p

f 0 Noise

v(tn)

f -2 Noise

vw

vc f -4 Noise

va,Mvj

f 0 Noise

v(tn)

vc f -4 Noise

va,M

f -2 Noise

vw

vj


HP Filtering of Noise Due toInformation Extraction

• Happens because LSQF Happens because LSQF can’t separate highly can’t separate highly correlated LF noisecorrelated LF noiseWith Fourier freqs f With Fourier freqs f ≤≤ 1/T 1/TFrom the causal behaviorFrom the causal behavior

• True for True for allall noise noiseImplicit in LSQF theoryImplicit in LSQF theoryOnly apparent for neg-p noise Only apparent for neg-p noise

because most power in f because most power in f ≤≤ 1/T 1/T

• This tracking causes HP This tracking causes HP filtering of noise in vfiltering of noise in vjj & & v-jv-j

long term error-1.xls

T

LSQF for Various p

f 0 Noise

v(tn)

f -2 Noise

vw

vc f -4 Noise

va,Mvj

vc f -4 Noise

va,M

f -2 Noise

vw

vj


HP Filtering of Noise in Spectral Integral Representation of v-j

2

• HHss(f) = System response function(f) = System response functionDescribed in Reinhardt, FCS 2006Described in Reinhardt, FCS 2006|H|Hss(f)|(f)|22 is used to replace upper cut-off freq f is used to replace upper cut-off freq fhh

Can show HCan show Hss(f) often HP filters L(f) often HP filters Lvv(f)(f)(as well as LP filters) (as well as LP filters) Helps Helps v-jv-j

22 converge converge

2c-vj-v

0

2sv

2j-v σdf(f)K|(f)H|(f)L2σ

|Hs(f)|2

replaces fh

|Hs(f)|2 replaces fh

Example of HExample of Hss(f)(f)

x

~ D d(t) (t-d)

|Hs(f)|2 f2 (f<<1)

Effect of Delay & Mix on System Effect of Delay & Mix on System



2

• KKv-jv-j(f) spectral kernel also HP filters noise(f) spectral kernel also HP filters noisePaper proves KPaper proves Kv-jv-j(f) (f) f f 2M 2M (f<<1) (f<<1)When vWhen va,Ma,M(t,A) is (M-1)(t,A) is (M-1)thth order polynomial order polynomial

KKv-jv-j(f) at least (f) at least f f 2 2 (f<<1) (f<<1)For any vFor any va,Ma,M(t,A) with DC component(t,A) with DC component

• Thus convergence of Thus convergence of v-jv-j22 depends on depends on

complexity of model function vcomplexity of model function va,Ma,M(t,A)(t,A)

• Convergence Convergence GuaranteedGuaranteed if free to choose if free to choose model functionmodel function

2c-vj-v

0

2sv




2

v-cv-c22 = Contribution due to model error = Contribution due to model error

Model error occurs when model function Model error occurs when model function cannot follow variations in vcannot follow variations in vcc(t) over T(t) over T

If model error present If model error present v-jv-j22 & & v-wv-w

22 will increase will increase

• Thus if model function can’t track causal Thus if model function can’t track causal behavior over Tbehavior over T

• The causal behavior will contaminateThe causal behavior will contaminatev-jv-j

22 (& (& v-wv-w22))

2c-vj-v

0

2sv



-3 -2 -1 0 1-200

-150

-100

-50

0

M=1

M=2

M=3 M=4 M=5

log10

(fT)

Kr-

av,

M(f

) in

dB

ao

ao + a1t

Simulation Verifying Kv-j(f) f 2M (f<<1) for Polynomial va,M(t,A)

SimulationResults

M < 1E-3

f 2

f 4

f 6 f

8 f 10K

v-j(f

) in

dB

Log10(fT)(N=1000)

HP Knee

fT 1/T

HP Knee

fT 1/T

(Unweighted LSQF)


Jitter and Wander

• Defined by ITU, IEEE BTS, SMPTEDefined by ITU, IEEE BTS, SMPTEFiltered residual x-error Filtered residual x-error

after removing causal (timeafter removing causal (time&) freq offset & freq drift&) freq offset & freq driftJitter Jitter HP filtered (> f HP filtered (> fcc))Wander Wander LP filtered (< f LP filtered (< fcc))

• Problem relating fProblem relating fcc to user system params to user system paramsITU ITU f fcc = 10 Hz = 10 Hz Standardizes HW producersStandardizes HW producersBut not related to parameters in user systemsBut not related to parameters in user systems

IEEE BTS, SMPTE IEEE BTS, SMPTE f fcc = PLL BW in system = PLL BW in systemWhat about systems without PLLs?What about systems without PLLs?

ffc

JitterHP Filter

WanderLP Filter


v(tv(tnn)) vvw,Mw,M(t(tnn,A),A)vvjj(t(tnn))

HP Filtering of Residual Error Resolves fc Relationship Problem

• Jitter Jitter Observable residual error Observable residual errorvvjj(t(tnn) = v(t) = v(tnn) - v) - vw,Mw,M(t(tnn,A),A)

• Wander Wander True causal function error True causal function errorvvww(t(tnn) = v) = vw,Mw,M(t(tnn,A) – v,A) – vcc(t(tnn) )

• Have property Have property vvjj(t(tnn) + v) + vww(t(tnn) = v) = vpp(t(tnn))

• These definitionsThese definitions apply to any type of apply to any type of causal function removal & any variablecausal function removal & any variable

• HP & LP properties generated by system HP & LP properties generated by system

vvcc(t)(t) vvww(t)(t)

vvpp(t)(t)


− Wander

− Jitter K(f)

ffTT

HP Filtering of Residual Error Resolves fc Relationship Problem

• LSQF HP filters vLSQF HP filters vjj & LP filters v & LP filters vww f fTT 1/T 1/T

• HHss(f) filters both the same (f) filters both the same f fll = HP f = HP fhh = LP = LP

• For as TFor as T (f (fTT << f << fll) wander shrinks to zero) wander shrinks to zeroIf HIf Hss(f) alone can overcome pole in L(f) alone can overcome pole in Lvv(f)(f)So wander can also converge for neg-p noise So wander can also converge for neg-p noise

• Jitter variance with brickwall fJitter variance with brickwall fcc & f & fhh

is just bandpass approximation of is just bandpass approximation of v-jv-j22

ff

|Hs(f)|2

ffll ffhh ff

ffTT ffll ffhh


-Variances as Measure of v-j2

When va,M(t,A) is Polynomialv,Mv,M(())22 = = MM x Mean Sq of x Mean Sq of (())MMv(tv(tnn))

(()v(t) = v(t+)v(t) = v(t+) - v(t)) - v(t)(())22v(t) = v(t+2v(t) = v(t+2) - 2v(t+) - 2v(t+) + v(t)) + v(t)MM All All v,Mv,M(())22 are equal for white noise are equal for white noise

v,Mv,M(())22 = = MMMS{MS{(())MMv(tv(tnn)})}

t●● ●

●

●● ● ● ●●

()v(tn+)

●

()v(tn)

()2v(tn)

●

MS over N samples in T MS over N samples in T


-Variances as Measure of v-j2

When va,M(t,A) is Polynomial• Paper shows Paper shows

For For v-jv-j22 “unbiased” mean square “unbiased” mean square

““Unbiased” Unbiased” Divide sum of squares by N - M Divide sum of squares by N - M

• Well-known for Allan (2-sample) varianceWell-known for Allan (2-sample) varianceIs 2-sample MS residual of y(t) with 0Is 2-sample MS residual of y(t) with 0thth order order

polynomial (freq offset) removed polynomial (freq offset) removed

• Hadamard-Picinbono variance is 3-sample Hadamard-Picinbono variance is 3-sample residual variance of y(t) for M = 2residual variance of y(t) for M = 211stst order polynomial (freq offs & drift) removed order polynomial (freq offs & drift) removed

v-jv-j22(N=M+1) = (N=M+1) = v,Mv,M(())22 when when = T/M = T/M


Can Extend Equivalence to Any N

• Can show biased RMS{vCan show biased RMS{vjj} } doesn’t vary much with Ndoesn’t vary much with NBiased Biased Divide sum sq by N Divide sum sq by NNote T fixed as N variedNote T fixed as N varied

• Thus for any N can writeThus for any N can write“unbiased” “unbiased” v-jv-j

22 as as

Approx true for any pApprox true for any pCan generate exact Can generate exact

relationship for each p like relationship for each p like Allan-Barnes bias functionsAllan-Barnes bias functions

v-jv-j22(N) (N) v,Mv,M(T/M)(T/M)22

N-MN-MNN

Errors vs N(T Fixed, M=2)

RMS{vj}f

0 Noise2

10

v-w 1 10 100 1K

Samples N 2

10

f -2 Noise v-w

RMS{vj}

N = M+10

2

1

f -4 Noise v-w

RMS{vj}


Consequences of v,M() as Approximate Measure of v-j

• Justifies using Justifies using v,Mv,M(T/M) in residual error (T/M) in residual error problemsproblemsWhen vWhen vw,Mw,M(t,A) is (M-1)(t,A) is (M-1)thth order polynomial order polynomial

• Don’t have to perform LSQF on data if use Don’t have to perform LSQF on data if use v,Mv,M(() to estimate ) to estimate v-jv-j Because Because (())MM of (M-1) of (M-1)thth order polynomial = 0 order polynomial = 0 Well-known insensitivity of Well-known insensitivity of v,Mv,M(() to (M-1)) to (M-1)thth polynomial causal behaviorpolynomial causal behavior

True even if there is model error (effect on both True even if there is model error (effect on both v,Mv,M(() & ) & v-jv-j the same) the same)


Consequences of v,M() as Approximate Measure of v-j

• Provides guidance in determining order of Provides guidance in determining order of v,Mv,M(() to use as stability measure) to use as stability measure vvw,Mw,M(t,A) = Polynomial aging function(t,A) = Polynomial aging functionAging is strictly fitted over T = MAging is strictly fitted over T = MFor For decoupled from T/M decoupled from T/Mv,Mv,M(() ) v-jv-j approx true approx true

• Causal terms not modeled in vCausal terms not modeled in va,Ma,M(t,A) are (t,A) are part of instability measured by part of instability measured by v,Mv,M(())Explains sensitivity of Allan variance to freq Explains sensitivity of Allan variance to freq drift (only freq offset modeled)drift (only freq offset modeled)

& insensitivity of Hadamard variance to such & insensitivity of Hadamard variance to such drift (both freq offset & drift modeled)drift (both freq offset & drift modeled)


• ssssssss

x & y Difference Variances Interpreted as Aging Removed v-j

2

M Var of y Aging ExcludedApplication Var of x Aging Excl

Application

0 MS{y}None

Synthesizers & Rel time dist equip

MS{x}None

Abs time dist equip

1 Allan yy offset

Oscillators (y drift in instability)

TIErms2/2MS{TIE}/2

x offsetSynth & rel

time dist

2 Hadamard Picinbono

y ofs & driftOscillators (y drift not in instability)

Allan xJitter 2

x & y offsetOsc (y drift in

instab)

3x,3

2() is equivalent toMS of x-jitter with time & freq offsets & freq drift removed

Hadamard Picinbono

x,y ofs y driftOsc (y drift

not in instab)



Application

0 MS{y}None


MS{x}None

Abs time dist equip

1 Allan yy offset


TIErms2/2MS{TIE}/2

x offsetSynth & rel

time dist



Allan xJitter 2


instab)

3 x,3


Hadamard Picinbono


not in instab)


2

• Explains why low order Explains why low order v,Mv,M(() used for ) used for synthesizers & distribution equipmentsynthesizers & distribution equipmentUncontrolled (but fixed) x & y offsets are part Uncontrolled (but fixed) x & y offsets are part

of “random” errorof “random” errorNot considered part of “random” error for Not considered part of “random” error for

oscillatorsoscillators



Application

0 MS{y}None


MS{x}None

Abs time dist equip

1 Allan yy offset


TIErms2/2MS{TIE}/2

x offsetSynth & rel

time dist



Allan xJitter 2


instab)

3x,3


Hadamard Picinbono


not in instab)


2

• Hadamard-Picinbono variance or Hadamard-Picinbono variance or x,3x,322(())

equivalent to MS of x-jitterequivalent to MS of x-jitter with time & with time & freq offsets & freq drift removed freq offsets & freq drift removed


What to Do When v-j Does Diverges

v-jv-j or or v,Mv,M(() can diverge for neg-p noise) can diverge for neg-p noiseWhen vWhen vw,Mw,M(t,A) is fixed by system spec or (t,A) is fixed by system spec or

physical problem being addressedphysical problem being addressedHas been considered mathematical nuisance Has been considered mathematical nuisance to be heuristically patchedto be heuristically patched

• Such a divergence indicates Such a divergence indicates realreal problem problemIn system design, specification, or analysis In system design, specification, or analysis vvw,Mw,M(t,A) fixed in spec for a reason(t,A) fixed in spec for a reasonNot free to change w/o changing problemNot free to change w/o changing problem

• Thus divergence is diagnostic indication Thus divergence is diagnostic indication of a system problem to be fixedof a system problem to be fixed


Divergence Example: 1st Order PLL in Presence of f -3 Noise

• Well-known that 1Well-known that 1stst order PLL will cycle order PLL will cycle slip when fslip when f -3 -3 noise is present noise is presentIndicated by divergence in Indicated by divergence in -j-j for M=0 for M=0Changing to M > 0 Changing to M > 0 -j-j eliminates divergence eliminates divergence

• But this will not stop the cycle slippingBut this will not stop the cycle slipping d

et

Cycle Slips~

Phase Lock Loopx

ref

out

~ VCOLoop Filt

det

= out - ref

RefOsc


Can Only be Fixed by Changing System Design or Spec

• Fix design Fix design Eliminate slips Eliminate slips By changing design to 2By changing design to 2ndnd order PLL order PLL

• Fix spec Fix spec Tolerate cycle slips but must Tolerate cycle slips but must change change -j -j spec to exclude cycle slipped spec to exclude cycle slipped data data Effectively changes KEffectively changes Kv-jv-j(f) (f) Should also specify mean time to cycle slipShould also specify mean time to cycle slip

• Non-essential divergencesNon-essential divergencesSystem OK but wrong HSystem OK but wrong Hss(f) or K(f) or Kv-jv-j(f) due to (f) due to

faulty analysisfaulty analysisExample: Failure to recognize HP filtering of Example: Failure to recognize HP filtering of residual errorresidual error


Final Summary & Conclusions

• Jitter, residual error, and Jitter, residual error, and variances can variances can be viewed as equivalent measuresbe viewed as equivalent measuresWhen polynomial used for causal model When polynomial used for causal model

• Residual error guaranteed to converge if Residual error guaranteed to converge if free to choose model for causal functionfree to choose model for causal functionBecause order of HP filtering increases with Because order of HP filtering increases with

complexity of model function used to estimate complexity of model function used to estimate causal behaviorcausal behavior

• Residual error often converges even when Residual error often converges even when model function fixed by spec or problemmodel function fixed by spec or problemWhen doesn’t is indication of real problem in When doesn’t is indication of real problem in

design, spec, or analysis of system in questiondesign, spec, or analysis of system in question


Final Summary & Conclusions

• Jitter should be defined as residual error Jitter should be defined as residual error without ad hoc HP filteringwithout ad hoc HP filteringBecause causal extraction provides HP filteringBecause causal extraction provides HP filtering

• True causal error (wander) only accessible True causal error (wander) only accessible by determining true noise in systemby determining true noise in system

• Paper is generalization & consolidation of Paper is generalization & consolidation of previous work by many authorsprevious work by many authorsAllan, Barnes, Gagnepain, Vernotte, Greenhall, Allan, Barnes, Gagnepain, Vernotte, Greenhall,

Riley, Howe, & many othersRiley, Howe, & many others

• Preprints: Preprints: www.ttcla.org/vsreinhardtwww.ttcla.org/vsreinhardt//

how extracting information from data highpass filters its additive noise victor s. reinhardt...

Documents

error measuresresidual

residual errorres error

residual errorcan

residual errorfor

negp noiseres error

filteringtrue function

white noise p

lower level error