negative power law noise, reality vs myth

Negative Power Law Noise,Negative Power Law Noise, Reality vs Myth Reality vs Myth

Victor S. ReinhardtVictor S. ReinhardtRaytheon Space and Airborne SystemsRaytheon Space and Airborne Systems

El Segundo, CA, USAEl Segundo, CA, USA

Precise Time and Time IntervalPrecise Time and Time IntervalSystems and Applications MeetingSystems and Applications Meeting

20092009

PTTI 2009 -- V. S. Reinhardt

Negative Power Law (Neg-p) Noise,Reality vs Myth• Not questioning the Not questioning the realityreality that that -variances-variances

Like Allan and Hadamard variancesLike Allan and Hadamard variancesAre convergent measures of neg-p noiseAre convergent measures of neg-p noise

• But will show it is But will show it is mythmyth that neg-p divergences that neg-p divergences in other variances like standard & N-sample in other variances like standard & N-sample Are mathematical defects in these variances Are mathematical defects in these variances That should be “fixed” by replacing them with That should be “fixed” by replacing them with -variances without further action-variances without further action

• Will show each type of variance is a statistical Will show each type of variance is a statistical answer to a different error question answer to a different error question And variance divergences are real indicators of And variance divergences are real indicators of

physical problems that can’t be glossed over physical problems that can’t be glossed over by swapping variancesby swapping variances

PSD LP(f) f p for p < 0


Negative Power Law Noise,Reality vs Myth• Will also show it is Will also show it is mythmyth that one can properly that one can properly

separate polynomial deterministic behavior & separate polynomial deterministic behavior & (non-highpass filtered) neg-p noise(non-highpass filtered) neg-p noiseUsing least squares & Kalman filtersUsing least squares & Kalman filters

Except under certain conditionsExcept under certain conditions• Will show the Will show the realityreality is that such neg-p noise is is that such neg-p noise is

infinitely correlated & infinitely correlated & non-ergodicnon-ergodicEnsemble averages Ensemble averages time averages time averages

And this makes neg-p noise act more like And this makes neg-p noise act more like systematic error than conventional noise in systematic error than conventional noise in statistical estimationstatistical estimation


tData Collection Interval T

= xc(tn) xr(t) = Noise + xr(tn)

xc(t) = True orDeterministic Behavior

x(tn) N Samples of Measured Data

Simple Statistical Estimation Model

Myth 1: Can “Fix” Variance Divergences Just by Swapping Variances

• Using a technique like a least squares or Kalman Using a technique like a least squares or Kalman filterfilter

• Note x(t) here is Note x(t) here is anyany data variable data variableNot necessarily the time errorNot necessarily the time error

Generate xa,M(t) an M-Parameter Est of xc(t)


tTT

Basic Error Measures and the Questions They Address

• Can form variances from these error measuresCan form variances from these error measuresPoint Variance: Point Variance: (t)(t)22 = = EE xx(t)(t)22 (Kalman) (Kalman)EE .. = Ensemble average .. = Ensemble average Assumes Assumes EE xx(t) = 0(t) = 0

Average Variance: Average Variance: 22 = a weighted average of = a weighted average of

(t)(t)22 over T (LSQF) over T (LSQF)

Data Precision xj,M(tn): Data variation from fit?

xj,M(t)

Accuracy xw.M(tn): Error of fit from true behavior?

xw,M(t)

Mth Order -Measures ()Mx(t): Stability over ?

()2x(t)()x(t+)()x(t)=x(t+)-x(t)


Interpreting -Measures as Mth Order Stability Measures

• Definition of MDefinition of Mthth Order Stability Order StabilityExtrapolationExtrapolation or or interpolationinterpolation error error xxj,Mj,M(t(tmm)) at an at an

(M+1)(M+1)thth point pointAfter a removing a perfect After a removing a perfect M-parameter fitM-parameter fit over over

only Monly M of those points of those points • Can show when Can show when xxa,Ma,M(t) = (M-1)(t) = (M-1)thth order polynomial order polynomial

& points separated by & points separated by xj,M(tm) ()Mx(t0) • Thus Thus -variances are statistical measures of -variances are statistical measures of

such stabilitysuch stability

xj,M(tm)xa,M(t)

Passes thru t0…tMbut not tm

xj,M(tM) xa,M(t)

Passes thru t0…tM-1

but not tM

x(tm)

Extrapolation Error Interpolation Error


A Derived Error Measure: The Fit Precision (Error Bars)

• Fit Precision: A statistical estimate of accuracy Fit Precision: A statistical estimate of accuracy based on based on measuredmeasured data precision variance and data precision variance and correction factors based on a correction factors based on a specificspecific theoreticaltheoretical noise modelnoise model

wj,M(t)2 = d(t) j,M(t)2 wj,M2 = d j,M

2

• For uncorrelated noise & an unweighted linear For uncorrelated noise & an unweighted linear least squares fit (LSQF) least squares fit (LSQF) d = M/(N-M)Not true for correlated noiseNot true for correlated noise

txw,M(t)

xj,M(t)2wj,M(t) or 2wj,M


The Neg-p Convergence Properties of Variances for Polynomial xa,M(t)

• For For -variances-variancesK(f) K(f) ff 2M2M (|f|<<1)(|f|<<1)

• Also true for Also true for MMthth order orderdata precisiondata precision

• Both converge for Both converge for 2M 2M -p -p• But for accuracy But for accuracy K(f)K(f)

is a lowpass filteris a lowpass filterAccuracy won’t converge unless Accuracy won’t converge unless |H|Hss(f)|(f)|22

provides sufficient HP filteringprovides sufficient HP filtering• Thus the temptation to “fix” a neg-p variance Thus the temptation to “fix” a neg-p variance

divergence by swapping variancesdivergence by swapping variances

K(f)|(f)H|(f)Ldfσ 2sp

2 ςSystem

ResponseVariance Kernel

PSD

-2 -1 0 1 2-150

-100

-50

0

1Log(fT)

M=5

f

10

M=4

f

8

M=3

f

6M=2 f

4M=1 f 2

dB

K(f) for Data Precision

1-M

0m

mmMa, ta(t)x


This Is No “Fix” Because Each Variance Addresses a Different Error Question

• Arbitrarily swapping variances just misleadingly Arbitrarily swapping variances just misleadingly changes the question changes the question

• Does not remove the divergence problem for the Does not remove the divergence problem for the original questionoriginal question

tw,M(t)

j,M(t)2wj,M(t),M(t)

Accuracy: Error of fit from true behavior?Data Precision: Data variation from fit?Fit Precision: Est of accuracy from data/model?Stability: Extrap/Interp error over ?


T

2w,12j,1

x(t) = f -2 Ensemble Membersxj,1

t0

Start Noise Here

Then What Does a Neg-p Variance Divergence Mean?

• Accuracy variance Accuracy variance as as tt00 while precision while precision remains remains misleadingly finitemisleadingly finite

• Thus an accuracy variance infinity is a real Thus an accuracy variance infinity is a real indicator of a severe modeling problemindicator of a severe modeling problemTo truly fix To truly fix Must modify the system design Must modify the system design

or the error question being askedor the error question being asked• Note for Note for ff -3 -3 noise noise j,1j,1 but but j,2j,2 remains finite remains finite• 1/f noise is a marginal case 1/f noise is a marginal case w,Mw,M

22 ln(f ln(fhhtt00))1/f contribution can be small even for t1/f contribution can be small even for t00 = age = age

of universeof universe

as t0


Myth 2: Can Use Least Squares & Kalman Filters to Properly Separate• True polynomial behavior from data containing True polynomial behavior from data containing

(non-HP filtered) neg-p noise (non-HP filtered) neg-p noise • Will show that such noise acts like systematic Will show that such noise acts like systematic

error in statistical estimation error in statistical estimation Which generates anomalous fit resultsWhich generates anomalous fit resultsExcept under certain conditionsExcept under certain conditions

• Will show this occurs Will show this occurs Because (non-HP filtered) neg-p noise is Because (non-HP filtered) neg-p noise is

infinitely correlated & infinitely correlated & non-ergodicnon-ergodic


NS & WSS Pictures of a Random Process xp(t)

• xxpp(t) starts at finite time(t) starts at finite time• Can have RCan have Rpp as t as tgg • Wigner-Ville functionWigner-Ville function

Is a tIs a tgg dependent PSD dependent PSD

• Two important NS Two important NS WSS WSS Theorems Theorems

• xxpp(t) active for all t(t) active for all t• xxpp(t) must have a (t) must have a boundedbounded

steady state steady state • PSD PSD

/2)(tx/2)(tx),(tR gpgpgp τττ E )(R)(tR pgp ττ ,

= Time Differencetg = Time from xp startE = Ensemble Average

),(tRLim)(R gpt

pg

ττ

),(tR f),(tW gpf,gp ττF

(Complex)Fourier Transform

τf,F

)(R (f)L pf,p ττF

f),(tWLim(f)L gpt

pg

NS Picturexp(t) = 0 t<0

0tg

WSS Picturexp(t) 0 all t


The Properties of (Non-Highpass Filtered) Neg-p Noise• Rp(tg,) is finite for finite tg • Rp(tg,) as tg for all

• WSS WSS RRpp(() is infinite) is infinite for all for all

• Can defineCan define L Lpp(f)(f) without Rwithout Rpp(())

• Its correlation Its correlation time time cc is infinite is infinite

• It is inherently It is inherently non-ergodic non-ergodic Theorem (Parzan):Theorem (Parzan):

A NS random processA NS random processis ergodic if and only if is ergodic if and only if

f),(tWLim(f)L gpt

pg

Neg-p Noise

tg t=0

Unbounded as tg

Rp(tg,) < as tg and c <

),R(td,0)0.5R(tLim g

1-g

tc

g

τττ

E .. <..>T

),(tRLim)(R gpt

pg

ττ


Ergodicity and Practical Fitting Behavior• Fitting Theory is Based on Fitting Theory is Based on EE• Practical fits rely on Practical fits rely on <..><..>TT • So noise must be So noise must be ergodic-likeergodic-like

over T over T <..>T E for a practical fit for a practical fitto work as theoretically predictedto work as theoretically predicted

• Even if noise is Even if noise is strictly ergodicstrictly ergodic<..>T = E May not have May not have <..>T E for the for the TT in question in question

• Most theory assumes Most theory assumes <..>any T = E for for N N CalledCalled local ergodicity local ergodicity for for TT00

• For correlated noise For correlated noise T >> c is also required is also requiredfor ergodic-like behavior (for ergodic-like behavior (intermediate ergodicityintermediate ergodicity))

Ensemble Av E

Time Av <..>T T


• A single neg-p ensemble memberA single neg-p ensemble memberhas has polynomial-likepolynomial-like behavior over behavior over TTxxpp(t)(t) is is systematicsystematic withwith

polynomial-like xpolynomial-like xcc(t)(t)Even an augmented Kalman filter can only Even an augmented Kalman filter can only

separate non-systematic parts of xseparate non-systematic parts of xpp(t) & x(t) & xcc(t)(t)• Fitting methodologies can only separateFitting methodologies can only separate

linearly independentlinearly independent variables variables• Cannot truly separate (non-HP filtered) neg-p Cannot truly separate (non-HP filtered) neg-p

noise and poly-like deterministic behaviornoise and poly-like deterministic behaviorNoise whitening cannot be used in such casesNoise whitening cannot be used in such cases

f -2 Noise

f 0 Noise

Kalman Filter

f 0 Noise

LSQF

Practical Examples of Neg-p Non-ergodicity Fitting Effects

long term error-1.xls

AugmentedKalman

f -2 Noise

StandardKalman

f -3 Noise

xa,M ± wj,M

–x --xc


Non-Ergodic-Like Behavior in Correlated WSS Processes

• T/T/cc = N = Nii Number of independent samples Number of independent samplesMust have NMust have Nii >> 1 for fit to be meaningful >> 1 for fit to be meaningful

• For non-HP filtered neg-p noise For non-HP filtered neg-p noise cc = = No No TT will produce ergodic-like fit behavior will produce ergodic-like fit behavior

- -True behavior

Kalman filter outputMeas dataFit precision

T

T/c=2000 T/c=200 T/c=20 T/c=2


Conclusions

• Arbitrarily swapping variances does not really fix Arbitrarily swapping variances does not really fix a neg-p divergence problema neg-p divergence problemSuch divergences indicate real problems that Such divergences indicate real problems that

must be physically addressedmust be physically addressed• Non-HP filtered neg-p noise acts like more like Non-HP filtered neg-p noise acts like more like

systematic error than conventional noisesystematic error than conventional noiseWhen fitting to (full order) polynomials When fitting to (full order) polynomials The surest way to reduce such problems is to The surest way to reduce such problems is to

develop freq standards with lower neg-p noisedevelop freq standards with lower neg-p noiseGood news for frequency standards developersGood news for frequency standards developers

• See: See: http://http://www.ttcla.org/vsreinhardtwww.ttcla.org/vsreinhardt// for for preprints & other related materialpreprints & other related material

negative power law noise, reality vs myth

Documents