show by example how evaluation of data performance in general will be carried out

Rome Navigation Innovations 2/7/06

1

• Show By Example How Evaluation of Data Performance in General Will Be Carried out

• Showcase Geometry Data Alignment Techniques as A Solution to Many Problems.

Evaluation of Geometry Data Error Performance on A Geometry Car Using

Geometry Data Alignment Techniques H. James Rome

Rome Navigation Innovations,Inc27 Old County Rd, Gloucester , MA

[email protected]

Purpose of the Presentation


2

Presentation Concentrates on a case Study: Comparison of “Alternate” and Standard

Gage

• Example of Performance Analysis

• Investigate Repeatability and stability of two measures of gage.


3

What’s This Geometry Data Alignment Package?

• Lines up data from several runs to data on Reference Run. Can align data to an Accuracy ~ 1 ft

• Used For:– Trend Analysis– Repeatability and Error Analysis

• Example Follows


42425

Left Profile Lined up with GPS to about 2 meters

3.78 3.781 3.782 3.783 3.784 3.785 3.786 3.787 3.788 3.789 3.79x 104

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.61 2 3 4 5 6 7 8 9 1011121314151617181920212223

Distance along track, ftX104 ->

Pro

file

, Inc

h,->

-12 -10 -8 -6 -4 -2 0-0.8-0.7-0.6-0.5-0.4-0.3-0.2-0.1

00.1 Plot .vs.time at ft location37825

time back from reference run, months

-20 2

Pro

file

, Inc

h,->

Example …Before Data Alignment:


52526

Plots after Data Alignment up with With Package,

3.78 3.781 3.782 3.783 3.784 3.785 3.786 3.787 3.788 3.789 3.79x 104

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.61 2 3 4 5 6 7 8 9 101112131415161718192021222324

Pro

file

, Inc

h,->

Distance along track, ftX104 ->

-12 -10 -8 -6 -4 -2 0-0.8

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1 Plot .vs.time at ft location37825

time back from reference run, months

-20 2

Pro

file

, Inc

h,->

Trend Apparent


6

Use of Alignment for Error Analysis

• Approach can be used to Evaluate Repeatability Errors. – If Data is Taken Close Enough in time,

Differences in Aligned Data imply the sum of the errors in Both measurements .. examples follow


7

NOTE!

• This alignment can be carried out over 10’s or even 100’s of miles with the click of a mouse.

• Thus no need to constrain evaluations to a several thousand ft “Test Track”.

• Occasional rare events, long term error error trends, and Data Reliability can be evaluated.


8

Example Comparison of Alternate and

Standard Gage

• Both Measures were available on the Same Car

• Two Runs over the same 70 mi of track were used for the Study

• Each Is Analyzed as if the other did not exist

From an FRA Car ~ 2006


9

1 2

mrgCRFG04

1.003E51.002E51.001E51E59.99E49.98E4

0.1

0.05

0

-0.05

-0.1

-0.15

-0.2

-0.25

-0.3

-0.35

-0.4

-0.45

1 2

mrgC04

1.003E51.002E51.001E51E59.99E49.98E4

0.1

0.05

0

-0.05

-0.1

-0.15

-0.2

-0.25

-0.3

-0.35

Snippet Aligned Alternate Gage, and Standard Gage

Alt

erna

te G

age

Sta

ndar

d G

age

“Standard”GageIs Noisier!

1000 Pt mean subracted


10

1 2

mrgC04

1.003E51.002E51.001E51E59.99E49.98E4

0.1

0.05

0

-0.05

-0.1

-0.15

-0.2

-0.25

-0.3

-0.35

x X+1000

Take Difference of Two CurvesFind Root Mean Square of Difference,RMSPlot RMS vs. X

What We Do Next


11

0.5 1 1.5 2 2.5 3 3.5x 105

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

plot of 1000 pt rms

alt gage differences standard gage differences

Plot of Running 1000 Pt. RMS differences vs. Distance for Both Gages

RMS’s ~ 40 % Less for Alternate Gage

RM

S (

Uni

ts)

Record # along track


12

Note There is usually a Calibration Error in Gage

Measurement • Is the Calibration Stable During the Run?


13

1 2

mrgC04

1.01E51.005E51E59.95E49.9E49.85E4

0.3

0.25

0.2

0.15

0.1

0.05

0

-0.05

-0.1

-0.15

-0.2

-0.25

-0.3

-0.35

Sample of Gage Aligned with ( 1000 pt) Bias Removed the Bias

Distance along track,ft

Gag

e, I

nche

s


14

1 2

mrgC04

1.01E51.005E51E59.95E49.9E49.85E4

56.8

56.75

56.7

56.65

56.6

56.55

56.5

56.45

56.4

56.35

56.3

56.25

56.2

56.15

56.1

Sample of Gage Aligned gage ..NOTE here Bias is not removed!

Is this “Bias” stable?

Distance along track,ft

Gag

e, I

nche

s


15

0.5 1 1.5 2 2.5 3 3.5x 105

0.05

0.1

0.15

0.2

0.25

0.3

0.35plot of 1000 pt mean vs distance

alt gage standard gage

Plot of 1000 pt Mean difference of same Paramter: for Alternate and Standard Gages Vs. Distance

Typical Max Shift… Standard, .1” Alternative: .06”

Record # along track


16

0 0.05 0.1 0.15 0.2 0.250

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1FRACTION ACTUAL DIFFERENCES LESS THAN X, mean subtrated

alt gage standard gage

From Histogram of All Differences, Find Cumulative Distribution

Alternative Gage: 70% of error<.05’’Standard Gage: ~ 70% of errors <.08’’

Error Limits, (Units) X


170 0.05 0.1 0.15 0.2 0.250

0.05

0.1

0.15

0.2

0.25

0.3

0.35 hisotragm of 1000 pt. RMS differences, mean subtratedalt gage standard gage

Histogram of 1000 pt RMS errors For Both Gages.

Most Likely Value Alternative~.055Noise Floor

Most Likely Value Standard~.075 Noise Floor

Fra

ctio

n in

Bin

Bin Value , Linear Units

Less than ½ # Outliers!


18

0 0.05 0.1 0.15 0.2 0.25 0.30

0.02

0.04

0.06

0.08

0.1

0.12

0 0.05 0.1 0.15 0.2 0.25 0.3

0

0.05

0.1

0.15

0.2

Period ~39 units

Frequency, 1/( Unit record) Frequency, 1/ (Unit Record)

Lots of High Frequency Noise.

Power Spectra, vs Frquency from Both Gages

Alternative Gage

Standard Gage

Note error power is about Double for Standard Gage


19

Conclusions Comparing Standard and Alternative Gages

• Alternative Gage has significantly:– Lower RMS errors– Fewer large Errors– More Bias Stability– Less high Frequency Noise

• Bottom Line: From the Point of View of Repeatability,Alternative Gage is Just Better!


20

R o m e N a v i g a t i o n I n n o v a t i o n s 2 / 7 / 0 6

4 3

E x a m p l e o f S u m m a r y t o S t r i v e F o r

P a r a m t e r C h a r R M S M o s t L i k e l y 9 0 % L i m i t S / N R a t i o B i a s R M S E r r o r S / N R a t i o S p e c t r a lE r r o r 1 0 0 0 p t . R M S o f E r r o r S e n s i t i v i t y W h e n G a g e W h e n G a g e C h a r a c t .

D e v i a t i o n D e v i a t i o n> . 3 > . 3

A l t G a g e 0 . 0 6 7 0 . 0 5 5 0 . 0 9 6 . 5 2 2 0 . 0 6 0 . 0 7 6 3 2 3 . 7 2 9 2S t d . G a g e . 0 . 0 8 7 6 0 . 0 7 5 0 . 1 3 4 . 0 4 6 4 0 . 1 0 . 1 0 7 1 2 . 9 7 2 8C r o s s L e v e l , . 0 . 0 9 0 . 0 5 0 . 1 2 0 . 4R i g h t P r o f i l e . _ _L e f t P r o f i l e ,


21

NOTE!

• Most of this Quality Information could not be obtained from a short stretch of Data

• With Automated Data Alignment, No test track required. Track of Opportunity can be used

• Simply Run over same ( say 20-50 mile) length of track twice within a few days or weeks.


22

Other Uses

• Compare Results of GRMS Vehicles without having them Coordinate their Runs..and over a long distance.

• Compare Geometry measurement Equipment

• Find Fraction of time when there are data outages


23

What About Other Parameters

• The Key is the the ability to Align Massive amounts of Geometry car Data. It puts an Entirely new spin on how extensive and how inexpensive Quality Evaluation can be!

• Similar studies can be carried out on Any measurement taken on the Geometry car

• And That includes GPS

show by example how evaluation of data performance in general will be carried out

Documents