correction of flux valve–based heading for improvement of

Correction of Flux Valve–Based Heading for Improvement of AircraftWind Observations

NEIL A. JACOBS

Panasonic Avionics Corporation, Morrisville, North Carolina

DANIEL J. MULALLY AND ALAN K. ANDERSON

Panasonic Avionics Corporation, Lakewood, Colorado

(Manuscript received 8 August 2013, in final form 6 April 2014)

ABSTRACT

A method for correcting the magnetic deviation error from planes using a flux valve heading sensor is

presented. This error can significantly degrade the quality of the wind data reported from certain commercial

airlines. A database is constructed on a per-plane basis and compared to multiple model analyses and ob-

servations. A unique filtering method is applied using coefficients derived from this comparison. Three re-

gional airline fleets hosting the Tropospheric Airborne Meteorological Data Reporting (TAMDAR) sensor

were analyzed and binned by error statistics. The correction method is applied to the outliers with the largest

deviation, and the wind observational error was reduced by 22% (2.4 kt; 1 kt5 0.51m s21), 50% (8.2 kt), and

68% (20.5 kt) for each group.

1. Introduction

Commercial aircraft have been providing observations

of wind and temperature for over 50 years (Hughes and

Gedzelman 1995; Moninger et al. 2003). In 2004, a sensor

network called Tropospheric Airborne Meteorological

Data Reporting (TAMDAR)was deployed, primarily on

regional airlines, adding relative humidity, icing, and

turbulence to the typical wind and temperature obser-

vations (e.g., Daniels et al. 2006; Moninger et al. 2010;

Gao et al. 2012). A subset of these regional airlines has

less sophisticated avionics than the larger commercial

planes providing Aircraft Meteorological Data Relay

(AMDAR) data. In some cases, dramatic differences

can exist in the quality of wind data provided by these

aircraft because of the type of heading instrumentation.

Moninger et al. (2010) conducted a 3-yr study of the

impact of TAMDAR data on the forecast skill of the

Rapid Update Cycle (RUC) model. Two identical

models, with the exception of the TAMDAR data, were

verified against radiosonde observations (raobs). While

the TAMDAR data were found to provide a significant

positive impact on forecast skill, it was discovered that

the wind observational error provided by a subset of

planes was considerably larger than other TAMDAR-

reporting planes as well as standard AMDAR. The dif-

ference in wind error originated from the heading

instrumentation, which provides data used in the cal-

culation of wind speed and direction. Many turboprops,

including the Saab 340B, as well as a few regional jets

(RJs), use a magnetic flux valve heading sensor, which is

subject to magnetic deviation errors not seen in the more

advanced laser-gyroscopic heading sensors on most jets.

Here, we present a method to correct the inaccuracies

associated with flux valve heading systems described in

Moninger et al. (2010) that can significantly improve the

quality of wind measurements.

2. Background

Observations collected by a multifunction in situ at-

mospheric sensor on commercial aircraft, called the

TAMDAR sensor, contain measurements of humidity,

pressure, temperature, winds aloft, icing, and turbulence,

along with the corresponding GPS location, time, pres-

sure altitude, and geometric altitude (height above mean

sea level). After sampling, the observations are relayed

Corresponding author address: Neil A. Jacobs, Panasonic Avi-

onics Corporation, 1100 Perimeter Park Dr., Morrisville, NC

27560.

E-mail: [email protected]

AUGUST 2014 JACOBS ET AL . 1733

DOI: 10.1175/JTECH-D-13-00175.1

� 2014 American Meteorological SocietyUnauthenticated | Downloaded 04/24/22 03:14 PM UTC

mailto:[email protected]

via satellite in real time to a ground-based network op-

erations center.

The TAMDAR sensor was originally deployed by

AirDat in December 2004 on a fleet of 63 Saab 340s

operated by Mesaba Airlines in the Great Lakes region

as a part of the National Aeronautics and Space Ad-

ministration (NASA)-sponsored Great Lakes Fleet Ex-

periment (GLFE). Over the last 10 years, the equipage of

the sensors has expanded beyond the continental United

States (CONUS) to include Alaska, Hawaii, Mexico, and

Europe on more than 12 fleets as well as a few research

aircraft. In April 2013, AirDat and the TAMDAR sensor

network technology were acquired by Panasonic Avion-

ics Corporation (PAC).

TAMDAR sensors can be installed on most fixed-wing

aircraft, from large commercial airliners to small, un-

manned aerial vehicles, where they continuously transmit

atmospheric observations via a global satellite network

in real time as the aircraft climbs, cruises, and descends.

Emphasis has been placed on equipping regional carriers,

as these flights tend to (i) fly into more remote and di-

verse locations and (ii) make more frequent flights that

produce more daily vertical profiles while remaining in

the boundary layer for longer durations.

For the purposes of this study, and the correction of

the magnetic deviation heading error, we will only be

addressing those planes with the flux valve heading in-

strumentation, which represent a very small subset of

the TAMDAR fleet (,5%). However, this correction

methodology can be applied to any flux valve heading

instrumentation and is not TAMDAR specific (Mulally

and Anderson 2011).

a. Calculation of wind speed and direction

The wind velocity VW can be calculated from the

vector difference between the ground track vector and

the air track vector:

VW 5VG 2VA . (1)

The air track (i.e., aircraft relative) velocity vectorVA is

calculated from the true airspeed (TAS), which is de-

termined by the differential pressure observed by a pitot

tube; static pressure; static temperature; and the aircraft

heading, which is observed by either a flux valve or laser-

gyro navigation system. The ground track (i.e., Earth

relative) velocity vector VG is calculated from the GPS-

observed ground speed and track angle. SinceVA andVG

are typically much larger than VW , it is critical that they

are measured with a high degree of accuracy.

Each TAMDAR system uses an integrated GPS, and

the associated error is nearly two orders of magnitude

smaller than the observed values, so it is assumed to be

negligible in the overall calculation. Themagnitude ofVA

(i.e., TAS) can either be measured from the TAMDAR

pitot and static pressure transducers and static tempera-

ture or the aircraft bus data. In this study, the ERJ-145s

use the TAS directly off the bus. The Saab 340s calculate

TAS from the bus-indicated airspeed (IAS), pressure

altitude, and the TAMDAR temperature. The heading

angle associated with VA is obtained from one of the

two types of heading systems mentioned above and is

by far the largest contributor to error in the wind velocity

calculation.

b. Flux valve heading systems

The magnetic flux valve, or flux gate, heading system

is an electronic magnetometer that measures the direc-

tion of the horizontal component of the earth’s geo-

magnetic field relative to the aircraft. To provide a stable

heading, especially during aircraft maneuvering, the final

heading is obtained from a system where a gyroscope is

slaved to the flux valve output. Prior to the wind vector

calculation, the true heading is calculated from the

magnetic heading by applying the magnetic declination

(i.e., variation) for the particular latitude, longitude,

and date.

Any long-term systematic errors or biases in the flux

valve will propagate through the system and cause errors

in the heading provided to TAMDAR, thereby degrad-

ing the accuracy of the wind calculation. The heading

error, as a function of measured heading, is called the

magnetic deviation. Two common sources of error are

those caused by soft iron effects and hard iron effects (i.e.,

subpermanent magnetism). Soft iron effects are due to

material that is temporarily affected by an external

magnetic field (e.g., the earth’s magnetic field), whereas

hard iron effects are due to permanently magnetized

material. The magnetic fields from these sources will add

to the earth’s field, and produce distortion in the magni-

tude and direction of the measured field. Since the hard

iron is fixed relative to the plane (e.g., a magnetized bolt),

its effect is almost entirely a function of heading, and to

a lesser extent, attitude.1 The general equation for mag-

netic deviation is

d5A1B sinz0 1C cosz01D sin2z01E cos2z0 , (2)

where d is the magnetic deviation and z0 is the magnetic-

based system heading. The coefficientsA,B,C,D, andE

are constants. A complete derivation of (2) is presented

in the appendix. The constant A is the general error

1Attitude is an aviation term used to describe aircraft orientation

about the center of mass, including pitch and roll.

1734 JOURNAL OF ATMOSPHER IC AND OCEAN IC TECHNOLOGY VOLUME 31

Unauthenticated | Downloaded 04/24/22 03:14 PM UTC

offset, while B and C are the semicircular sin(z0) and

cos(z0) deviation curves and are related to the parallel

and perpendicular asymmetrical components, respec-

tively, of the hard iron error. ConstantsD and E are the

quadrantal sin(2z0) and cos(2z0) induced magnetic de-

viation curves and are related to the symmetrical and

asymmetrical components, respectively, of the hori-

zontal soft iron error.

At low levels, hard iron errors produce an approxi-

mate sinusoidal curve in the heading distribution. The

sinusoidal nature of the heading errors observed on

TAMDAR-equipped flux valve planes suggest that hard

iron effects are the dominant factor; therefore, we make

the assumption that theD and E components of the soft

iron error are very small and can be ignored, so that the

fit can be characterized by

d5A1B sinz0 1C cosz0 . (3)

The fixed offset component A is typically caused by

either inaccurate flux valve mounting or calibration is-

sues, whereas the sinusoidal components are typically

caused by hard iron effects. The strength of the horizontal

component of the earth’s magnetic field is strongest near

the magnetic equator and weakest near the magnetic

poles. As a result, the magnetic deviation curve will vary

with magnetic latitude for a constant hard iron effect,

while the attitude (i.e., pitch and roll) will also contribute

a very minor effect. The aircraft pitch is unknown, but all

TAMDAR wind observations are flagged as unusable if

the roll angle exceeds 108.A small fraction of the TAMDAR-equipped turbo-

prop fleets has detectable heading errors in the yaw that

can significantly degrade the accuracy of the wind cal-

culation. It is technically possible to reduce this error

by carefully performing compass swings and applying

a calibration; however, accurate swings are difficult to

accomplish, and do not fit well with the standard main-

tenance practices of most commercial airlines. Further-

more, errors on the order of 48–58 may be considered

acceptable to an airline, and no additional adjustments

will be performed during the routine compass check.

However, errors this large can have a serious negative

impact on the accuracy of the wind observations. These

errors are unacceptable, yet are easily corrected by the

method described below.

3. Methods

The method described below is designed to charac-

terize the magnetic deviation as a function of measured

heading on an aircraft-specific basis. This is performed

by comparing wind vector observations from thousands

of TAMDAR reports to wind vectors extracted from

high-resolution rapid-cycling model analysis fields. An

additional cross check compares both the model analysis

winds and the TAMDARwinds to neighboringAMDAR

and raobs. This is not a ground system–based correction;

the ground processing is only done once to determine the

aircraft heading system biases. A lookup table based on

this characterization is then uploaded to each TAMDAR

system and is used to correct the heading in real time

before the wind calculation is performed. The general

protocol follows this series of steps:

d Calculate the ground track vector for each weather

observation.d Calculate the air track vector from the TAMDAR-

observed wind vector.d Calculate the air track vector from the reference

model analysis wind vector.d Take the difference between the two air track vector

headings to obtain the heading error.d Form a database of heading error versus aircraft-

measured heading based on a large sample (e.g., 1

year of multiple daily flights).d Fit a sinusoid curve to the distribution of heading

errors as a function of measured heading.d Derive themagnetic deviation correction lookup table

from the inverted sinusoid.

The ground track vectorVG in Fig. 1 can be calculated

from the latitude and longitude change between adja-

cent points via theGPS, as discussed in section 2. The air

track vector VA is calculated using (1) for two different

values:

VA5VG 2VWTAM

(4)

FIG. 1. Vectors showing VG, VWTAM, VWREF

, VA based on VWTAM,

andV0A based onVWREF

. Themagnitude ofVA is the TAS; the angle

of VA is the heading (c). The ground track angle is h.



and

V0A5VG 2VW

REF, (5)

where VA and V0A are the aircraft track vectors. The

value for VA is based on the TAMDAR-observed wind

vector, VWTAM, and V0

A is based on the reference model

analysis wind vector,VWREF. The ground track vectorVG

has an angle h from north, while the heading angles for

VA andV0A are c and c0, respectively. The heading error

is defined by the difference between these two angles:

cerr 5c2c0 . (6)

The accuracy of the calculated ground track angle (h)

has no impact on the accuracy of the heading error

calculation (cerr). This is because the same ground track

is used for both air track calculations, so any ground

track angle error will cancel. This can be seen in Fig. 1,

where any change in h due to an error will affect both c

and c0 equally. A ground speed error will, however, have

some impact on heading error. Also, an error in h will

result in the associated heading being slightly in error,

but since magnetic deviation changes rather slowly with

heading, a small heading error is not expected to be

significant.

4. Data analysis

Each plane has a unique heading error dataset that is

constructed from several months to over a year of model

comparisons with multiple model analyses. The models

used in this study were the National Oceanic and At-

mospheric Administration (NOAA) Global Systems

Division (GSD) Rapid Refresh (RAP), the RUC, and

the PAC Real-Time Four-Dimensional Data Assimila-

tion (RTFDDA) model. These models are ideal, as they

generate analysis fields hourly. It should be noted that no

forecast output is used, only the model analysis. Since the

duration of the study spanned over a year that coincided

with the transition from the RUC to RAP at GSD, the

RAP was only employed in the second half. Since the

dataset is used to map heading-against-heading error, it

is independent of when each model was used as well

as seasonal biases, provided the compiled data spans

a lengthy time period. The North American Mesoscale

Model (NAM) and Aircraft Communications Address-

ing and Reporting System (ACARS) data were also used

in this study, when available, to validate the other model

comparisons. With the exception of the PenAir Alaskan

fleet analysis, which used only RAP because of RUC

and RTFDDA domain limitations, no fewer than two

independent models were used in the comparison at any

given time throughout the entire dataset construction.

a. Dependence on magnetic latitude

To simplify the data analysis process, the method

presented here uses true heading, which is obtained

from magnetic heading with a magnetic declination ap-

plied. The magnetic declination is not a constant offset

and does vary over a geographic region. This can result

in a dataset with slightlymore noise, and a final magnetic

deviation curve that might be shifted along the x axis by

a couple of degrees depending on the geographic region.

Because the influence of the magnetic declination is

small in general, and further minimized by the fact that

regional airline fleets cover limited geographical areas

(i.e., short flights), these effects are considered to be

negligible and were ignored for this study.

The magnetic latitude variation will result in a hori-

zontal shift of the curve by the amount equal to the

average magnetic declination. Since the points on the

resultant curve contain no information as to the decli-

nation, these errors will present as noise or a slight bias if

the net variation is not zero. The root-mean-square er-

ror (RMSE) for a given swing in magnetic declination

will be proportional to the amplitude of the error curve.

Magnetic declination as a function of geographic lo-

cation for the three fleets tested is presented in Table 1.

The magnetic declination varies according to the right-

most column (Mag dec range). If we assume the varia-

tion of the data is random noise, then this will not have

a large effect when the sinusoid curve fit is done; how-

ever, the average or median magnetic declination (Me-

dian mag dec) will have an effect and is discussed below.

The magnetic field strengths are based on the eleventh-

generation International Geomagnetic Reference Field

(IGRF 11) model assuming a 6-km height above mean

sea level.

The PenAir (Alaska) fleet is shown in Fig. 2a. The

RMSE for the correction for this example is 0.828. Theother two fleets have less error because the median

magnetic declination is less. It is possible that a correc-

tion can be applied during the ground analysis, so that

magnetic heading is used to develop the error curves.2

The Chautauqua fleet is shown in Fig. 2b, and Mesaba is

shown in Fig. 2c. The Chautauqua RMSE for the cor-

rection is 0.478, while the Mesaba RMSE for the cor-

rection is 0.058. PenAir is expected to be impacted the

most based on their fleet location, while Chautauqua is

much less but still larger than Mesaba. This is likely

2 This may be a future TAMDAR analysis software improve-

ment.



because Chautauqua has many ERJs that make longer

flights (i.e., greater range) exceeding 1 h, and Mesaba

primarily makes very short flights.

b. Assumptions

d Model errors are not correlated with the heading of

the aircraft (i.e., the model does not know which

direction the airplane is flying).d Model wind speed bias is ,1m s21.d The primary cause of TAMDAR wind error is due to

heading errors, not TAS errors.d The heading error of the aircraft is a function of

heading only.3

d The magnetic declination correction applied by

TAMDAR (from the Garmin GPS) is accurate.4

d The calculated heading used for the table of heading

and heading error is sufficiently accurate. Since the

magnetic deviation changes rather slowly with head-

ing (section 4e), sufficient accuracy is expected.d The particular plane being used has not had the flux

valve system recalibrated during the period of data

analysis.5

d For the purpose of this study, we assume the plane stays

within a certain region or band of similar magnetic

latitude. In theory this could be accounted for, but with

regional airlines, there is not a significant variation.d Induced errors are dominated by hard iron effects that

do not change significantly over time or the operational

geographic area.6

d The strength of the horizontal component of the earth’s

magnetic field does not change significantly over time

or the geographic area covered by most regional

airline flights. The effect of a fixed hard iron error

will be a function of the strength of the earth’s field.

This means the calibration may only apply over a lim-

ited geographic region and may be dependent on

magnetic latitude (see Table 2).

c. Considerations and caveats

d For dataset construction, it is probably best to restrict

the data to when the airplane is in cruise (i.e., flying in

a straight line). Turning, banking, ascents, descents, or

any other maneuvering other than a linear flight track

can add uncertainty to the vector difference. Although

this would appear as random noise, and likely be

averaged out over a long time period of sampling, it

was not included in this analysis.d The resolution of the latitude and longitude GPS

position reports will affect the accuracy of the ground

track vector calculation. This appears as a nonbiased

random (quantization) error and is expected to be

averaged out in the curve fit process. The resolution of

the latitude and longitude is 0.1 arcmin. At a speed of

250 kt (1 kt5 0.51m s21), this errorwould be a random

ground track angle error of about 0.58.d The optimum dataset for a plane includes flights in-

volving all headings (i.e., a complete 08–3598 range).This may require a few months of data collection for

a particular plane depending on the airline’s scheduled

routes. Airlines tend to reuse the same planes for

certain routes; however, they do move the planes

around every few weeks. This results in clusters of

data points because of the abundance of route-specific

headings. This is why the analysis can last up to a full

year.d There are several numerical weather predictionmodels

that can be used, and when used in parallel, the

robustness can be greatly improved. In this study, we

use the PAC RTFDDA, RUC, RAP, and the NAM.

Additionally, it is possible to use ACARS and raob

data, but the limited space–time proximity of those

observations coincident with TAMDAR data result

in only a few points.

TABLE 1. Magnetic declination with geographic location for the three fleets tested.

Airline

Upper

lat (8)Lower

lat (8)Upper mag

lat (8)Lower mag

lat (8)Max horizontal

field (nT)

Min horizontal

field (nT)

Mag lat

range (8)Median mag

dec (8)Mag dec

range (8)

Mesaba 48.09 32.2 57.03 40.87 23 630.1 15 648.6 16.16 21.02 13.2

PenAir 63.85 54.54 66.58 54.43 20 254.6 12 896 12.15 16.69 8.31

Chautauqua 45.3 32.19 54.07 40.93 23 863.7 17 182.3 13.14 29.5 17.48

3 This is likely the case, as heading errors are typically caused by

fixed, local magnetic effects in the aircraft.4 Since we see significant variations in wind quality from the

Mesaba fleet and they all use the same GPS, the magnetic decli-

nation is assumed not to be a significant contributor to the error.

Likewise, the same GPS unit is used in the laser-gyro systems on

other planes, which have much more accurate winds.5 This is beyond our control but from discussions with Mesaba

and Saab, we know that even though heading checks are done, an

actual calibration does not happen very often. Other maintenance

that may affect flux valve accuracy is also possible but not within

our ability to determine. Since wind quality is constantly moni-

tored, any changes due to maintenance that degrade winds signif-

icantly will be quickly noted.6 The sinusoidal nature of the magnetic deviation suggests this is

the case.



FIG. 2. Error in the correction curve based on the median magnetic

declination for the (a) PenAir, (b) Chautauqua, and (c) Mesaba fleet.

We assume a typical magnetic deviation of648. The correction curve

is the solid line, and the correction curve with additional compensa-

tion for magnetic latitude variation is the dotted line. The dashed line

is the error from not accounting for this influence.



d Magnetic declination (i.e., variation) can result in an

error that would contribute to an offset in the curve,

which is inherently corrected, to a certain extent, by

this method. This requires the underlying assumption

that the plane’s flight routes are limited to a smaller

geographical domain. For longer flights (e.g., trans-

continental), this correction may not be helpful. Fortu-

nately, almost all of the planes with larger geographical

routes employ laser-gyro heading instrumentation and

are not subject to these errors.

d. Longitudinal and transverse wind errors

To quantify the quality of the wind observations,

we analyze the total RMSE of the wind vector mag-

nitude, the RMSE of the longitudinal (i.e., along

track) component, and the RMSE of the transverse

(i.e., across track) component. Using the total RMSE

of the wind vector magnitude is convenient because it

includes the error contributions from both speed and

direction.

The effect of TAS error versus heading error on the

wind accuracy can be seen by plotting the components

independently. Additional discussion of speed versus

directional error of aircraft observations can be found in

Huang et al. (2013). These errors are monitored by one

of the multistage quality control processes called Delta

Hound run by PAC. The transverse wind error compo-

nent is almost entirely dominated by the contribution of

heading error, while the TAS primarily affects the lon-

gitudinal component.

An example of the performance difference between

both types of heading systems is shown in Fig. 3. Two

groups of ERJ-145 planes from the same fleet were

analyzed for twomonths. One group has the Honeywell

AH-900 Attitude and Heading Reference System

(AHRS), which provides heading from a laser-gyro

source, denoted by _LG in Fig. 3. The other group has

the Honeywell AH-800 ARHS system, which provides

heading from a magnetic flux valve, denoted by _FV in

Fig. 3. For the laser-gyro system, the longitudinal (solid

circle) and transverse (solid triangle) wind RMSE are

both small and quite similar, which suggests that neither

the TAS nor heading errors are the dominant contributor

to the total RMSE (solid square).

However, for the magnetic flux valve system, it is

clearly evident that the transverse component (dashed

triangle) is the dominant contributor to the total RMSE

(dashed square), while the longitudinal component

from the flux valve (dashed circle) is no different than

that of the laser-gyro system. Not surprisingly, this

eliminates TAS and implicates the magnetic flux valve

heading system as the source of the error seen in the total

RMSE. The trend of transverse error increasing with al-

titude is consistent with the fact that the vector-based

heading errors are magnified as the speed of the plane

increases.

e. Data analysis examples

Approximately one year of data from RTFDDA,

RUC or RAP, and ACARS, when available, are used for

each plane. Two examples of the raw data for two dif-

ferent planes can be seen in Figs. 4a and 4b. The trends

appear sinusoidal, which is expected for magnetic de-

viation. Every plane studied had a similar sinusoidal ap-

pearance to the data shown but with differing phase,

offset, and amplitude. For reasons noted above, it was

decided to use a sinusoidal fit as a function of heading (c).

The sinusoid fits the data well and also has the advantage

of wrapping around at 3608 with no discontinuity.

Rather than using an actual sinusoid for the final cor-

rection, an eight-point lookup table is used and a piece-

wise linear fit closely approximating the sinusoid is used

in the TAMDAR to obtain the corrected heading, which

is used in the wind calculation.

TABLE 2. Observation comparison counts and wind component error for test phases 1 and 2 for control (A) and experimental (B) aircraft

groups. Numbers in parentheses are normalized errors based on the model performance difference between the two time periods.

Aircraft group Type (No.) Test period Obs

Total wind

RMSE (kt)

Lon wind

RMSE (kt)

Transverse wind

RMSE (kt)

Control A Saab 340 (17) Phase 1 143 248 8.8 5.4 7.1

Phase 2 (normalized) 132 887 9.4 (8.8) 5.8 (5.4) 7.6 (7.1)

Expt A Saab 340 (19) Phase 1 165 846 10.8 5.4 9.8


Control B Saab 340 (3) Phase 1 15 901 11.0 6.5 8.8


Expt B Saab 340 (4) Phase 1 16 833 16.4 6.3 15.6


Control C ERJ-145 (7) Phase 1 99 064 8.8 6.2 6.2


Expt C ERJ-145 (3) Phase 1 29 723 30.0 6.4 32.1




The plane in Fig. 4a flew several flight legs a day, and

to a wide range of locations, so the distribution of com-

parison points was extremely robust. The plane in Fig. 4b

did not fly as frequently, so the number of comparisons

per same time period is smaller than those in the other

example. Also, this plane largely flew the same routes

repeatedly, so the comparisons appear grouped along the

heading axis. As an additional means of comparison,

ACARSare also included in Fig. 4b.Datawere filtered to

eliminate points where the aircraft was maneuvering.

Since the phase, offset, and amplitude are unique to each

plane, the possibility of biases existing that are not related

to the plane is essentially eliminated.

The eight-point piecewise linear fit approximates the si-

nusoid, and the points are every 458 (i.e., 08, 458, 908, 1358,etc.). A linear interpolation is applied between each of

the eight points. The sinusoid function takes the form

y(c)5 1:05[A sin(c)1B cos(c)]1C , (7)

where A is the in-phase component, B is the quadrature

component, and C is the offset. A typical regression fit

undercuts the curve slightly, so an adjustment factor of

1.05 is included to unbias the undercutting of the straight

lines between points on the curve. This adds 5% to the

sample points on the sinusoid, so that the resulting

error between the piecewise liner fit and the sinusoid is

reduced.

The curve of this equation is sampled every 458 to get theeight values retained in the lookup table. It should benoted

that this table of calibration constants can be overridden to

provide values outside of the sine curve constraints (i.e.,

the TAMDAR software is not restricted to the sinusoidal

assumption); however, this is not currently done.

5. Results

Results from detailed field experiments to validate the

magnetic deviation correction are shown in Table 2.

Three different fleets (i.e., regional airlines) were ana-

lyzed and are categorized by group—A, B, and C.Airline

A (Mesaba) is based in theGreat Lakes region but covers

routes over most of the Ohio River valley, B (PenAir) is

based in southern Alaska, and C (Chautauqua) covers

most of the mid-Atlantic and southern New England.

Two airplane types were included: the Saab 340 and

the ERJ-145. Each airline group was divided into two

subgroups: the experimental and control. The experi-

mental subgroup contains planes that were found to have

significant errors in their heading systems according to

the tests described in section 3c. After the yearlong

analysis from section 3d was performed, and the mag-

netic deviation lookup table, derived from the sinusoid,

was uploaded to the TAMDAR firmware on the exper-

imental subgroups, a controlled test was performed to

quantify the level of error reduction. The control sub-

group was left unchanged throughout the experiment.

FIG. 3. Vertical profiles (pressure altitude in thousands of feet) of total (square), longitudinal

(circle), and transverse (triangle) wind RMSE (kt) for the LG (solid) and the magnetic flux

valve (FV; dashed) heading systems from the Chautauqua ERJ-145 fleet.



Phase 1 of the test compiles the error statistics for both

the control and experimental subgroups prior to the

correction being applied. In Table 2, the number of ob-

servations and the wind observation RMSE are shown.

Phase 2 compiles the error statistics over a window of

time, discussed below, after the correction has been

applied to the experimental subgroup. Since phases 1 and

2 were conducted at different times, the model error may

not have been the same for both periods; therefore, in

phase 2, the wind RMSE was normalized to account for

the model error of both RAP and RTFDDA. Both error

values from phase 2 are shown in Table 2.

a. Group A results

In group A, there were 17 Saab 340 control planes

and 19 experimental planes from theMesaba fleet in the

Great Lakes region. The data gathering period to build

the sinusoid for this group was from 3 August 2008 to 28

July 2009. Phase 1 for group A ran from 15 August 2009

FIG. 4. Examples of raw data with sinusoidal fits for two different planes for magnetic de-

viation relative to the (a) RAP and RTFDDAmodels and (b) RAP and RTFDDAmodels and

ACARS observations vs heading (degrees from north).



to 15 September 2009. For the control A subgroup, the

wind error was 5.4, 7.1, and 8.8 kt for longitudinal,

transverse, and total RMSE, respectively. The error for

the phase 1 experimental A subgroup was 5.4, 9.8, and

10.8 kt for the longitudinal, transverse, and total wind

RMSE, respectively. Phase 2 of group A ran from 21

October 2009 to 21 November 2009.

The control group error differed between phases 1 and

2, and was assumed to be a result of general model error.

An adjustment (i.e., normalization) factor was derived

from the difference in the control group phases and was

applied to both phase 2 subgroups. After being nor-

malized to account for model error, the phase 2 exper-

imental A subgroup normalized error was 5.1, 6.3, and

8.4 kt for the longitudinal, transverse, and total wind

RMSE, respectively.

The before (phase 1) and after (phase 2) wind RMSE

from groupA is shown in Fig. 5. Asmentioned in section

4c, the transverse component dominates the error seen

in the total wind RMSE. Since there is minimal contri-

bution from the longitudinal component, there is es-

sentially no change in error seen in Fig. 5. However,

a significant reduction in error is seen in the transverse

component and as a result the total error is reduced.

The total wind RMSE of the experimental subgroup

A was improved by 22%. As expected, this change was

almost entirely a function of the error reduction in the

transverse wind component, which improved by 36%,

compared to the 6% improvement in the longitudinal

wind RMSE. The distributions of these results are con-

sistent with the findings of TAS versus heading error

impact on the observed wind vector components dis-

cussed in section 3c.

b. Group B results

In group B, there were three Saab 340 control planes

and four experimental planes from the PenAir fleet in

Alaska. This region typically has higher magnetic declina-

tion and lower model accuracy. The data gathering period

for this group was 1 February 2009 to 9 March 2010. The

RAP,NAM, andACARSwere used for this group but not

the RTFDDA, as this region was beyond the domain.

Phase 1 for group B spanned 9 January–9March 2010.

For the control B subgroup, the wind error was 6.5, 8.8,

and 11.0 kt for longitudinal, transverse, and total RMSE,

respectively. The error for the phase 1 experimental B

subgroup was 6.3, 15.6, 16.4 kt for the longitudinal,

transverse, and total wind RMSE, respectively. Phase 2

of group B spanned 27 March–27 May 2010. Phase 2

experimental B subgroup normalized error was 4.8, 6.2,

and 8.2 kt for the longitudinal, transverse, and total wind

RMSE, respectively.


from group B is shown in Fig. 6. Unlike groupA, group B

saw a very slight improvement in the longitudinal com-

ponent. In addition to this, there was a very substantial


(circle), and transverse (triangle) wind RMSE (kt) for phase 1 (solid) and phase 2 (corrected;

dashed) of group A (Mesaba Saab 340) planes.



improvement in the transverse component that greatly

improved the overall wind RMSE. The improvements

grew with altitude. This is primarily because at higher

speeds, a small heading error can result in a larger

transverse component error. As seen in Fig. 6, this was

almost entirely eliminated in phase 2.

The total windRMSEof the experimental subgroupB

was improved by 50%. The percent improvement in the

transverse wind component was 60%, and the percent

improvement in the longitudinal wind RMSE was 24%.

c. Group C results

In group C, there were seven ERJ-145 control planes

and three experimental planes from the Chautauqua

fleet. The data gathering period for this group was from

9 March 2009 to 9 March 2010. Phase 1 for group C

spanned 10 January–10 March 2010. For the control C

subgroup, the wind error was 6.2, 6.2, and 8.8 kt for

longitudinal, transverse, and total RMSE, respectively.

The error for the phase 1 experimental C subgroup was

6.4, 32.1, and 30.0 kt for the longitudinal, transverse,

and total wind RMSE, respectively.

The transverse component error for this group is ex-

tremely large compared to the Saabs in the other groups.

This is because the ERJs fly at speeds almost twice as

fast as the Saabs. TAMDAR uses a time-based sample

window to calculate thewind vector, which is the same for

all planes. The ERJ travels nearly twice as far in the same

amount of time as the Saab, so the resulting transverse

wind error from a similar angular heading uncertainty

will grow by the same factor.

Phase 2 of group C spanned 11 March 2010–11 May

2010. The experimental C subgroup normalized error

for phase 2 was 5.3, 8.2, and 9.5 kt for the longitudinal,

transverse, and total wind RMSE, respectively.


from group C is shown in Fig. 7. As with group A, the

longitudinal component of group C was essentially un-

changed. Group C saw themost substantial improvement

from the phase 2 correction in the transverse component

and the resulting total wind RMSE. The group C im-

provements grew with altitude. As mentioned in sec-

tion 5b, the small heading error in group C was creating

a significant transverse component error at high speeds,

which appears, coincidentally, as a function of altitude.

This error was most significant in group C, likely because

this group was composed of ERJ-145s, which fly at higher

speeds. Because of this, applying the phase 2 correction to

the planes had a tremendous positive impact on overall

wind error reduction.

The total windRMSE of the experimental subgroupC

was improved by 68%. The percent improvement in the

transverse wind component was 74%, and the percent

improvement in the longitudinal wind RMSE was 17%.

As expected, this improvement was dominated by the

error reduction in the transverse wind component.



dashed) of group B (PenAir Saab 340) planes.



6. Conclusions

Historically, it has been assumed that heading infor-

mation supplied by flux valve magnetic sensor devices

to an aircraft’s compass system would not be able to

provide wind data accurate enough to add value in nu-

merical weather prediction. With the TAMDAR-based

technique employed here, the majority of the aircraft’s

unique magnetic deviations can be filtered out, and the

limitations of the heading system overcome.

When employing this correction methodology, the

total wind RMSE was reduced by 22%, 50%, and 68%

for subgroups A, B, and C, respectively. While the lon-

gitudinal components of the error were improved in all

cases, these were small compared to the substantial im-

provements seen in the transverse wind components,

whichwere clearly the source ofmost of the original error.

Additional long-term analysis will be conducted to

refine this technique; however, when comparing the

corrected (after) data in Figs. 5–7 to the laser-gyro (LG)

data of Fig. 3, preliminary analysis suggests that flux

valve–based heading systems are capable of providing

wind data approaching the quality of ring LG-based

inertial navigation systems on similar sized aircraft.

Acknowledgments. The authors thank Heather

Richardson, Peter Childs, Feng Gao, Jamie Braid, and

Richard Ferguson for their valuable comments and

suggestions. We are grateful for the code support of

TAMDAR assimilation and quality monitoring pro-

vided by Yubao Liu and Xiang-Yu Huang (NCAR).

We also thank three anonymous reviewers for their

very helpful and detailed comments.

APPENDIX

Approximation for Magnetic Deviation

The existence of magnetic declination in compass

heading has been known by mariners since the early

1400s. The effects of magnetic deviation induced by iron

within the ship were first documented by Joao de Castro

of Portugal in 1538 and became more evident by the

1800s, when iron hulls were introduced. The theory be-

hind iron-induced magnetic deviation in ship navigation

was originally proposed by Siméon-Denis Poisson in an1824 memoir to the French Institute, and the firstmathematical derivation of the set of equations by Pois-son to compensate for the compass heading deviation inships appeared in the article ‘‘Mémoire sur les dévia-tions de la boussole, produites par le fer des vaisseaux’’(Poisson 1838). The refined derivation with a set of em-

pirical constants, credited to Scottish mathematician

Archibald Smith, appears in Airy (1839) and Sabine

(1843). After lengthy testing, the protocol for ascertain-

ing these constants on ships was published (Evans and

Smith 1865; Smith and Evans 1869).



dashed) of group C (Chautauqua ERJ-145) planes.



The mathematical correction is not specific to ships

and can easily be applied to any platform that relies on

magnetic-based heading instrumentation, including air-

craft with magnetic flux gate navigation systems. We

begin with the set of equations proposed by Poisson:

X 05X1 aX1 bY1 cZ1P , (A1)

Y 05Y1dX1 eY1 fZ1Q , (A2)

Z0 5Z1 gX1 hY1 kZ1R , (A3)

where X, Y, and Z are the components of the earth’s

magnetic force relative to the ship (i.e., forward, star-

board, and downward, respectively); and X0, Y0, and Z0

are the combinedmagnetic force of the earth and the ship

along the same coordinates. The coefficients a, b, c, d, e, f,

g, h, and k, are dependent on the vessel-specific amount

and arrangement of ‘‘soft iron’’ (i.e., nonmagnetizedmetal

that can distort the earth’s magnetic field). A complete

derivation of these coefficients can be found in Muir

(1906). The constants P,Q, andRwere added by Smith to

account for the effect of vessel-specific ‘‘hard iron’’ (i.e.,

permanently magnetized iron within the platform).

The magnetic deviation d is defined as the difference

between the angle from magnetic north z, and the

heading according to the instrumentation z0, so that d5z 2 z0. The horizontal force of Earth’s magnetic field is

given by H2 5X2 1Y2, and the horizontal force of the

combined magnetic fields of Earth and the platform is

given by H02 5X 02 1Y 02. These relations are used to

form the following set of equations:

8>>>>><>>>>>:

X5H cosz

Y52H sinz

Z5H tanu

X 05H0 cosz0

Y 052H0 sinz0

, (A4)

where u is the magnetic inclination (i.e., dip angle).

For the purposes of our horizontal magnetic deviation

correction, the force along the Z axis defined in (A3) is

ignored. Substituting (A4) into (A1) and (A2) yields

H0 cosz05H cosz1 aH cosz2 bH sinz1 cH tanu1P ,

(A5)

2H0 sinz0 52H sinz1 dH cosz2 eH sinz

1 fH tanu1Q . (A6)

After multiplying (A5) by sinz and (A6) by cosz, and

adding them together, we obtain the following equation

with some reductions:

H0 sindH

5d2 b

21

�c tanu1

P

H

�sinz

1

�f tanu1

Q

H

�cosz1

�a2 e

2

�sin2z

1

�b1 d

2

�cos2z . (A7)

As done in (A7), we nowmultiply (A5) by cosz and (A6)

by sinz, and subtract them before applying similar sim-

plifications to obtain

H0 cosdH

5 11a1 e

21

�c tanu1

P

H

�cosz

2

�f tanu1

Q

H

�sinz1

�a2 e

2

�cos2z

2

�b1 d

2

�sin2z . (A8)

We define the mean horizontal force in the direction

of magnetic north as lH, so that

l5 11a1 e

2, (A9)

and the following set of constants can be used to further

simplify (A7) and (A8):8>>>>>>>>>>>>>>>>><>>>>>>>>>>>>>>>>>:

Al5d2 b

2

Bl5 c tanu1P

H

Cl5 f tanu1Q

H

Dl5a2 e

2

El5b1 d

2

. (A10)

The constants A, B, C, D, and E are considered to be

exact values. They are traditionally written in Fraktur

Blackletter font in historical manuscripts. After substitut-

ing (A10) into (A7) and (A8), we obtain

H0 sindHl

5A1B sinz1C cosz1D sin2z1E cos2z ,

(A11)

H0 cosdHl

5 11B cosz2C sinz1D cos2z2E sin2z .

(A12)

Here, (A11) is then divided by (A12), which yields the

exact calculation for the horizontal magnetic deviation d:



tand5A1B sinz1C cosz1D sin2z1E cos2z

11B cosz2C sinz1D cos2z2E sin2z.

(A13)

While (A13) is the exact equation to calculate the

magnetic deviation, it is not practical to apply in an op-

erational setting because the angle from true magnetic

north, z, is not realistically attainable with the in-

strumentation. If it were, then there would be no de-

viation. To be a functional solution, we need to have

(A13) in terms of the instrument heading, z0.We begin by

multiplying (A11) byffiffiffiffiffiffiffi21

p, adding it to (A12), and ap-

plying Euler’s formula:

H0

Hl(cosd1 i sind)5

H0

Hleid 5 11 iA1B cosz1 iB sinz

2C sinz1 iC cosz1D cos2z

1 iD sin2z2E sin2z1 iE cos2z .

(A14)

After taking the natural logarithm of both sides, we

obtain the Taylor series form:

id1 ln

�H0

Hl

�5 iA1 (B1 iC)(cosz1 i sinz)1 (D1 iE)(cos2z1 i sin2z)2

1

2[iA1 (B1 iC)(cosz1 i sinz)

1 (D1 iE)(cos2z1 i sin2z)]211

3[iA1 (B1 iC)(cosz1 i sinz)1 (D1 iE)(cos2z1 i sin2z)]3 . . .

(A15)

The constants A, B, C, D, and E are less than 1, and in

practical applications B is typically less than 0.4, while

the others are less than half that value. As a result, the

higher-order terms, which contain only squares and

products of these constants, aremuch smaller and can be

neglected.We then substitute (A12) forH0/Hl, and apply

the same Taylor series expansion for ln(1 1 x):

d2 ln(cosd)i21 5A1B sinz1C cosz1D sin2z

1E cos2z . . . . (A16)

At this stage, we make the assumption that we are

dealing with small deviation angles. Smith and Evans

(1869) suggest deviation angles less than 208 are acceptable.

Since the typical magnetic deviation error in a flux gate

heading system on aircraft almost never exceeds 108, and ismore often less than half that error, this is a very reasonable

assumption. When dealing with a small angle of deviation,

we can assume cosd 5 1. By making this assumption, we

alter the original exact constants. We can express (A16)

with inexact constants in (A17), which are not in bold.

From this point on, the equation is an approximation:

d5A1B sinz1C cosz1D sin2z1E cos2z . . .

(A17)

To obtain (A17) in terms of z0, we substitute z 5 d1 z0,which eliminates z but leaves terms containing d:

d5A1B sinz0 cosd1B cosz0 sind1C cosz0 cosd2C sinz0 sind1D2 sinz0 cosz0 cos2d1D2 sind cosd cos2z01E cos2d cos2z0 2 2 sind cosdE sin2z0 . . . (A18)

Applying the small angle assumption from above (i.e.,

cosd5 1) simplifies (A18) to amore common expression

of an Nth harmonic function, which is typically trun-

cated to the form below when employed for the calcu-

lation of magnetic deviation:

d5A1B sinz0 1C cosz01D2 sinz01E cos2z0 .(A19)

REFERENCES

Airy, G. B., 1839: Account of Experiments on Iron-Built Ships:

Instituted for the Purpose of Discovering a Correction for the

Deviation of the Compass Produced by the Iron of the Ships.R.

and J.E. Taylor, 47 pp.

Daniels, T. S., W. R. Moninger, and R. D. Mamrosh, 2006: Tropo-

spheric Airborne Meteorological Data Reporting (TAMDAR)

overview. 10th Symp. on Integrated Observing and Assimilation

Systems for Atmosphere, Oceans, and Land Surface, Atlanta,

GA, Amer. Meteor. Soc., 9.1. [Available online at http://ams.

confex.com/ams/pdfpapers/104773.pdf.]

Evans, F. J., and A. Smith, 1865: On the magnetic character of the

armour-plated ships of the Royal Navy, and on the effect on the

compass of particular arrangements of iron in a ship. Philos.

Trans. Roy. Soc. London, 155, 263–323, doi:10.1098/rstl.1865.0005.

Gao, F., X. Zhang, N. Jacobs, X.-Y. Huang, X. Zhang, and

P. Childs, 2012: Estimation of TAMDAR observational error

and assimilation experiments. Wea. Forecasting, 27, 856–877,

doi:10.1175/WAF-D-11-00120.1.

Huang, X.-Y., F. Gao, N. A. Jacobs, and H. Wang, 2013: Assim-

ilation of wind speed and direction observations: A new



http://ams.confex.com/ams/pdfpapers/104773.pdf

http://ams.confex.com/ams/pdfpapers/104773.pdf

http://dx.doi.org/10.1098/rstl.1865.0005

http://dx.doi.org/10.1175/WAF-D-11-00120.1

formulation and results from idealized experiments. Tellus,

65A, 19936, doi:10.3402/tellusa.v65i0.19936.

Hughes, P., and S. D. Gedzelman, 1995: The new meteorology.

Weatherwise, 48, 26–31, doi:10.1080/00431672.1995.9925972.Moninger, W. R., R. D. Mamrosh, and P. M. Pauley, 2003: Auto-

mated meteorological reports from commercial aircraft. Bull.

Amer. Meteor. Soc., 84, 203–216, doi:10.1175/BAMS-84-2-203.

——, S. G. Benjamin, B. D. Jamison, T. W. Schlatter, T. L. Smith,

and E. J. Szoke, 2010: Evaluation of regional aircraft obser-

vations using TAMDAR. Wea. Forecasting, 25, 627–645,

doi:10.1175/2009WAF2222321.1.

Muir, W. C. P., 1906: A Treatise on Navigation and Nautical As-

tronomy: Including the Theory of Compass Deviations. U.S.

Naval Academy, 718 pp.

Mulally, D. J., and A. K. Anderson, 2011: Correction of aircraft

flux valve based heading for two-dimensional winds aloft

calculations using weather model comparisons. 15th Symp. on

Integrated Observing and Assimilation Systems for Atmo-

sphere, Oceans, and Land Surface, Seattle, WA, Amer. Me-

teor. Soc., 5.6. [Available online at https://ams.confex.com/

ams/91Annual/webprogram/Manuscript/Paper184928/5.6_

Correction_of_Aircraft_Flux_Valve_Based_Heading_for_

Two-Dimensional_Winds_Aloft_Calculations_Using_Weather_

Model_Comparisons.pdf.]

Poisson, S.-D., 1838: Mémoire sur les déviations de la boussole,produites par le fer des vaisseaux. C. R. Séances Acad. Sci., 6,755–766.

Sabine, E., 1843: Contributions to terrestrial magnetism. No. V.

Philos. Trans. Roy. Soc. London, 133, 145–231, doi:10.1098/rstl.1843.0011.

Smith, A., and F. J. Evans, 1869: The Magnetism of Ships and the

Deviations of the Compass: Comprising the Three Reports of the

Liverpool Compass Commission. U.S. Government Printing

Office, 344 pp.



http://dx.doi.org/10.3402/tellusa.v65i0.19936

http://dx.doi.org/10.1080/00431672.1995.9925972

http://dx.doi.org/10.1175/BAMS-84-2-203

http://dx.doi.org/10.1175/2009WAF2222321.1

https://ams.confex.com/ams/91Annual/webprogram/Manuscript/Paper184928/5.6_Correction_of_Aircraft_Flux_Valve_Based_Heading_for_Two-Dimensional_Winds_Aloft_Calculations_Using_Weather_Model_Comparisons.pdf







correction of flux valve–based heading for improvement of

Documents