beru.univ-brest.frberu.univ-brest.fr/~singhoff/doc/papier_a_trier/safety_proceed1.pdf1 1996 seminar...

1996 Seminar Proceedings1

1996 Seminar ProceedingsPart I

Table of Contents

With Reasonable Doubt: Australia as a Case Study of a Safe System.................................2Impact Reconstruction: Flight Path to Causation ..................................................................8Using Voluntary Incident Reports for Human Factors Evaluations ................................. 11International Accident Investigation in the Asia Pacific Region

An Airline Pilot’s Perspective ..............17The Air Accident Investigation Tool — Impact Tool and FDR Visualiser ........................20Adding Value to Aviation Operations:

New Directions for the 21st Century Investigator...............24American Eagle Flight 4184: “An Icing Crisis” ....................................................................32Helicopter Logging Mishaps: Applying Lessons In An Expanding Industry .................39Incidents: the Route to Human Factors in Engineering? ....................................................50Bogus Parts—Myth or Fact? ...................................................................................................56Accident Analysis with Advanced Graphics .......................................................................66Applying Perrow’s Complexity Model to Aviation Hazard Identification .....................72Flight Test Experience With Cockpit Audio/Video Recorders (CAVRs)..........................76Accident Investigation and Amateur-Built Experimental Aircraft ...................................80Management of Aviation Operations — Was the Board Room Active

in Preventing the Accident? .................93Light Aircraft Accident Investigation: An Engineering Guide ..........................................99Latent Failure and Human Factors ......................................................................................120The Airplane Safety Awareness Process .............................................................................129

AIR SAFETY

AUGUST 31, 1964INCORPORATED

INVESTIGATORS

International Society of Air Safety Investigators


Introduction

Since 1993, research at Loughborough University hasincluded examining the commercial air safety record ofAustralia in an attempt to rise to the challenge posed byProfessor James Reason: “Should we not be studyingwhat makes organisations relatively safe rather thanfocusing upon their moments of unsafety? Would it notbe a good idea to try and find out what makes themgood and whether or not these ingredients could bebottled and handed on?” (1)

The objective is not to prove Australia to be the safestplace to fly, nor its airlines the safest in the world. It issimply to examine what sort of defences are in place thatmay be strengthened or emulated.

Accidents occur as a result of the combination of severalcausal factors. Reason (2) describes these as latent defectsand active failures, and shows how multiple defencesmust be breached for an incident or accident to occur. Ittherefore seems logical that a safe system is such becauseof multiple strong defences.

What do we really mean by safety?

Safety may be described as “a judgment of the accept-ability of risk” (3) or, more simply, safety is a conditionwhere risks are minimised to an acceptable level. Thisbegs the question: how do we know what acceptable is?

We achieve safety by quantifying risk and attempting tobalance it with suitable countermeasures or ‘safetymeasures.’ That balance may produce a condition ofover-safety, absolute safety, or even unsafety.

Does Australia represent an area of low risk acceptability andtherefore high safety measures?

Geography

Australia’s climate is generally benign and does notsuffer the low extremes of temperature that other coun-tries of similar latitude do. However, parts of Australiaare prone to severe thunderstorms, especially aroundBotany Bay and Sydney. The subject of windshearremains relatively unexplored and, although there have

only been two large aircraft accidents due to windshear(F27 at Bathurst and RAAF F1-11), the only two studiescarried out(4)(5) reveal a higher than expected level ofactivity. Is a perception of low risk becoming based on alack of incidents rather than a lack of incidence?

Terrain is quite flat—only 2% of the landmass is over1000m above sea level. This may minimise the hazards of‘hot and high’ operations, but has a relatively small effecton controlled flight into terrain type collisions. (Accord-ing to Boeing, 48% of CFIT accidents occur in areas ofrelatively flat terrain.(6))

Traffic density is quite low—according to former Austra-lia Civil Aviation Authority (ACAA) Chairman, DickSmith(7), Australia only boasts 3% of the traffic flyingover the similarly sized United States. However, traffic isconcentrated in a few areas, and the risk of collision incontrolled airspace is more a function of air trafficcontrol than traffic density. It only takes two aircraft tocause a mid-air collision.

How does Australia react to its risks?

Decisions on risk acceptability are reached at either anindividual or group level. Each individual’s decisionswill be governed to a large degree by his or her personal-ity. At group level, the combination of personalitiescreates a culture. “Culture is to human collectivity whatpersonality is to an individual.”(8)

If we overestimate risk, and therefore set our level ofacceptable risk at an artificially cautious level, then weare, by definition, safer due to the extra margin of safetyafforded within our safety measures.

Analysis of any accident will reveal a number of fallibledecisions. These may be in the form of errors of commis-sion or omission, and may be latent or active. Everydecision involves an assessment of risks over con-sequences. How we make those decisions will ultimatelydictate how safe we are.

Do Australian personalities and sub-cultures lend themselvestowards a low acceptability of risk?

With Reasonable DoubtAustralia as a Case Study of a Safe System

Graham R. Braithwaite ST3644Loughborough University, England

With Reasonable Doubt: Australia as a Case Study of a Safe System


The Australians

The stereotyped Australian is a loud individual who ‘isnot backward about coming forwards, says what he likesand likes what he says.’ The Crocodile Dundee or Sir LesPatterson image is an endearing if not an enduring one.

More scientific studies of culture have includedHofstede’s much cited work using the IBM Corpora-tion.(8) Australia was found to be a country of highindividualism and a low power distance relationship. Inother words, the actions of others have a small effect onthe way individuals act, and there is a relatively flatpower gradient between manager and subordinate.

Further studies by Trompenaars (9) included a survey of15,000 staff around the world and make some interestingconclusions about different cultures. For example, whenasked the question “What makes a good manager?”, thepercentage opting ‘to be left alone to do the job’ was asshown in figure 1.

Such a clear result may be of concern to advocates ofcrew resource management (CRM). The concept of aCaptain being a team manager in a culture that ap-parently believes managers should be left alone seemscontrary to one of the principle aims of CRM. Somereassurance may be taken from Trompenaar’s nextquestion, which asked people how they would respondto a request from their manager to help paint his house.(See figure 2.) Once again, the Australian result repre-sents one extreme, this time demonstrating a lack ofdeference to authority.

Anecdotal evidence from the flight deck has created animage of even the most lowly second officer questioningthe actions of a captain without concern. The greatAustralian image of “larrikin irreverence to authority”(10) is one that many are proud of and happy to perpetu-ate. It does, however, seem to depart from one of themain factors which apparently lies behind Qantas’ safetyrecord, namely ‘strict adherence to Standard OperationalProcedures.’ (11)

Historical Factors

In the early days of civil aviation, Australia’s maintrading partners were Europe and North America. Theso called ‘tyranny of distance’ meant that trade voyageswere long and, in the absence of a land link, aviationpromised the only faster way of transport over sea. If areliable, fast transport link between Australia and itstrading partners could not be guaranteed then, economi-cally, it would have been left behind.

However, in the rush to exploit the new potential ofaviation, there were a number of tragic accidents. If anaircraft failed around Australia, assuming it was able toland, the chances of being found and rescued were quite Percentage

EgyptOman

SingaporeVenezuela

NepalHong Kong

East GermanyPhilippines

RomaniaBurkina Faso

RussiaNigeria

ChinaUAE

MalaysiaIreland

CzechoslovakiaThailand

JapanSweden

ArgentinaPoland

PakistanAustria

BelgiumItaly

UKSouth Africa

MexicoNetherlands

EthiopiaCuracao

USANorway

West GermanySwitzerland

CanadaAustralia

House PaintingPercentage who would refuse to help the boss

Percentage

Figure 2. Redrawn from Trompenaars (9)

Percentage

EgyptOman

SingaporeNepal

Hong KongEast Germany

PhilippinesKuwait

Burkina FasoRussia

NigeriaChina

UAETurkey

MalaysiaIreland

CzechoslovakiaThailandBulgaria

JapanSpain

ArgentinaBrazilGreece

BelgiumItaly

UKSouth Africa

MexicoNetherlands

EthiopiaUSA

FinlandDenmark

NorwayWest Germany

SwitzerlandCanada

Australia

What Makes a Good Manager?Percentage opting to be left alone to get job done.

Percentage

Redrawn from Trompenaars (9)Figure 1.


slim. In a country roughly the same size as the conti-nental United States lives a population of only about aneighteenth the size. As an example, the Southern Cloudcollided with terrain between Sydney and Melbourne in1931. Nobody witnessed the crash, and the wreckagewas not found until 1958.

Since Australia was colonised, success was based on‘survival of the fittest’ from which necessity became themother of invention. One example of the Australianaptitude for problem solving came when faced withweight limitations for Qantas 747 aircraft. It was one oftheir own engineers who invented the slide raft that isnow standard equipment on widebody aircraft the worldover.

The need for innovation and reliability was establishedat an early stage not only by the circumstances men-tioned above, but also as a result of certain key personali-ties such as Hudson Fysh and Fergus McMaster ofQantas and Reginald Ansett. The power of these indi-viduals in crafting the ‘personality’ of the organisationsthey created should not be underestimated, as will bementioned later.

Communication

Returning to the subject of adherence to SOPs, it is worthmentioning a specific example. For many years, Qantashas carried Second Officers on its long distance routes.Although not cleared to fly takeoffs and landings, theymay control the aircraft in cruise and allow other crewmembers the opportunity to rest. One of the SOPs forSecond Officers was to cross-check the actions of all crewmembers. Under such rules, a Captain making an errorand not being informed of such by a Second Officer wasmore likely to reprimand him/her than if he was in-formed. Although such a rule has now disappeared fromprint, it remains implicit to the corporate culture of theairline and continues.

There is an openness of communication which hasdeveloped within the Australian culture. It is especiallystrong between ‘mates,’ as Ward observed in his bookAustralia Since the Coming of Man; “The emphasis wason…masculine friendships and team-work and onmateship.” (12) Even when the commercial departmentsof Ansett and Qantas lock antlers, the safety departmentsremain in touch. When an Ansett Australia BAe 146suffered roll-back on all four engines during flight, themanager of airline safety conveyed the informationdirectly to Qantas for its franchised 146 operations. Bothairlines had imposed a new operating ceiling on theirBAe 146 aircraft long before ACAA issued its owndirective.

When pilots were asked how they would react to theintroduction of a new rule they considered to be unsafe,there was a marked difference in reaction even between

the UK and Australia. Figures 3 and 4 indicate a direct-ness of communication in Australia. Senior managers inAustralia appear to be more approachable even if onlybecause they have little choice in the matter!

0

20

40

60

80

Australians

A senior manager introduces a new rule you consider to be unsafe which of the following best describes your actions?

Percentage

© Braithwaite, 1996Figure 3.

0

10

20

30

40

50

British

A senior manager introduces a new rule you consider to be unsafe which of the following best describes your actions?

Percentage



Many of these traits of culture—openness, mateship, andinnovation—are invisible to those living within it, just aswater is invisible to a fish swimming within it. Culture isabout norms and we can only see its effects whenlooking at others and thinking “Why do they do that?”This makes the study of culture a difficult exercise thatrequires international effort. It is a subject with few realexperts and many ‘bloody experts.’

Factors…? What Factors?

When asked what had contributed to Australia’s goodrecord for airline safety, (See figures 5 and 6) the resultsfrom pilots and air traffic controllers were perhaps alittle surprising.

The most commonly cited factor according to ATC wasluck (5th most popular amongst pilots). Luck is a trickysubject for any scientific study; “any PhD. thesis whichattempts to prove the existence of luck would be scratch-ing on stony ground.” (13) If we assume that luck isdistributed randomly among the airlines (14), then any‘extra luck’ can only be as yet unexplained factors, suchas the influence of culture.

The contribution of sound maintenance (e.g., the rule ofno aircraft leaving maintenance with any MEL defi-ciencies) and crew training are of no surprise in a coun-try that has attained a world-wide reputation for both.The general perception that the influence of weather andlow traffic density are both highly significant is a littlequestionable for reasons already mentioned.

Few mentioned the importance of crew interaction;indeed, why should they? If such a phenomenon is theresult of deep rooted cultural norms, then they will bedifficult to distinguish.

When RAF’s Institute of Aviation Medicine asked Britishpilots to describe their colleagues,(15) the terms usedwere far from complimentary, with captains describingco-pilots as “overconfident, uncooperative, lazy, andcompetitive,” and co-pilots describing captains as“arrogant, abrasive, tyrannical, and aggressive.” Weasked Australian large jet pilots to describe each otherusing just three words. The majority of comments werevery positive, such as “professional, approachable, andexperienced” to describe captains, and “enthusiastic,supportive, and friendly” to describe subordinates. Evencomments aimed at their managers were predominantlypositive, suggesting a co-operative atmosphere. Themost used words being “professional, experienced, andapproachable.”

Of course, this is not to say that there are no problems orthat communication is perfect.

Problems afoot…

In recent years, Australia has suffered from an apparentdecline in safety standards. High profile crashes, includ-ing the loss of Piper Chieftain VH-NDU at Young, NSW,raised public and political interest to a level where aHouse of Representatives Inquiry was held into aviationsafety. The media coverage of Ansett Australia’s mishapinvolving a B747-300 at Sydney in 1994 was at an unprec-edented level. Airline safety is obviously important tothe Australian people—not necessarily because they area ‘safe’ people, but more because it is what they havecome to expect.

But how could an apparently good safety culture start togo into decline? Surely no one lets the safety margin startto erode? One word haunts anything that is apparentlysafe—complacency.

As mentioned earlier, if we overestimate risk, then weafford an extra margin in our safety measures. By the

Percentage

CurrencySmall Industry

Professional PrideDiscipline

Flight ChecksAircraft Reliability

Commercial PressureModern Aircraft

SelectionRegulation

S. O. P'sGood ATC

Crew InteractionAtmosphere

TerrainLuck

MaintenanceLow Traffic Density

CrewWeather

What factors have contributed towards Australia's safety record?(Answers from large jet pilots)

Percentage

© Braithwaite, 1996.Figure 5.

Percentage

Comm. PressureSmall Industry

Crew InteractionCurrency

Aircraft ReliabilitySelection

TCASS. O. P's

Ground SupportModern Aircraft

Professional PrideAtmosphere

TerrainRegulation

CrewGood ATC

MaintenanceWeather

Low Traffic DensityLuck

What factors have contributed towards Australia's safety record?(Answers from air traffic controllers)

Percentage

© Braithwaite, 1996.Fig 6.


same token, if we underestimate risk, then we lose thatsafety margin and become susceptible to accidents.Complacency is the result of either underestimating risk,overestimating benefits, or simply not understandingrisks.

Australia’s safe operations were attained through mucheffort, but the roots of its safety culture are not immedi-ately obvious. Anecdotal evidence may suggest that therecord is a result of ‘good weather and low terrain’, butaccident cause statistics indicate 70-80% of accidents arethe results of human error. If that is true, then where areAustralia’s human factors airline accidents?

The seeds of such accidents may already be in place asthe result of fallible decisions. Accidents are preventedby recognising those mistakes and compensating forthem. Fundamental changes to ACAA in the early 1990s,not least under the influence of the charismatic Chair-man, Dick Smith, seem to have degraded the margin ofsafety. BASI concluded in a recent accident report that“inadequate resources, which restricted the ability of theCAA to conduct regulatory activities concerning thesafety of flight operations” were latent failures whichcontributed to a fatal accident.

When asked what they considered to be the greatestthreats to the safety of commercial operations in thefuture in Australia, pilots answered as shown in figure 7.The majority of answers are human factors issues at alllevels of industry, not just at the ‘sharp end.’

Is it possible that a safety culture can be altered so quickly orby key individuals?

Key personnel have managed to craft the fortunes ofmany companies around the world. Bill Gates atMicrosoft, Victor Kiam at Remington, and RichardBranson at Virgin are a few examples of the power of oneover many. So, too, individuals placed in the ‘right place’can have a negative effect on a safety culture. Suchdecision makers, with incomplete understanding of risk,are open to making the fallible decisions that are foundin every accident.

What Do We Need To Do?

The communication of “facts, analyses and findings tothose people or organisations which may use suchinformation effectively”(16) is the duty of all air safetyinvestigators. If we accept that there are no new acci-dents, only variations on old ones, then perhaps it isnecessary to take stock of why fallible decisions are stillbeing made.

There is a great deal we still do not understand about thecauses of incidents and accidents. The influence ofculture is one such area that has been filed in the “toohard basket” and will continue to be until investigators

are bred to deal with it. The continuing globalisation ofairlines and mixing of crew nationalities will not makethings easier unless a true understanding of the culturefactor is achieved.

A close study of a country’s aviation system by a thirdparty is one way of auditing the true safety picture. Theinvisibility of culture to those who live within it makesthe need for international collaboration imperative. Thequestion is, who will be brave enough to do it, and willthose ‘brave’ countries actually be the ones who have anopen enough culture to be in the least need of such astudy?

Conclusions

Australia provided the opportunity to examine a ‘good’aviation system in a positive light to determine whatmade it so. Just like any large socio-technical system, ithas certain problems, but so far it has been held togetherby a strong ‘glue’ that appears to be a function of work-group, corporate, industry and national cultures. It doesnot represent a utopian system or the sole way of operat-ing, but it does provide a number of valuable lessons forthose who wish to learn. Although there are few of theso-called “culture induced accidents,” it is near impos-sible to find an accident that has not had a culturalinfluence upon it.

Finally, to answer the question posed by the conferenceorganisers, ‘Where To From Here?’ from the viewpointof one of ISASI’s most junior members: The culturaldimension needs to be understood and tackled. We havethe expertise and resources to breed thoroughbredcultural human-factors investigators for the future. Thebig question is, who has the organisation culture in placeto make that dream a reality?

Footnotes

(1) Reason, J., (1993) Human Factors In Aviation. Pro-ceedings of 22nd IATA Technical Conference,Montreal, Canada.

Percentage

Speed of ChangeMEL Carry Overs

Security DeficienciesPoor Management

Poor Nav. AidsMid Air Collision

DeregulationLow Experience

Low Maintenance StandardsComplacencyPoor Training

Sydney AirportAirspace ManagementPolitical Interference

FatigueMisdirected Regulation

Increased TrafficATC Inadequacies

Economic RationalisationCommercial Pressure

What do you consider to be the greatest future threat?

Percentage



(2) Reason, J., (1990) Human Error. Cambridge Univer-sity Press, Cambridge, England.

(3) Lowrance, William, W., (1976) Of Acceptable Risk.William Kaufmann Inc., Los Angeles, California.

(4) Potts, Rodney, J., (1991) Windshear Observations InTropical Australia. Paper presented to the FourthInternational Conference on Aviation WeatherSystems, June 24-26, Paris, France.

(5) Spillane, K. I., and Lourensz, R. S., (1986) TheHazards of Horizontal Windshear to Aircraft Operationsat Sydney Airport. BMRC Research Report No. 3,Melbourne, Australia.

(6) Russell, Paul, D., Chief Engineer, Airplane Safety,Boeing Commercial Airplane Group. Personalcorrespondence, September 1994.

(7) Smith, Dick, Ex Chairman of the Australian CivilAviation Authority. Personal correspondence,February 1994.

(8) Hofstede, G., (1980) Culture’s Consequences; Interna-tional Differences in Work-Related Values. Sage Publi-cations, Beverly Hills, California.

(9) Trompenaars, Fons, (1993) Riding The Waves OfCulture - Understanding Cultural Diversity In Busi-ness. Nicholas Brealey Publishing Ltd., London,England.

(10) Patience, A., (1991) Softening The Hard Culture.Mental health in Australia, December, pages 29-35.

(11) Lewis, Ken S., Director of Safety and Environment,Qantas Airways. Personal Correspondence, January1995.

(12) Ward, Russel (1982) Australia Since the Coming ofMan. Lansdowne Press, Sydney, Australia.

(13) Ashford, Norman, J., Professor of Transport Plan-ning, Loughborough University. Personal cor-respondence.

(14) Barnnett, A., M. Abraham, and V. Schimmel. (1979)Airline Safety: Some Empirical Findings. ManagementScience, Volume 25, Number 11, November, pages1045-1056.

(15) cited in Taylor, Laurie. (1988) Air Travel - How Safe IsIt? BSP Professional Books, Oxford, England.

(16) ISASI Code of Ethics, (1983) Member’s Handbook,ISASI Internal Publication.

Graham Braithwaite is a postgraduate researcher fromLoughborough University’s Aviation Safety Group, UK.Supported by the Rotary International Foundation and thegenerosity of a number of airlines, his research has included ayear in the field in Australia.


The determination of the flight track or path of anaircraft has changed considerably in recent years. Plot-ting from radar recordings has gone from DART/DLOGto ITAP to NTAP/CDR. The information derived whencombined with transcripts and audio recordings of Air/Gnd, Gnd/Gnd, and CVR can shed a great deal ofinformation on the actions of a flight crew and thekinetics of an aircraft just prior to an accident or incident.The aircraft type, the equipment, and pilot capability allenter into the equation of accident causation. The afore-mentioned are truths all of us as investigators under-stand.

One of our tools of investigation, the CVR recording, isvalid for air carriers required to have CVR/(D)FDRrecorders, and for the more generously equippedcorporate machines. Most investigations of generalaviation aircraft accidents and incidents do not have theadvantage of that nice source of information—thecockpit conversation.

We have experienced a large number of General Avia-tion accidents involving the pilot’s loss of control.Because of the lack of CVR information, many of theseaccidents have been attributed to “pilot error, due topilot disorientation.” The correlation of the radar dataand the aircraft aerodynamics, combined with theaircraft equipment, presents an unmistakable definitionof many of these accidents that can easily be overlooked.

There is a repeating scenario that shows that an equip-ment malfunction, which can not be overcome by thepilot, may be the most probable cause of many of theseaccidents attributed to pilot error. This by no meansdiscounts those disorientation accidents that stem frommechanical failure or lack of training or flight currencyon the part of the pilot and are truly “pilot error.”

An aircraft in flight always responds to the aerodynamiclaws of physics. When combined with the flight track,the combination begins to tell us something about thecrash causation. Of course, the attitude at impact must betaken into account along with the flight path.

To review, an aircraft has known motions around thethree flight axes: ROLL, PITCH, and YAW. For a pilot,these motions are basic and learned early in a flighttraining syllabus. There are factors that affect these

motions that are taken into account by the designers, butare only partially understood by most pilots. One ofthese factors, flight stability, has two types—dynamicand static. Each type is expressed as stable, neutralstable, or unstable. The measure of stability is the ten-dency and rate of return of an aircraft from a disturbedcondition of flight to its previous state of flight equilib-rium.

As an example, while a deviation in either ROLL orYAW will usually result in a couple between the two, thelateral stability combined with the directional stability ofthe aircraft predicates the return to balanced flight. If rollis extreme, PITCH excursions can result and the equationthen becomes complex.

Along the longitudinal axis, a properly designed andcertificated aircraft has a positive PITCH static/dynamicstability; this tends to return the aircraft to its trimmedstate of balanced flight. When an aircraft experiences aPITCH excursion, i.e., nose down from a gust or slightcontrol movement, the airspeed will increase and thenose attitude will oscillate up and then down, in amotion known as a phugoid, and, with no pilot input,the aircraft pitch attitude will return to the original state.Longitudinal pitch stability is very strong for the certifi-cated aircraft.

Power plus attitude establish a state of balanced flightequilibrium based on the PITCH trim condition of anaircraft. The crash site parameters that indicate finalpath/impact in a steep nose down attitude can be theclue for the necessity of determining the airborne flighttrack and aerodynamic conditions just prior to the upset.

A number of accidents have involved an aircraft on aconstant heading, in normal level flight or in an attitude,with a relatively moderate rate of vertical movement.Generally, there is a minor change in altitude or ratedeviation followed by a very high rate of descent andincreasing airspeed. The aircraft and its transponder maygo below the radar horizon in one or two sweeps of theradar. To manually achieve such a profile, the pilotwould have to be pushing on the yoke and maintaining avery high stick force. The aircraft could also be subjectedto an adverse trim condition. Aerodynamic stability tellsus that if the aircraft departs from a normal flight equi-librium attitude under normal trim conditions, the

Impact Reconstruction: Flight Path to Causation

Paul Bray, Jr. M02072President, Aviation Technical Consultants, Inc.

Westport, CT 06880

Jack Lipscomb, M02290President, Lipscomb Associates

Moneta, VA 24121


aircraft will return to that normal flight attitude. No pilotwill manually trim an aircraft to an extreme conditionthat may require many tens of pounds of stick force tomaintain normal level, climbing, or descending flight.

To continue the process, we must now consider theequipment aboard the aircraft. An electric pitch trim andan autopilot, either analog or digital, are usually in-cluded on the equipment list of most sophisticatedgeneral aviation aircraft. Most autopilots have an“autotrim” feature that keeps the servo control actuationforce well below the required servo clutch force setting.In this manner, the autopilot acts just like a pilot; it justdoesn’t want to hold a high stick force. The problembegins when the autotrim circuitry receives a com-manded but unrequired pitch trim signal. The pitch trimservo may drive the aerodynamic trim toward an ex-treme, while the pitch servo will hold the nose of theaircraft opposite to maintain the initial flight status. Trimwill always win, and, at some point, the pitch servoclutch will start to slip and will not be able to overcomethe aerodynamic pitch trim setting. The autopilot willnot maintain the desired flight condition, and the pilotmay notice a continuing flight path deviation, mostlikely of not great proportions, but enough to indicatethat something is amiss. The decision will be made todisconnect the autopilot.

Now, when the pilot disengages the autopilot and thepitch servo is no longer applying a stabilizing force, theaircraft will be in an extreme pitch trim nose condition.The yoke will fly against the stop, and, if nose down,occupants will come off their seats, probably with theirheads against the ceiling, and will be hanging on theirseat belts. The aircraft will be accelerating rapidly, andthe force to return and maintain any semblance of levelflight could be far in excess of fifty pounds. On someaircraft, under specific stated conditions, the manufac-turer has even calculated stick forces to exceed onehundred pounds. For those of you calculating in kilos,that is almost fifty kilograms. That is a lot of force whenthe pilot may not even be in a position to apply the forcenecessary to affect the recovery.

The key in the stability scenario is the aircraft tendencyto return to the TRIMMED state of flight. However, if thetrimmed condition of the aircraft is at or near eitherextreme of nose up or down, the resulting motions of theaircraft may be beyond the pilot’s physical ability toovercome. The aircraft will not return to a normalbalanced state of flight, but, rather, to its trimmed pitchstate based upon the attitude/power combination. Theaircraft attitude may result in either a high nose upcondition, approaching stall, or steep nose down withbuilding airspeed. In the nose up scenario, after the stall,the resulting post-stall gyration may well create a totallyout of control flight situation and/or result in a true spin(1 ‘g’). In nose down, the aircraft may strike the groundbefore control is reestablished, or the aircraft may exceed

Vne and result in an IFAFF through control input orother aircraft motion. Either scenario is potentiallydisastrous to machine, crew, and passengers.

In real life, under hand control flight, both pitch sce-narios seem nonexistent because the first thing a fledg-ling flight student learns is how to “trim.” This is furtheramplified during instrument training as the secret isTRIM—TRIM—TRIM for every attitude established for agiven flight regime. Nobody rolls trim to abnormal orextreme positions, because the pilot must turn into agorilla holding an extreme stick force to maintain controlof the aircraft. However, the aircraft equipment can placethe aircraft in just such a condition.

While the maximum controllable stick force for pitch isconsidered to be fifty pounds, pitch trim excursions canresult in over one hundred pounds of stick force. Onehundred pounds of stick force is absurd: only 35-40pounds can easily surprise and tire a pilot in short order.

Some aircraft autopilot systems employ the attitudeindicator (gyro) as the source of the pitch and rollstabilization. The attitude indicator becomes a criticalcomponent in the behavior of the pitch trim system. Ifthe gyro starts to precess or tumble, the input to theautopilot will result in the aircraft attempting to followthe destabilized gyro. An intermittent or completeinterruption of a drive signal can also result in theautopilot pitching or rolling, or a combination of both,while the autopilot is engaged. This can present a seriousproblem in some autopilot systems.

There are some attitude indicators in autopilot systemsthat have a very low reliability rate. One million dollaraircraft has had eight attitude indicators installed inthree years, of which three tumbled in flight. Anothermillion dollar aircraft has experienced numerous pitchproblems, including a pitch over on ILS approach. Everycomponent of the autopilot system on this aircraft hasbeen changed out, and the entire aircraft wiring has beenreplaced. To date, the aircraft is still experiencing pitchproblems. So, people who change out individual compo-nents to correct intermittent pitch problems must be-ware.

One of the most popular autopilot systems has a char-acteristic that precludes a servo from physically dis-engaging when the system is electrically disengaged ifthere is pressure being applied to the yoke when thedisconnect button is depressed. If an aircraft, on autopi-lot, pitches nose down for any reason, the naturalreaction of a pilot is to grab the yoke and pull to counter-act the pitching. If the pull is prior to the depression ofthe disconnect button, the pilot will receive all the visualindications that the autopilot is disconnected when, infact, the pitch servo is still mechanically engaged. Theautotrim system still has electrical power, so it will beginto trim against the pulling force and will drive the pitch


trim further in the nose down direction. The airspeedwill continue to build, and the forces required to raisethe nose will eventually exceed the capabilities of thepilot.

It is absolutely normal for a pilot experiencing an un-commanded nose down pitch to grab the yoke and pullprior to actuating the disconnect switch, especially at lowaltitudes. Now the yoke must be pushed in the directionof the upset to allow the disengagement to take place.This is opposite to any natural reaction. Who wants topush the nose further down, when you really want toraise it to a normal safe attitude.

The human factors implied with an annunciation indi-cator marked “trim fail,” along with a sophisticatedpreflight regimen, are such that a pilot may be lulled intobelieving any adverse trim excursions will be indicated.This may not be the case: a close inspection of conditionsunder which the fail indicator will illuminate is required.One autopilot trim fail light will come on under onlythree conditions:

1. Failure of pitch trim to run when commanded.2. Pitch trim running when uncommanded.3. Pitch trim running in opposite direction of that

commanded.

A failure may occur when the pitch trim, uncommanded,thinks it should run and does, and no indicator will light.

An accident can simply be the result of a short in theyoke elevator trim switch, which today is supposed to bea double fail-safe type, or in the wiring going to theswitch. However, it must be noted that there are stillmany single yoke trim switches still flying. Such a short,occurring during take-off phase of flight when the pilotis busy with the transition from ground roll to flight, canbe masked by a cross wind, the runway condition, orrotation of the aircraft. The elevator forces can becomevery heavy as the aircraft accelerates, and under someenvironmental conditions, night or weather, can result inground impact shortly after takeoff. Again, the key isgoing to be the aircraft attitude, elevator, and pitch trimconditions at impact.

This autopilot/autotrim type of accident is very differentfrom that of the spatial disorientation type. In the spatialdisorientation upset of a pilot, either instrument or non-instrument trained, the flight track will contain a hap-hazard series of climbs, descents, or turns as a prelude.The trim may or may not be in a normal range.

The flight path of a mistrimmed aircraft, as well as thatof a spinning aircraft, can result in a steep nose downimpact. Each entry and impact pattern, however, has its’own distinguishing indications.

To date, it remains a mystery which specific componentor components may be failing and resulting in the pitchtrim being driven to extreme nose trim conditions. We

may never know, but one thing we do know for certain isthat there isn’t a normal situation where a pilot wouldintentionally place the pitch trim manually in an extremenose up or nose down position.

In many aircraft, the manual trim wheel is not visible tothe pilot, and it may be moving unbeknownst to the pilotunder autopilot autotrim flight control. Any action thatresults in a flight control deviation without the pilot’sknowledge has the potential of being lethal. The best,simplest, and least expensive warning is for the aircraftor autopilot manufacturer to install a TRIM MOTORRUNNING annunciator to alert the pilot when the pitchtrim motor is actuating aircraft pitch trim. Then the pilotwill know when the trim is being applied to an extreme.

Under autotrim, the pitch trim motor runs for only asecond or a second and a half until the pitch servo forceis relieved. Any long term running of the trim motor canbe observed by the pilot, and the trim motor manuallyturned off before an out of trim condition is reached.Most air carriers have such an indicator: why not generalaviation. After two MU2 accidents in Australia, thatcountry’s Bureau of Air Safety Investigation (BASI) hasalso recommended such an indicator be added.

The foregoing scenarios are typical of several accidentsthat have resulted in a number of aircraft disappearingfrom radar and impacting the ground at very steepangles (eighty degrees) for no apparent reason.

Most, if not all, of these accidents have been attributed topilot error, because the reconstruction of the flight profileand the aircraft trim conditions was not properly relatedto the designed aerodynamic characteristics of theaircraft. There are definite clues that differentiate thevertigo/disorientation accident from the pitch trimmalfunction accident.

If we, as investigators, apply the aerodynamic laws ofstability along with flight path reconstruction, we can tella great deal about the derivation of an upset that leads tothe extreme attitudes of an impact. We can also attemptto move the manufacturers into displaying those “auto-matic” actions that lead to flight upset.

Paul Bray, Jr., received a B.S. from Rensselaer PolytechnicInstitute in Troy, New York. He has been involved in aviationfor 52 years—as pilot, a mechanic, and a flight instructor.ATP, CFI, Gnd. Instr., A & P, FAA Accident PreventionCounselor at Large. He has produced, managed, and directedover 100 aviation training films: USN, USAF, AOPA ASF.Organized stall/spin training syllabus.

Jack C. Lipscomb received a B.S. from the US MerchantMarine Academy in Kings Point, New York. He is a formerNTSB Air Safety Investigator. Currently he is a seniorInstructor at the National Accident Investigation School.Designated Naval Aviator. ATP, CFI Rotorcraft.


A valuable source of information for achieving the goalof improving aviation safety is incident data. Not onlydo incident data provide a metric of aviation systemsafety, but they also offer insights from incident partici-pants as to the underlying factors, sequence of events,and conditions associated with safety anomalies. Inaddition, incident data provide critical information onhow the incident was resolved and an accident avoided.These data can, and should, play an important role increating policies and procedures for the safe operation ofaircraft and air traffic control.

In tracing the causal chains of aviation accidents, safetyinvestigators and researchers have generally beeneffective in determining what happened. They have beenless effective in determining the why of events—whypeople acted as they did, why a system failed, why ahuman erred. Incident data are a unique means ofobtaining first-hand evidence on the factors associatedwith mishaps from event participants themselves.

Reporters are willing to admit errors, and quite often areable to identify the sequence of events that resulted in anincident. Reporters provide insights into both theirperceptions and their correct and erroneous behaviors.They describe the predisposing relationships betweenstimuli and their actions. They also provide their inter-pretations of the effects on human performance of factorssuch as fatigue, interpersonal interactions, and distrac-tions. Many reporters are able to offer valuable sugges-tions for remedial action.

When a safety hazard is suspected, incident data canoften provide facts that prove or disprove its existence.These data are ideally suited for understanding thepossible causes of safety hazards, defining potentialintervention strategies, and tracking the safety conse-quences once intervention has begun. An incident reportmay paint a complete picture of the safety issue andsuggest remedial action, or it may identify a hazard thatmay not have been known through other sources. Forexample, the markings along a taxiway may not clearlyidentify the ‘hold short’ position for an active runway.More often, however, safety issues are more insidious.This paper will focus on those instances where a study isrequired to distill the underpinnings of safety problems,and investigate the optimal remedies for these situations.Incident data can provide valuable insights that arelacking from other information sources.

This paper addresses incident reports that are submittedvoluntarily and confidentially. Its aim is to facilitate theuse of incident data as a rich source of information inaeronautical safety, and to promote the use of these databy policy makers, resulting in an increase in aviationsafety. (For a more complete treatment of this topic, seeChappell, 1994.)

Using Incident Data

The Incident Data

Aeronautical incident reports come from many sources,including pilots, air traffic controllers, flight attendants,mechanics, and airport ground personnel. Databaseincident records generally contain key words withincategories that can be searched: the role of the reporter(e.g., pilot or air traffic controller), the type of incident,where and when it happened, the type of weatherconditions, the role of the other individuals involved andtheir qualifications, and the type of aircraft and flightoperation. In addition, the database records often includea description of the incident in the reporter’s own words,as well as the circumstances that contributed to the lossof safety. If multiple reports of the same incident arereceived from different individuals, for example, eachflight crew member and air traffic controller, theirdiverse perspectives may provide a better understandingof the incident. In addition to incident reports, reportersprovide information on unsafe conditions that have notresulted in an incident, but have the potential to do so.This information is extremely valuable in preventing anincident and, more importantly, an accident.

The Strengths of Using Incident Data for Evaluations

Information from participants. Incident data providedetailed information from the participants in the events.This first-hand information is often not available follow-ing an accident. Self-reported incidents have manyadvantages over accident data. The reporter can providevaluable insight into the factors that contributed to anunsafe situation. This information is in the form ofnarrative descriptions of the sequence of events from theperspective of the flight crew or other participants.Telephone conversations between the reporter and thesafety analyst can, when necessary, augment and clarifythe information originally submitted. Descriptions of thesame incident by a flight crew and participating air

Using Voluntary Incident Reports for Human Factors Evaluations

Sheryl L. ChappellNASA Ames Research Center

Moffett Field, CA USA


traffic controllers can provide an even fuller understand-ing of the event chain.

Large numbers of observations. Fortunately, aviationaccidents are rare events. Accidents are investigatedmore thoroughly than incidents, and many valuablelessons have been learned from these tragedies. How-ever, the limitations of small samples exist when usingaccident data. The wrong conclusions may be drawn, orfeatures of the real picture may be obscured. Incidentdata provide a large quantity of detailed informationprovided by the participants in the events. For example,at the time of writing, the largest voluntary incidentdatabase in the United States, the Aviation Safety Re-porting System (described in Reynard, Billings, Cheaney,& Hardy, 1986) held 203,120 reports of which 1848described altitude deviations occurring in the previoustwelve years. The availability of such a large sample ofreports on many topics allows the investigator to beselective in choosing those that will be studied in depth.Only reports that contain an adequate level of detail maybe used for analysis.

Ecological validity. Another advantage of using incidentdata in human factors evaluations is that these incidentsoccurred in the real-world operating environment. Inother words, the incident is ecologically valid, and notmerely a laboratory phenomenon. A severe shortcomingof laboratory research is its lack of context. Many times, astrong finding in a laboratory study dissolves in the realworld, where the stimuli are very rich, and tasks aresubject to interruption and time sharing. The importanceand influence of contextual effects are increasinglyrecognized by behavioral scientists (see, for example,Vicente, 1990).

The Limitations of Using Incident Data

Information not validated. In some countries, voluntary,confidential reports can be fully investigated, andinformation from other sources brought to bear on theincident. In other systems, confidentiality precludes anyadditional investigation and reports go unverified. Evenwhen reported information cannot be substantiated, thereports are reviewed by experts in the various domainsof aviation, e.g., flight crew performance. If large num-bers of reports on a topic are available, it is reasonable toassume that consistently reported aspects are likely to betrue. It is doubtful that a large number of reporterswould exaggerate or report erroneous data in the sameway.

Reporters may have a tendency to understate their errorsand blame the occurrence on other parties. Incidents alsomay be embellished to benefit the reporters. For ex-ample, the controllers at an airport tower facility may bemotivated to report every traffic conflict to support theaddition of air traffic personnel, which might result indecreased workload. When these reports are analyzed,

these factors are usually very apparent to the experi-enced report analysts, and their suspicions can bereflected in the analyses of those reports.

Reporter Biases. There are two factors that bias vol-untary incident data: who reports and what gets reported.The demographics of who submits reports results in afaction of the aviation community being over-repre-sented in the incident database. Reporters must befamiliar with the program, they must have access toreporting forms or phone numbers, and they must bemotivated to report. A pilots’ or controllers’ organizationmay support the reporting system and make reportingforms available to its members. Portions of the pilot andcontroller community that are not members of theorganizations may find it more difficult to contributesafety reports.

What gets reported is influenced by several factors. First,reports will reflect the reporters’ subjective views andperceptions that result from their perspective of andcontribution to a situation. This is both a strength and aweakness of self reporting. It is a strength in that thedescription of the reporters’ perceptions can be enlight-ening as to causes of safety problems. It is a weakness inthat the user of the data can sometimes not separate thesubjective from the factual components of the report.

Which events get reported can also be subject to biases.These biases can affect the type and the number ofreports received. When an individual makes an error in aprocedure, generally that individual takes responsibilityfor ensuring that the error is not repeated. If there wereno significant consequences of the error and regulatoryimmunity is not needed, the individual derives littlebenefit from reporting this type of event. However, whenanother individual makes an error, the only method foraffecting a solution may be to report that error.

Biases also stem from what is not reported. Reportersmust be cognizant of a safety factor to submit a report.Errors that go undetected are not reported. It is unlikelythat an optical illusion would be reported, unless it wasidentified through some means. There is also a bias toreport only operational problems, not safety improve-ments. When a new procedure goes into effect that curesa hazard, only the decline in incident reports marks thatimprovement. If the new procedure benefits one factionof the industry (e.g., the pilots) and creates problems foranother (e.g., the controllers), only the controllers arelikely to submit safety reports. The old adage that nonews is good news generally applies to safety data.

Using Incident Reports—The Process

Scoping the Issues

Aeronautical human factors studies begin by setting thescope of the project. The dimensions of the topic must be


explored to identify the pertinent predisposing factors.Incident data are ideally suited for this process, in manytypes of study. This initial phase is devoted to under-standing the issues to be investigated, formulatinghypotheses, and selecting the best methodology toevaluate the issues identified. Incident data can also helpsubstantiate the need for further study on a topic. Theoriginal concept of Crew Resource Management wasdefined through the study of incident reports describingmanagement, communication, and leadership problems.Once the concept evolved, incident data suggested aneffective orientation for the program. Incident reportswere also used as examples in communicating thisimportant concept to the operational community.

Few studies have the luxury of being well specified atthe onset, and those that are well-specified are some-times found to be off-target once the study begins. Forexample, a recent study began as an investigation offlight crew perceptions/misconceptions of the mechani-cal systems on advanced technology aircraft. This studywas to culminate in a full-mission simulation. A surveyof incident data revealed few misconceptions by flightcrew members regarding mechanical systems; however,there was evidence of much confusion about the auto-flight systems. The project was redirected to address thisimportant safety issue. Other studies have begun byattacking a specific issue where safety is believed to becompromised. Upon delving into the topic using incidentdata, other facets of the safety issue were discovered. Forexample, a study of aircraft deicing problems (Sumwalt,1993) discovered that, in addition to the problems ofgetting aircraft deiced within a few minutes of takeoff,there were a substantial number of problems in detectingthe ice. One impediment to detection was the deicingfluid covering the cabin windows. Another factoremerging from examination of the icing incidents wasthat pilots erroneously believed that the snow wouldblow off on takeoff.

Studies of all types can benefit from the use of incidentdata to scope the issues. One airline, in an attempt toreduce the number of altitude deviations by its pilots,evaluated voluntarily submitted altitude deviationreports, and discovered that a common element was theincorrect setting of the altitude in the altitude window.By requiring both pilots to verify the setting of thatvalue, the number of altitude deviations was dramati-cally reduced. Aircraft simulation studies on the use ofnew information displays can use incident data tounderstand how the tasks of interest are currently beingconducted, and to identify the shortcomings of theexisting technology. Additionally, dependent measuresmay be suggested by the incident reports. A study ofpilot performance in response to a traffic collisionavoidance system (Chappell, et al., 1989) began with asurvey of the evasive actions reported by pilots. Theseactions were used as a baseline in evaluating the efficacyof the collision avoidance system.

Retrieving the Data

The first step in an evaluation of incident data is toretrieve the data. This can either be accomplished by arequest to the agency that is responsible for managingthe database, or by performing the database searchoneself. The databases containing voluntary incidentreports commonly have both data fields and textualdescriptions, e.g., time of day of the occurrence andnarrative information about the sequence of events in thereporter’s own words, respectively. Both the data andtext are usually searchable to retrieve relevant safetyreports. If a topic is not directly addressed by the data ortext fields, an iterative approach can be employed. Thereports resulting from the first search can be screened todetermine which search characteristics yielded the mostapplicable reports, i.e., which terms applicable reportshad in common that might be used to retrieve others.These terms can then be used for the second search, andthe process repeated for further refinement, until thedesired number of reports is retrieved.

Reports retrieved that are not relevant (false positives)present a nuisance, but not a research problem. Thesecan be screened in the compilation process and dis-carded. However, it is quite reasonable to assume thatthe search strategy, unless very straightforward, willresult in false negatives—relevant reports that are notretrieved. These present a significant problem for theinvestigator. The nature and significance of the reportsnot retrieved is not known. As a simple example, anindividual may wish to evaluate how close aircraft cometo each other in traffic conflicts occurring in the proxi-mate area of airports. If a search is performed and onlythose incidents labeled ‘airborne conflicts’ are retrieved,the yield may be limited and the investigator willconclude that aircraft do not pass very close to eachother. In reality, safety analysts commonly use the term‘near mid-air collision’ to describe those incidents withextremely close miss distances. These reports would nothave been retrieved with the above search strategy.

Caution should always be used when employing inci-dent data to determine the prevalence of a safety prob-lem. The failure to retrieve a substantial number ofreports on a particular topic does not necessarily meanthere are no reports on that topic in the database. Thenumber and nature of false negatives or misses cannot beknown.

Compiling the Data

When performing an evaluation using incident data, theinvestigator commonly must subject the incident reportsto further analysis than was performed in the initialcoding. The extent of effort involved in this process willvary greatly from study to study. If the number ofreports on a particular topic is of primary interest, thatnumber may be tallied with little or no further analysis.


If, on the other hand, the underlying causes of a particu-lar human error are of interest, the reports will requirefurther scrutiny. An additional coding process may benecessary. This is generally very time-consuming, andcould possibly threaten the resources available for otherphases of the project.

The standardized narrative is a technique of recodingincident reports that was developed by Thomas (Thomas& Rosenthal, 1982) in a study of altitude deviations. Thistype of coding enables the investigator to retain much ofthe structure and content of the reporter’s narrative,while constraining the information sufficiently to usequantitative techniques. To construct a standardizednarrative, the reporter’s description of the events isrearranged and rephrased to fit one of several standardsentence types. As successive narratives are interpreted,the number of standard types is adjusted to preserve thelevel of detail that is appropriate for the study. Thepower of this technique is that elements can emerge thatwere not anticipated. This is not necessarily true instudies where a rigid coding is performed.

Another technique has proven to be useful in providingmore information than the reporter originally submittedin the incident report. This technique involves a struc-tured telephone interview with the reporter. This obvi-ously must be performed before the data are de-identi-fied. The heart of the structured interview is a carefullydesigned questionnaire. Questions are designed to beneutral, i.e., written so as not to elicit a particular re-sponse. An analyst conducts several trial interviewsduring the design of the questionnaire to evaluate theadequacy of the questionnaire, and to assess reporters’reactions to participation in the study. After the ques-tionnaire is finalized, the analysts employ it to conductthe telephone interviews. Comments in response to eachquestion are collected, paraphrased (if the response islengthy), and incorporated into a database. The question-naire may encompass specific incident details and/or itmay range beyond event-specific data to probe areas ofspecial interest to the investigator.

Analyzing Incident Data

Once incident data have been retrieved and compiled,the analysis can begin. Analyses of incident data maytake many forms. There are both quantitative andqualitative techniques that are useful in evaluating safetyissues (see later sections).

As with any analysis, the most important step is toformulate meaningful null and alternative hypotheses.This is often a precarious task for incident data evalu-ations. The hypotheses must be couched in terms of theprobability of an event being reported, not in terms ofthe probability of its occurrence. Many times it is in-appropriate to apply inferential analysis techniques; only

descriptive statistics are valid. (For a discussion of theapplicable statistical tests, please see below.)

Drawing Conclusions

When drawing conclusions from the analysis of volun-tary incident data, one must be very cautious. It is notappropriate to infer the prevalence of incidents. Aninvestigator has no reliable knowledge of the totalnumber of incidents occurring, only of the number beingreported. An investigator can also have no knowledge ofthe nature of those incidents that are not reported. Forexample, the database may hold more traffic conflictsthat occurred at airports with operating air traffic controltowers than at airports with no control tower. One coulderroneously conclude that a traffic control tower resultsin more traffic conflicts. There are several faults in thislogic: correlation versus causality; normalization bynumber of opportunities (usually the number of flights);and reporting biases.

The most important factor in drawing conclusions fromanalyses of voluntary incident data is that the rela-tionship between reported incidents and all incidentsthat occur is not known. The reports are not necessarilyrepresentative of the population of events; therefore, thevalid conclusions can only relate to the incidents re-ported, not to the total population of events. It is neces-sarily true that if an attribute is reported, it will also bepresent in the population of all events, in some unknownnumber. This presence in itself may be of sufficientconsequence to draw important conclusions regardingsafety.

Analysis Methods for Incident Reports

Qualitative Uses of Incident Data

A case study of a single incident may be all that isrequired to change a cockpit or an air traffic procedure.Often, however, many reports are required for ananalysis to determine the scope of an event type, or theextent to which these events represent a safety hazard. Asuspected hazard can also be confirmed by incidentreports. Note that the failure of a database search toproduce incidents of a particular type must not beinterpreted as conclusive evidence that the problem doesnot exist. (See the previous discussion on the retrieval ofincident data.)

Quantitative Uses of Incident Data

The most common, and often the only valid, quantitativeanalyses of incident data are descriptive, rather thaninferential. (See the next section on statistical tests.)Probably the most salient analysis involves a categoriza-tion of incident types and a report of the relative fre-quencies of those types. Several caveats apply whenconducting a study of the relative frequencies of incident


types. First, an event of each type must be equally likelyto be included in the reports retrieved on the topic ofinterest. As described earlier, the search strategy used toretrieve the reports should not bias the likelihood of onetype of report being selected over another type. Second,care must be taken in determining the level of aggrega-tion of categories so that the frequencies are largeenough to be meaningful, while preserving essentialdetails. The appropriate level of aggregation of the dataunder study should be determined by the data and thetarget audience. Third, if the categories are not mutuallyexclusive—for example, if a report has several types oferrors described—this must be considered in the evalua-tion. This may change the unit of measure. In thisexample, the unit of measure becomes errors, not re-ports.

A powerful technique when exploring a database forsafety issues is to use multiple methods of looking at thedata. Hazards are sometimes invisible unless viewed in aparticular way. For example, it may not be possible touncover a safety problem when looking at all the events,but a distribution by location may reveal that there arean unacceptable number of occurrences at a particularlocation, while other locations have a good safety record.It is valuable to take many cross sections through thedata to uncover factors that may be critical.

Statistical Tests for Voluntary Incident Data

Given the nature of voluntarily submitted incident data,the number and type of valid statistical tests is extremelylimited. Often it is inappropriate to test hypotheses; onlydescriptive statistics are valid. Many statistical tech-niques customarily employed in the human factors field,such as analysis of variance, are often not appropriate foruse with voluntarily submitted incident data. The datacannot, in general, be assumed to be normally distrib-uted. The data categories are often not mutually exclu-sive. Many parameters are not continuous or evenordinal. The population is the incidents that are reported,not the incidents that occur. These attributes of voluntar-ily submitted incident data pose a statistical challenge forthe investigator.

In describing a set of data from a voluntary incidentdatabase, the frequency and percentage of a particulartype of report are useful statistics. For example, perhaps40% of all reports received from pilots and controllersdescribe a pilot deviation from an assigned altitude. Ofthose altitude deviations, maybe 87% involve an over-shoot/undershoot when climbing/descending to anewly assigned altitude. Perhaps 10% involve excursionsfrom level flight at an assigned altitude, and 3% fall intothe ‘other’ category. These percentages may be useful tothe developer of an algorithm for alerting pilots toaltitude deviations. Such statistics may first providesupport for a need to improve existing systems, since therelative frequency of this type of incident is high (40%),and, second, the developer may optimize the alerting

algorithm for climbing/descending, rather than levelflight, due to the prevalence of those incidents.

Analysis of proportional frequencies. An extremelyuseful inferential statistic for incident data is the χ2 (chisquare). This statistic can be used to determine if two ormore incident types differ in the proportions of reportsfalling into various classifications. Both the incidenttypes and their classifications must be mutually exclu-sive. In addition, the expected values of the cells mustmeet the minimums for the χ2 distribution.

Analysis of trends. To monitor safety, it is often desir-able to detect if a particular type of incident is on theincrease. Techniques used in quality control are appli-cable for detecting trends in incident data. The rate ofreports describing a particular incident type can beplotted over time, e.g., by month. These data will followthe Poisson distribution. The data should vary aroundthe mean of the database to date, within ±2 standarddeviations (2 times the square root of the mean rate). If adatum exceeds this limit, e.g., it is higher than expected,it should be investigated. Naturally a high number ofreports for a particular period may be due to chance.

Sources of Aeronautical Incident Data

The largest source of voluntary aeronautical incidentdata is the Aviation Safety Reporting System (ASRS),operated by the National Aeronautics and Space Admin-istration in the United States (see Reynard, et al., 1986,for a description). This organization provides informa-tion on a wide range of safety issues. Requests for ASRSdata should be directed to:

Aviation Safety Reporting SystemPost Office Box 189

Moffett Field, California 94035-0189 USA

At the time of this publication’s printing, systems similarto ASRS have been instituted in Great Britain, Germany,Canada, Australia, New Zealand, and South Africa. TheInternational Civil Aviation Organization also providescopies of its reports on accidents, although these must berequested by government agencies.

References

Chappell, S.L. (1994). “Using voluntary incident reportsfor human factors evaluations.” In N. Johnston, N.McDonald & R. Fuller (Eds.), Aviation Psychology inPractice. Aldershot, England: Ashgate.

Chappell, S.L., C.E. Billings, B.C. Scott, R.J. Tuttell, M.C.Olsen, & T.E. Kozon. (1989). Pilots’ use of a Traffic-alert andCollision Avoidance System (TCAS II) in simulated air carrieroperations, (NASA Technical Memorandum 100094, Vol. Iand II). Moffett Field, CA: National Aeronautics andSpace Administration.


Reynard, W.D., C.E. Billings, E.S. Cheaney, & R. Hardy.(1986). The Development of the NASA Aviation SafetyReporting System (Reference Publication 1114). MoffettField, CA: National Aeronautics and Space Administra-tion.

Sumwalt, R.L., III (1993). “Air carrier ground deicing/anti-icing problems.” Proceedings of the Seventh Interna-tional Symposium on Aviation Psychology. Columbus, OH:Ohio State University.

Thomas, R. & L. Rosenthal. (1982). Probability distributionsof altitude deviations (NASA Contractor Report 166339).Moffett Field, CA: National Aeronautics and SpaceAdministration.

Vicente, K. J. (1990). “A few implications of the ecologicalapproach to human factors.” Human Factors SocietyBulletin, 33(1 1), 1-4.

Sheryl L. Chappell is the Principal Scientist of the AviationSafety Reporting System at the National Aeronautics andSpace Administration’s Ames Research Center, Moffett Field,California. She is also the President of the Association ofAviation Psychologists. She is an instrument-rated commer-cial pilot and flight instructor.


The purpose of this presentation is to focus on theimportance of having an experienced pilot representativeassigned to an international accident investigation unitconducting an ICAO Annex 13 investigation. It will giveexamples of current US National Transportation SafetyBoard policy, and procedures regarding the assignmentof trained line pilots to the investigation team. It willthen expand this concept into the international arenaand, specifically, in the Asia Pacific Region.

Accident investigations in the United States normallyallow professional line pilot assignment to an accidentinvestigation. This is usually through the “party” system.This procedure has both its good and, in my opinion, badpoints.

Good points:

• Many different parties are part of the factfinding portion of the investigation.

• These parties are encouraged and allowed tomake submissions to the NTSB for consid-eration prior to the final report.

• The investigating authority is primarily chargedwith an impartial investigation with an empha-sis on recommendations to prevent futureoccurrences. Its focus is not to apportion blameor liability. It is a totally independent andseparate branch of the government.

Bad points:

• The official findings, conclusions, and recom-mendations are formally announced withoutprior feedback from the parties. This sometimesresults in a report that is less than complete or,perhaps, does not address some importantissues.

• There is no chance for the parties to review a“draft report” prior to final publication.

• There is no specific responsibility to investigateserious incidents.

International accident or incident investigations, includ-ing those in the Asia-Pacific region, do not always havesuch provisions. In many cases, the investigation may bepartially completed by civil authorities with emphasis onpunitive measures and blame assessment. In such cases,this could result in an incomplete investigation, and littlein the way of valid recommendations to avoid futureoccurrences.

Three Observations and a Recommendation

Reflecting back on both my military accident investiga-tion and commercial aircraft accident reports, I haveoften heard similar comments, “Didn’t they ever think toask some experienced line pilots about that finding,conclusion, or recommendation?” or, in the military,“Was this investigation done at headquarters by Colo-nels or Commanders sitting around a desk?”

In the past we have been asking, “Has anyone at theNTSB really flown the line as a commercial airline pilotin the last 20 years?”

Observation Number One

A current experienced line pilot can add a measure ofcredibility to the accident investigation team that is notavailable from any other source. Valid comments suchas, “Yes, I know that is the position of Boeing and ouroperational manuals, but ask any other pilots how theaircraft really performs,” or “We are not trained onvisibility measurements and must generally rely on thereported weather,” or “It is not unusual to be dispatchedwith just two Inertial Navigational units and later findthat one is somewhat unreliable.”

The list could be continued. The greatest value is to besure that the accident investigation team does not eitherchase off on a tangent area of little relevancy or, worse,inadvertently overlook areas of potential air safetysignificance.

Observation Number Two

If my first observation has any validity, then I must alsomake this additional comment.

International Accident Investigation in the Asia Pacific RegionAn Airline Pilot’s Perspective

Captain Richard DuxburyNorthwest Airlines


Having a qualified, experienced line pilot as part of theaccident investigation team is a plus. It could also be aminus. If the individual assigned has absolutely nobackground or training in accident investigation proce-dures it could offset his/her usefulness. Cheap “hipshots” during the investigative process are of limitedvalue. They may occasionally be on target but really donot contribute to the full and (hopefully) impartialinvestigative process. The result could be what we in themilitary once called “incomplete staff work.”

Observation Number Three

I have become increasingly convinced that it is importantto investigate significant incidents (so called “closecalls”) with almost as much vigor as an accident. Thereare numerous examples of aircraft accident investiga-tions that discovered that the fatal problem was previ-ously known but unreported. The TWA Flight # 514crash at Dulles International on December 1, 1974 couldhave been avoided if one or two similar close calls hadbeen immediately investigated and preventative actiontaken.

Thus I believe that a line pilot assigned to an accidentinvestigation should be a trained and experiencedaircraft accident investigator. This requirement should bemandatory if assigned to an international accidentinvestigation. (It is true that such a restriction wouldtend to limit the so-called assignment pool.)

How would such a list be generated? Frankly, it is inexistence today within most commercial pilot communi-ties and the International Federation of Air Line PilotsAssociations (IFALPA). The Accident Analysis Commit-tee of IFALPA has been reviewing applications for yearsand, when appropriate, certifying pilots as accreditedaccident investigators.

Prior to IFALPA certification at least four items mustbe established:

• The applicant must be an experienced line pilot(almost 100 percent certified are Captains.)

• The applicant must be specifically recom-mended by his/her sponsoring organization.

• The applicant must have formal recognizedaccident investigation schooling, often includ-ing both military and civilian.

• The applicant must have prior experience withmajor aircraft accident investigations.

While I have used the example of IFALPA, it is notimpossible for other pilot organizations to developsimilarly trained and qualified pilots. Assuming thisrigorous screening process, the pilots assigned to the

accident investigation team should add a perspectivethat is missing from any other source.

Recommendation

The recommendation itself is easy to state but oftendifficult to achieve. Specifically, I recommend that atleast one experienced line pilot from the involved aircarrier be assigned to any international accident investi-gation team. This would not preclude more than onesuch pilot, but it must be understood that the screeningstipulations mentioned above would be a prerequisite tosuch assignments. This recommendation is in agreementwith the position taken by the International Civil Avia-tion Organization (ICAO). The ICAO accident investiga-tion manual (Doc. 6920, AN/855/4 para 2.2) states:

Normally, specialized technical investigatorsfrom the State conducting the investigation willhead the various Working Groups (GroupChairman) and the membership of such Groupsmay include, as appropriate, experts from theoperator involved, the manufacturers of theaircraft, powerplant and accessories, and fromthe various flight crew representatives and otherinterested parties who can contribute throughtheir technical knowledge and experience.

Achieving this recommendation for US carriers willrequire help from NTSB. Part of this recommendationwould include a request that NTSB become even moreaggressive in its approach to be a part of internationalaccident investigations (accidents involving US carriersor aircraft types frequently flown by US carriers).

I should note that there has been notable NTSB involve-ment in recent international accidents/incidents, includ-ing the Boeing 747 freighter engine separations and thefire and emergency evacuation of a Northwest B747/400in Japan. In this last case, NTSB was instrumental inallowing comments from both Northwest Airlines andthe Northwest Air Line Pilots Association to be for-warded to the Japanese authorities prior to publication oftheir final report.

However, with increased globalization of airlines Iwould encourage NTSB to continue their thrust in theinternational arena of accident investigation. I shouldalso note that both Mr. Bud Laynor and Mr. RonSchleede would quickly point out that NTSB involve-ment in the international arena is usually on a “requestor invitation basis.” It is likely that such investigationsmay be beyond the funding guidelines of the organiza-tion.

It is vital that NTSB pursue active involvement in twoclasses of international accident investigations. First,where a United States flag carrier is involved, NTSBmust participate. When this is not done, such as the mid-


air collision between a Delta 727 and a Cessna atGuadalajara, Mexico, valuable safety information can belost.

The second class is somewhat more controversial,because it involves an offshore accident to a foreigncarrier where the airworthiness of a US product is notinitially suspect. There are cases where such an accidentmay be very relevant to an ongoing domestic safetyinvestigation. For example, shortly after the USAir F-28accident at LaGuardia, an F-100 crashed at Skopje,Yugoslavia under what seemed to be similar circum-stances. Participation in that investigation may havegreatly aided the USAir F-28 investigation.

Speaking as a pilot flying for a major international UScarrier, if additional NTSB funding is needed for thisincreasingly vital air safety area, then perhaps it is timefor all of us to address the issue. Absent these con-straints, we would strongly encourage any country witha significant aviation community to pursue an aggressiveaccident investigation organization, including line pilotinvestigators.

From many viewpoints, the prospect of a major interna-tional accident investigation is a nightmare scenario. Tothe extent that we can ensure that the interested andqualified parties, including trained and experienced linepilots, are part of this time, it will reduce this nightmare.More importantly it will significantly add to the qualityof the investigation and the resultant recommendationsto improve international aviation safety.

Summary

• Experienced airline pilots can add a greatmeasure of credibility to any accident investi-gation team. This routinely happens in domesticinvestigations in the United States.

• Pilots assigned should also be trained andexperienced in aircraft accident investigation.

• It is equally important to have this pilot repre-sentation on international accident in-vestigations.

• NTSB and all member states under ICAOAnnex 13 must maintain and attempt to in-crease their involvement in internationalinvestigations, including experienced line pilotparticipation.

References

1. ICAO Annex 13 (Aircraft Accident & IncidentInvestigation, 8th Edition - July 1994 with Supple-ment dated 28/12/95)

2. ICAO Accident Investigation Manual (Doc. 6920,AN/855/4 para 2.2)

3. IFALPA List of Accredited Accident Investigators

Captain Richard Duxbury is a Northwest Airlines Boeing 747Captain. He has been a pilot with Northwest Airlines for 28years. Prior to that he was a pilot with the United StatesNavy. He is currently the Chief Accident Investigator for theNorthwest Air Line Pilots Association and an accreditedinvestigator for IFALPA.


Introduction

The Air Accident Investigation Tool (AAIT) has twoelements, the Impact Tool and the Flight Data RecorderVisualiser (FDRV). The Impact Tool is a computerprogram which links various aircraft crash-analysiscodes, enables the user to customise the impact to besimulated, and provides advanced graphical means toexamine their output. FDRV takes positional and orienta-tion data to display a Computer Aided Design (CAD)type solid model of an object using commercially avail-able Virtual Reality software. AAIT has been designedfor use by accident investigators.

This paper focuses on the development and use of FDRV.The aim was to produce a data-driven visualisationpackage which would run on a PC, and would provideenough features to allow the user to ‘explore’ the data,and analyse what the the data was showing, rather thanjust viewing an animation. Modern Flight Data Record-ers have a large number of parameters; our currentmethods of display allow each piece of information—CVR, FDR, Radar, datalink messages—to be presented tothe investigator individually. Visualisation could allowall this data to be presented together in one or twodisplays and be combined with audio information.

Background

After the accident to the Boeing 737-400, G-OBME, nearKegworth in January 1989, which had highlightedcrashworthiness and survivability issues, AAIB and theUK Ministry of Defence (MoD) commissioned theCranfield Impact Centre (CICL) to develop a practical‘Aircraft Accident Investigation Tool’ (AAIT). InitialAAIT development work was completed in 1995. Experi-ence showed the need for better representation of theairframe-ground interaction, and this led to the develop-ment of the ‘Soft ground’ module. Similarly, AAIB andMoD interest in helicopter ditchings has led to the ‘Waterimpact’ module.

In April 1995, another component was added to AAIT—the ‘Virtual reality’ module. The PC-based virtual realitymodel within the Impact Tool was developed initally todisplay the impact sequence. This software was thendeveloped to include the facility to display a CAD typemodel, using only positional and orientation data. The

primary use of this FDR Visualiser so far as been for thedisplay of Flight Recorder information.

AAIT—The Impact Tool

The aircraft structural analysis code used is KRASH, a‘hybrid’ code developed for FAA/NASA for aircraftimpact research. The resulting KRASH signals are usedin occupant simulation (SOMTA). KRASH requires amodel of the aircraft to be created which comprises aseries of lumped mass points linked together by deform-able beam elements. The mass points define the weightand inertia properties of the structure, and the beamsdefine the strength and ultimate collapse characteristics.Contact springs are attached to selected mass points torepresent the crush properties of the lower structure ofthe aircraft, which is likely to contact the ground. Thetask of creating the KRASH aircraft model is usually bestleft to an impact dynamicist.

AAIT has a pre- and post-processor. Through the pre-processor, the accident investigator has the ability tomodify the model to allow for any pre-impact damage tothe structure. This is particularly useful in a scenario thathas two impacts. The user defines the speed and orienta-tion of the aircraft at impact, and the program thenpredicts the motion and deformation of the aircraftduring the course of the simulation. The output is thenavailable to the user as either a set of time history plots,3-D wire frames, or through the ‘Virtual reality’ moduleas a simulation of the impact sequence. The pulse signalof impact derived from KRASH can then be used foroccupant simulation (SOMTA). Animated sequences ofthe aircraft occupant can be displayed.

In the post-processor, the sequence of motion of theaircraft during the impact can be visualised using a 3-Dwireframe graphics program; one feature of this allowsthe presentation of contact marks on the ground indicat-ing where various aircraft structures have touched andmarking the path of the aircraft after impact. Thesepredicted marks can be compared with the actual groundmarks on site.

Development of the Virtual Reality Module

In order to develop a more realistic animation of theimpact sequence, it was decided to use a commercially

The Air Accident Investigation Tool — Impact Tool and FDR Visualiser

Anne Evans M03512Senior Inspector of Air Accidents (Engineering),

Air Accidents Investigation Branch


available PC-based Virtual Reality software.The Sense8 WorldToolkit was purchased, andCICL again was used to customise the pack-age using the ‘C’ programming language totake in the AAIT Impact Tool output anddisplay the impact sequence.

The development of PC systems in the VirtualReality market, particularly recently with highspeed graphics cards being available asstandard, has created a powerful platform forvisualisation packages. The WorldToolKit is apowerful set of software tools that allowsusers to develop their own applicationsquickly and easily. Routines are provided forthe import of 3D files from all popular CADformats, such as Autocad and 3D Studio. Italso includes an environment editor, Genesis -World Builder, that allows users to edit theimported models or to create their ownobjects.

The hardware chosen for the Virtual Reality Module wasa Pentium 90MHz with 32mb RAM, and a SPEA FIREgraphics card. AAIT Impact Tool can run on a 486 PC,and does not require any special graphics. However,with the ‘Virtual reality’ module, Impact Models can beviewed either as ‘wireframe’ or filled polygons; featuressuch as shading and lighting have been added, and theuser has more flexibility in setting up viewing positions.

The next step in the development of the ‘Virtual reality’module was to take in data files of position and orienta-tion information, and to manipulate a CAD modelaccording to this visualisation.

FDR Visualiser

The FDR Visualiser takes basic position and orientationdata and displays a CAD type model of the aircraft. It isa data driven method of visualising flight data. TheVisualiser has four input files.

• Data files

The first input file is the .dat file, which is an ASCII filecontaining engineering values of any parameter, andsample rate of the data file. The information required ispositional (x, y, z) data and orientation (pitch, roll, yaw),although additional parameters can be included fordisplay.

Flight Recorder data is used to provide the basic infor-mation, although often positional information has to bederived from the recorded parameters, such as airspeedand heading. Corrections may also be required to allowfor the effect of wind. Some aircraft do now recordlatitude and longitude information that can be displayeddirectly. Integration of the accelerometer values is

another way of calculating the actual position if a start orend point is known. This type of analysis was performedon the FDR data for the accident to a Super Puma G-TIGH and the results presented in the AAIB report (AAR2/93) (1). A visualisation of this accident has beenperformed using the flight profile (x, y, z) positionsderived from the integration of the accelerometers in theaccident report.

Another source of the positional data which has alsobeen used in the Visualiser, is radar.

The other information required in the .dat file—aircraftpitch, roll and heading—are taken directly from the FDRwhere available. These parameters are used to controlthe orientation of the model.

• Aircraft Model files

The next input file for FDRV is the aircraft model, whichis in a suitable CAD format such as DXF. Models cangenerally be purchased commercially, although somechanges may be required to present the correct degree ofcomplexity. A very detailed model will necessarily slowthe visualisation; however, it is important to include allthe detail required for the accident scenario. A typicalmodel used in the Visualiser, such as the Super Puma,has 2000 polygons. Models here differ from the AAIT -Impact Tool in that they are not constructed as masspoints and beams but are solid models. The Visualisercan display the view as meshed or rendered.

Figure 1 shows the Super Puma model lifting from theplatform.

Figure 1. CAD Model of Super Puma


• Message files

FDRV allows a message file to be input; this isa simple ASCII text file which has a series ofmessages with time tags, related to the timingsof the .dat file. These messages can be takenfrom the Cockpit Voice Recorder, Air TrafficControl, or warning flags.

• Set-up files

There is also an optional set-up file whichallows the user to customise the display for aparticular visualisation, such as the display ofsimple shapes, e.g., runways, rigs, buildings,etc. It can also control the number of nodesand height indicators to be displayed.

Running the Visualiser

The FDR Visualiser is currently a DOS pro-gram, with customised keys to allow the user to obtainthe various functions. Once the data input is complete,the user runs the program. The visualisation thendisplays the model, with nodes at each x, y, z position toshow the fight path. The model moves from each node,displayed at the correct orientation (pitch, roll, yaw). Thevisualisation can be slowed to run in real-time; however,for a complex model with a large number of polygons, itmay not always be possible to run as fast as real-time.Possible upgrades to the graphics card would improvethe speed of the Visualiser. The .dat file defines the real-time speed of the visualisation—this contains the framerate of the derived data.

Generally there is no interpolation of the data from FDR;the frame rate is defined by the sample rate of theparameters. In the case of the Super Puma G-TIGHvisualisation, pitch and roll data were available fourtimes a second, as was the accelerometer data fromwhich the flight path was derived, so this was the framerate which was used. This frame rate produces anacceptable visual impression within the limitations ofPC-based virtual reality. The user can freeze thevisualisation, and step backwards and forwards in eachframe. Height indicators and ground position nodes arealso available for display.

As with any virtual reality software, the user can viewthe scene from any position, and a number of set views(north, south, east, west and from above) are availablefrom the function keys. Users can also ‘attach’ their eyeposition to the model, and fly as if in a chase plane. Anew feature added has been to set up, via the set-up file,a number of views, and these can include a pilot’s eyeposition. Users can then position themselves in thecockpit, and then rotate that eye-position around, or upand down. This simulates the pilot view when he moveshis head.

This feature was performed on the Super Puma G-TIGHvisualisation, where cockpit visibility limits were criticalin maintaining a view of the destination platform duringthe flight, which was under VMC flight rules. The AAIBReport had presented information on the visibility limits,and these were used in the visualisation. Users can sitinside the model of the Super Puma, look out of thewindows, and rotate the head to the limit of visibility ofthe side windows. This feature, of course, requires anaccurate model of the aircraft visibility in order toconduct this sort of investigation. A cockpit view isshown in Figure 2.

The limitation of any visualisation system is, of course,that it cannot reproduce the actual view the pilot saw. Inthe case of the Super Puma, the flight was at night, withheavy snow showers.

The data contained in the .dat file, i.e., any engineeringvalue from FDR, can also be displayed on screen in textform. The user can set up which of these parameters areto be displayed; this feature is particularly useful todisplay discretes such as autopilot disconnection, warn-ings, or other parameters relevant to the visualisation.

As a result of a military accident in March 1996, theVisualiser was upgraded to include the display of morethan one aircraft. The accident on which it was used wasa mid-air collision between two Tornado aircraft, wherethere was good wreckage information available on therelative aircraft attitudes and positions at impact. Theflight path was derived from the very basic FDR data,using airspeed, altitude and heading. Here was a goodexample of the limitations of such visualisation systems:the basic recorded data would not have been sufficient to

Figure 2. View from Cockpit


perform this exercise had the impact informa-tion not been available. Therefore, had theaircraft had an air-miss rather than a collision,it would not have been possible to locate theaircraft without another independent datasource such as Radar. Figure 3 shows therelative attitudes at impact.

Further proposed upgrades include theinclusion of a terrain model using digital mapinformation. The FDR Visualiser currentlyshows a flat plane beneath the aircraft model.It would be more representative to show theactual terrain, and this would be essentialwhen the terrain had been a factor in theaccident. In a mid-air collision between aTornado and Jet Ranger helicopter, there was agood example of this, where a 3-D terrainmodel was, in fact, used in the final AAIBreport (AAR ) (2), but without animation.

Conclusions

AAIB experience has been that it is an effective tool inimpact investigation; it is hoped to develop the ImpactTool through its use on accidents. The Impact Tool isreadily accessible to accident investigators withoutnecessarily the expertise of an impact dynamicist.

The FDR Visualiser provides a data driven method ofdisplaying aircraft positional and orientation informa-tion. Its use in various accidents has revealed areas fordevelopment, and demonstrated the flexibility andusefulness of such visualisation techniques. Care wastaken in the specification and development of theVisualiser to ensure that the data is displayed as from theASCII file without any interpolation. It is essential, then,for the user to beware in using such a tool of the limita-tions of the input data, and of the system itself. Thedisplay is only as good as the data, whether FDR orRadar, that has been used.

Further developments proposed include the terrainmodel, and a Windows-based version that would allowthe integration of all the information now available fromFlight Recorders and other sources of data.

Footnotes

(1) Aircraft Accident Report 2/93, Report on the Acci-dent to As 332L Super Puma, G-TIGH, near theCormorant ‘A’ platform, East Shetland Basin, on 14March 1992. HMSO

(2) Aircraft Accident Report 2/94 Report on the Acci-dent Involving Royal Air Force Tornado GR1, ZG754and Bell 206B JetRanger III, G-BHYW at FarletonKnott near Kendal, Cumbria on 23 June 1993.

Anne Evans is a Senior Inspector at the Air AccidentsInvestigation Branch, specialising in the replay and analysis ofFlight Recorder information. She graduated from ImperialCollege of Science and Technology with a Bachelors degree inAeronautical Engineering, and then worked for BritishAerospace as a Performance Flight Test Engineer and the CivilAviation Authority, before joining AAIB in 1987. She alsoholds a current PPL.

Figure 3. Tornado Mid Air Collision


The views expressed in this paper are the author’s, anddo not reflect official policies or positions of the UnitedStates Air Force or the Department of Defense.

Introduction

There is a small piece of land in the Potomac River,adjacent to Washington, DC, called “Roosevelt Island.” Itis accessible on foot from the Virginia side of the river. Ifyou make your way to the center of the island, followingany of several nature trails, you come upon an impres-sive memorial to President Theodore Roosevelt. Thememorial’s walls are inscribed with a number of quota-tions from President Roosevelt’s public life, but onestands out for both its economy of words and its acuityof vision. It is this passage that serves as a theme for thispaper:

Combine worthy ideals with practical good sense.

Air safety investigators, like safety professionals in otherdisciplines, suffer an undeserved (or at least somewhatundeserved) reputation for zealotry. We often standaccused of a lack of vision, of too narrow a perspective.However, some safety professionals wear such labels asbadges of honor, defending extreme nay-saying as “theconscience of the organization,” and arraying themselvesin the armor of righteousness.

Students of military history will doubtless recall that thereason suits of armor fell out of favor was twofold: thesingularly dispiriting effectiveness of gunpowder-propelled projectiles, and the helplessness of the weareronce unhorsed. Since being “shot out of the saddle” isnot a bad analogy for the ruthlessness and randomnessof corporate downsizing today, it is reasonable toconclude that a certain moderation of tone may be inorder for the sage professional wishing to remain em-ployed.

This is not to say that the analysis and prevention ofaccidents do not offer intrinsic value to those whoincorporate such considerations into the conduct ofbusiness. On the contrary, as safety professionals know,safe operations can enhance any enterprise in numerousways. However, given today’s business environment,our challenge as aviation safety experts is to communi-

cate our value in ways that make it clear that we areessential to the business of aviation, as well as to all forwhom aviation operations are essential to the success oftheir overall venture. In other words, we must developstrategies for combining our worthy ideals with hardbusiness sense.

Consider the following question: why do we investigateaircraft accidents? There are two schools of thought, bothof which have merit:

• Pragmatists hold that the purpose of investigations isto assign responsibility for given losses. Some maychallenge the validity of this view, but it is fair to saythat the state of the aviation art is sufficiently ad-vanced that accountability may be expected of thosewho fail to perform their duties properly, be theypilots, mechanics, designers, or managers.

• Visionaries, on the other hand, would more likelypoint to the words of Article 5 of the InternationalSociety of Air Safety Investigators’ Code of Ethics:“to develop findings and recommendations that willimprove aviation safety.” However, this admittedlylaudable objective begs the question as to how suchbenefits are to be realized from an investigator’slabors; investigations are increasingly expensivepropositions, and there is natural resistance to anycostly activity that is not acknowledged to haveinherent value of its own.

If we are to achieve, or even strive for, the more idealisticof these two visions, we have our work cut out for us.Pursuing a more activist role in both the prevention ofaviation accidents, and in the broader business environ-ment in which they occur, is a daunting proposition. Itmeans the process of investigating itself, as well as allother activities intended by safety professionals to resultin the prevention of accidents, must be aggressivelymarketed to those who control the resources needed formeaningful corrective action.

Statement of the Problem

How do we, as air safety investigators and by extensionas safety professionals, relate what we do to what thebusiness world both wants and needs? Answering this

Adding Value to Aviation Operations:New Directions for the 21st Century Investigator

Thomas A. Farrier M03763Chief, Flight Safety Issues, Office of the Chief of Safety

Headquarters, United States Air Force


question requires a common perspective on how aviationand business intersect, overlap, and complement eachother. Accordingly, the first half of this paper is by wayof stage-setting. It will briefly touch upon the variouscomponents of the aviation system as a whole, and ontheir relationship to the system’s commercial users. Itwill then examine the characteristics and priorities of thebusiness end of aviation in terms of profitability, de-mand, marketing, and systemic pressures. Once thisgroundwork has been laid, we may then logically relatewhat we as investigators and safety professionals do towhat commercial users of aircraft require for success; thesecond half of the paper will explore the nature of airsafety expertise and some proposed strategies for match-ing that expertise with the needs of the marketplace.

The Aviation System

What constitutes the overall “aviation system”? Briefly,the system requires aircraft, airfields (or at least somereal estate dedicated to flight operations on an occasionalbasis), enroute communications, navigation and sequenc-ing equipment and procedures, and personnel to fly theaircraft and make them airworthy. Each aircraft, airfield,air traffic control and navigational system, and indi-vidual working within the system is responsible to somehigher controlling body for standards of constructionand conduct (Figure 1).

By way of comparison, the United States Air Force’ssafety community arrays these factors within its “5M”model, whereby a man operating a machine in theaviation medium accomplishes a mission within anoverarching management structure (Figure 2). Eithermodel is valid, so long as the distinctiveness of eachelement and their inherent interrelationship is clearlyunderstood.

It may seem intuitive, and perhaps unnecessary, to listthese elements in this manner. However, bear in mindthat this discussion is aimed at helping safety profes-sionals relate what they do to what corporate aviationdoes in a way that can in turn be explained to con-sumers.[NOTE: For the purposes of this paper, the term“corporate aviation” will be used to refer to the entirespectrum of businesses that involve themselves inaviation operations, either as their primary focus (e.g.,major air carriers, air freight concerns, etc.), or as anadjunct to and facilitator of their primary enterprise (e.g.,multinational corporations that operate their ownaircraft to meet the demands of their business in a timelymanner)].

The Business of Flying

Given the complexity of the aviation system describedabove, a significant investment must obviously be madeto participate efficiently. This is why in their purestincarnation—that of enterprises solely dedicated to theproviding of aviation services—individual commercialaviation operations in the aggregate represent an“oligopolistic industry.”(1) The characteristics of suchindustries are many, but four will be readily rec-ognizable to those associated with the business of flying:

• Economy of scale realized by large-scale productionand delivery of services, and facilitated by special-ization, efficient technology, and efficient use of by-products.

• Growth through mergers.

• Mutual dependence among members of the industrywith respect to price.

• Price rigidity and non-price competition.

17

Manufacturer Management

Nat’l TransportAuthority Airport Authority

THE AVIATION SYSTEM

Management

Man Media

Machine

Mission

5M CONCEPT

Figure 1. Elements of the Aviation System

Figure 2. The “5M” Concept


The last characteristic holds an early clue to the safetyprofessional’s dilemma. As we will see presently, airlinescannot compete on the basis of their safety records(although there have been some recent, ill-consideredefforts to create a regulatory basis for doing so). Thismeans that they do not have an overtly market-basedincentive for pursuing an aggressive safety program. Byextension, this suggests that safety must be made appeal-ing with respect to its impact on those attributes of theorganization that directly affect its ability to compete.

The critical lesson to be drawn here is that firms competedifferently with equivalent resources; this is why somesucceed and others fail. The difference is in strategicvision and focus, in using all available resources to theirmaximum potential, and through “playing to theirstrengths”: in other words, making the most of their corecompetencies. An effective, well-grounded process ofinvestigation and prevention, positioned as such a corecompetency, thus becomes integral to the success of thecompany. This theme will be expanded upon presently.

Profitability in Corporate Aviation

The means by which corporate aviation operations striveto be profitable, particularly those that rely upon passen-ger or cargo operations for the bulk of their business, arerelatively easy to list. All the following may be consid-ered “goals,” since the more efficiently they are realized,the better for the bottom line:

• Shorter enroute times between points;

• Shorter ground times (for both loading/unloadingand maintenance);

• Maximized use of infrastructure (maintenancebuildings, training facilities, etc.);

• Maximized “up time” through efficient maintenancescheduling and good preventive maintenance;

• Greater fuel efficiency (through both aircraft designand route selection);

• Lower labor costs (through two-pilot cockpits,minimum cabin crew, containerized baggage han-dling, etc.); and

• Minimum non-revenue movements (with theirattendant low load factors in the case of passengeroperations).

Note that those factors that are not directly dependentupon the reduction of operations-based outlays (fuel,labor, etc.) have in common the element of time. Thus,the ability to conduct high-speed/high-tempo opera-tions, and the scheduling of both flight operations andtheir supporting maintenance operations, are critical

factors whose importance grows in direct proportion tothe amount that the corporation relies upon its flightoperations for profit. Only some of the variables thatinfluence scheduling are controllable by the operator;others are dependent upon external agencies, otherindustries, or even the tempo of competitors’ operations.However, this simply means that effective managementof those variables that can be directly affected becomesall the more important.

Safety in the workplace, be that the flight deck or theflight line, usually can be demonstrated to be closelyallied with overall productivity in terms of the goalslisted above. An example would be linking accidentreduction in specific industrial activities, such as cargoloading, to the reduction or avoidance of excess downtime and repair costs due to physical damage to aircraft.On the basis of this self-evident truth, the whole spec-trum of safety activities, from investigations and inspec-tions to well-reasoned recommendations to implementa-tion, clearly becomes attractive (or at least more palat-able) whenever time savings can be realized.

Safety can also be touted for its proven effectiveness inloss reduction. This naturally raises the specter ofidentifying what we do with the “compliance” side ofsafety, i.e., conformity to established standards due tothe threat of legal or administrative sanctions. Still, manysafety standards have greatly helped reduce fiscal andresource losses in industrial operations with minimaladministrative burdens. Qantas Airways, for example,has long tied its superlative prevention efforts to a soberappreciation of exactly what they save the company inboth insured and uninsured costs.(2)

The bottom line is that corporate aviation operations areconditioned by, and respond to, a number of well-defined objectives in the pursuit of profit. Those objec-tives represent opportunities for safety professionals toprove their worth in tangible and practical ways.

The Attractiveness of Aviation to Consumers

Having briefly outlined the basic business environmentwithin which corporate aviation exists, we must nowconsider the reasons why corporate aviation exists in thefirst place. In other words, what makes the use of avia-tion as a means of transportation attractive to consum-ers?

First, the nature of demand in some businesses(unpredictability, infrequency, surges beyond localability to supply, and “seasonality”) calls for the speedand accessibility aerial shipment affords. Second, thenature of some commodities (perishability, short-noticeneed, value relative to weight, expense to handle andstore) makes aviation the transportation mode of choice.Third, the means of aerial distribution itself can offerusers such advantages as less loss exposure or potential


in transit, lower packaging costs, and a reduced need forwarehousing because of the immediacy of backfill airtransportation can provide.(3)

It should be evident from these considerations thatcommercial aviation operations are driven by twodistinct sets of consumer-oriented priorities: one tangibleand easily measured, the other intangible and far moredifficult to capture empirically. The tangible priorities—schedule reliability, passenger comfort, and efficiency ofcargo and baggage movement—are all easy to analyze.The intangibles, most notably the “image” and reputa-tion of the aviation concern and customer confidence init, are far more difficult to quantify. Nevertheless, howwell the aviation concern manages all of its prioritiesconstitutes the principal focus of the “non-price competi-tion” mentioned previously.

Clearly, these priorities exist for all corporate aviationentities. They may be reordered somewhat for thosecompanies that use aviation to facilitate their business, asopposed to it being their primary business, but they areall present to some degree. This means that all corporateaviation concerns must contend with the same quantifi-able and unquantifiable influences on how they employtheir aircraft. In fact, an aviation department within alarger non-aviation business must handle its own “non-price competition” the most aggressively of all, since it iscompeting against businesses for which aviation is theprimary focus. If a corporate aviation department fails tosatisfy its “consumers,” it may not simply lose marketposition—it may be eliminated altogether!

“Safety” as a priority straddles the line between thetangible and the intangible. It is difficult to quantify andperilous to trumpet, but a lapse in safety can be far moreinjurious to the bottom line than temporary inconve-nience or a few instances of poor meal service, since itstrikes at the heart of the “intangibles”—the company’simage and reputation. At the same time, it is all butunthinkable for airlines to market themselves on thebasis of a superior safety record, since doing so wouldalmost certainly result in acrimony among competitorsand an elevated level of concern among the very custom-ers they are trying to attract.

Thus, the dichotomy: few companies can afford torelegate safety to a completely inconsequential role, butvirtually all struggle with exactly how to justify itsexpense in times of belt-tightening and cost reduction.Their problem is compounded by the fact that safeflights, while undeniably important to consumers, mustbe in effect a “given,” and an unspoken one at that. So,another of our challenges as safety professionals is tomake our usefulness at preventing accidents as clearlyallied with the continued good name of the company aspossible, but to do so in as subtle and non-public amanner as possible.

Marketing the Aviation Resource

Even given that aviation has become integral to moderncommerce, and that the number of providers of aviationservices is extremely limited, competition definitelyremains a driving force in corporate aviation operations.Major air carriers and air cargo concerns fight to increasetheir revenues and market shares; aviation departmentswithin non-aviation businesses must demonstrate anability to meet their internal customers’ needs moreefficiently than primary aviation service providers. Inother words, corporate aviation has an ongoing require-ment to market itself, and simultaneously must bestructured in such a way as to enhance its marketability.This means balancing four variables:

• The product or service (which must be matched tothe market it seeks to reach).

• The price (which must be set on the basis of value ofthe service delivered while allowing for profit).

• Promotion (which must first attract customers andthen facilitate subsequent sales).

• Placement (actually having service available whereand when it is supposed to be).(4)

In this vein, some business experts suggest looking atwhole firms as consisting of “portfolios of resources”(what they have to work with) and “portfolios of ser-vices” (what they offer toward making use of thoseresources, both within the firm and to the public).(5)Such a model is useful when considering aviationoperations, since it embraces the concept of aviation notonly as a business in its own right, but also as a “busi-ness multiplier.” Under this theory, the “market” mustbe recognized as being the general public (in the case ofprimary aviation providers) or as other divisions of thecompany (in the case of subordinate aviation depart-ments) as appropriate.

The elements of product, price, promotion, and place-ment can be broken into their component parts andapplied against the model of resource and serviceportfolios fairly readily (Table 1). However, this begs thequestion of how safety fits into the overall equation. Ingeneral, loss prevention and risk reduction tend to beseen as services, applied within the company to reducecosts and avoid legal sanctions. However, from a broaderperspective, safety expertise, when married to the largerbusiness objectives of the company, is no longer simply apart of doing business; it can be part of the businessitself.

Following this reasoning, our challenge as safety pro-fessionals is to position ourselves as resources, since theleveraging of resources, as opposed to the allocation ofthem, forms the basis of a company’s strategic focus.


Expenditures associated with making the company’sresources work better are considered strategically justifi-able, and, as such, are more readily acceptable. Byextension, outlays for preventive activities (as opposedto those which are geared toward compliance) becomemore palatable when seen as enhancing the company’sresource base.

Pressures on Corporate Aviation Today

It should be clear from the foregoing discussion ofcorporate aviation’s composition, concerns, and historythat there are numerous pressures on every businessmaking use of aviation for commercial purposes today.Ralph Nader and Wesley J. Smith make a case for nearlya dozen such factors in the introduction to their 1994advocacy book Collision Course: The Truth About AirlineSafety.(6) However, in the context of the observationsmade above, their list can be distilled down to a fewmore broadly drawn contemporary concerns:

• Corporate aviation, particularly the passenger andcargo-carrying segments of the industry, is stillfeeling the pinch from the economic pressures thatresulted from deregulation.

• Aviation-oriented businesses need to obtain morecapacity from existing infrastructure.

• Aircraft owners and operators are increasinglypressed to keep their fleets up-to-date, both by theircustomers and by regulatory authorities, and faceever-growing customer expectations of reliability,comfort, safety, and security.

• The aviation system itself is being taxed by thenumerical growth of less stringently regulated (andless expensively equipped) users.

• Every corporate aviation concern is engaged in thenever-ending struggle to reduce the impact of errorand human performance limitations on its opera-tions.

The last of these is particularly relevant to the subject ofthis paper; the study of human factors in aviation ishardly a new development insofar as scrutiny of crewmembers is concerned, but consideration of the role ofhuman factors in maintenance, air traffic control, and thesupervision of aviation operations has only become ofinterest within the past decade or so. As David Beatyobserved in 1991,

[It] is only recently that very dubious man-agement malpractices are being identified andtheir contribution to accidents given sufficientweight. For though the pilot’s actions are atthe tip of the iceberg of responsibility, manyother people have had a hand in it—facelesspeople in aircraft design and manufacture, incomputer technology and software, in mainte-nance, in flying control, in accounts depart-ments and in the corridors of power. (7)

This late-blooming attention to the management andsupervisory aspects of aviation accidents is addingimpetus to corporate safety consciousness. That height-ened level of interest may be our best ally in our quest toprove our value to the business of flying.

Safety and the Air Safety Investigator as “CoreCompetencies”

Members of ISASI, as well as other aviation safetyprofessionals, bring a wealth of diverse skills andknowledge to the task of investigating. Many of these aredirectly applicable to the business of aviation, but threestand out:

PORTFOLIO COMPONENTS

SERVICES RESOURCES

Product/Service What is deliverable The means to deliver

Price Internal outlays required Infrastructure maintained

Promotion Intrinsic attractiveness What enhances attractiveness

Placement Availability Means of responsiveness

Prevention ? ?

Table 1. Aligning Market Variables with their Place in the “Portfolios”


• Air safety investigators have hands-on expertise andfirst-hand understanding of one or more of theelements of modern aviation described previously,especially with respect to the rules and processesassociated with aviation.

• Air safety investigators possess specialized knowl-edge of aerodynamics, structures, systems, humanfactors, and many other subjects, all of which can bebrought to bear on the problems of flight and groundsupport planning and execution.

• Air safety investigators normally demonstratetypical “pilot” personality traits—decisiveness,aptitude for problem-solving, and so forth—but alsotend to leaven these attributes with creativity,imagination, and the ability to consider novelpossibilities—to “think outside the box”—when theunexpected rearranges the otherwise orderly worldof flying.

In a broader sense, we are defined by our breadth ofknowledge and our professional responsibilities. TheAmerican Society of Safety Engineers says it best in theirdescription of what constitutes professionalism in safety:

Because safety is an element in all humanendeavors, safety professionals perform theirfunctions in a variety of contexts in bothpublic and private sectors…Within thesecontexts, safety professionals must adapttheir functions to fit the mission, operationsand climate of their employer.(8)

There are a number of signal successes we as safetyprofessionals can point to as proof that the marriage ofsafety expertise and real-world operations can bearuseful progeny, beginning in the earliest years of pow-ered flight. The United States Army’s Air Service, whichdid much to systematize the investigation of aviationaccidents in the 1920s and 1930s, developed an institu-tional strategy for the handling of aviation safety prob-lems that we would do well to remember and emulatetoday:

• Identify a safety problem;

• Develop the structure necessary to institutionalize afix for the problem;

• Find the right organization within the Army topermanently manage the fix, pass the structure on,and move on to the next problem.

This pattern was repeated several times by the AirService’s successor, the Army Air Force (AAF), andparticularly by AAF’s “Office of Flying Safety” (OFS). Asan example, consider the history of air traffic control(ATC) in the United States. Originally established by

airlines and subsequently taken over by the “Bureau ofAir Commerce,” ATC was a spotty and unreliableproposition away from major metropolitan areas.(9) Thishad to change with the sudden flood of flying thatimmediately preceded the United States’ entry intoWorld War II, and it did.

For a time, OFS was responsible for the United States’first national system of air traffic control and flightfollowing. However, within less than two years of itsinception, “Flight Control Command” broke away fromits safety parent and became a separate entity respon-sible to AAF’s operations (A-3) staff. This freed theOffice of Flying Safety to address problems in operationssafety (which led to the development of the first stan-dardized operational manuals, initially published byOFS); maintenance safety (which eventually was institu-tionalized by the logistics community, armed withsafety-authored standards); and, following World War II,breaches of flight discipline (which eventually becamerecognized for the hazards they were, and began beingdealt with appropriately, thanks to an array of well-chosen accident statistics).

More contemporary examples of applying safety exper-tise to costly (or potentially costly) problems abound. Forexample:

• Foreign object damage prevention programs (main-tenance savings).

• Bird hazard reduction, control, and avoidanceprograms (maintenance savings, reduced chance ofliability losses).

• Crew communications, productivity, and effec-tiveness programs (increased operator efficiencywith applicability to other disciplines, e.g. mainte-nance).

• USAir’s “altitude awareness” partnership with theFederal Aviation Administration (simultaneouslyallowing study of a human factors problem andsystematic application of possible corrective strate-gies while avoiding the usual sanctions associatedwith altitude violations).

The last example suggests how future safety efforts mustbe directed: toward reduced emphasis on institu-tionalized, punitive aspects of aviation safety (which arethose normally reacted to by management), in favor ofpreventive strategies that are also geared toward im-proving the relationship between safety and the rest ofmanagement.

Positioning safety and its allied investigative and pre-ventive activities as a “core competency” will require usto prove our intrinsic worth in areas well outside thenarrow confines of formal investigations and routinized


compliance with standards. If we can identify andalleviate some or all of the pressures associated withcorporate aviation today, we will go a long way towardmarkedly increasing our worth to the aviation commu-nity as a whole.

Where We Can (and Should) Make a Difference

Aviation safety professionals can and should be involvedin the business of aviation at every level imaginable. Ourexpertise, body of knowledge, and unified vision repre-sent a resource rarely exploited—or even tapped—untilpreventive efforts have already failed. However, we willhave to insert ourselves into other people’s business ifwe are to realize our potential, and we will have to do soknowledgeably. The following are a few ways we canand should start to change our professional focus:

• We need to engage more fully in system safetyefforts related to aircraft design (especially in theareas of survivability, automation, and degradedmodes of operation), since these affect both thereality and the public perception of how “safe” anaircraft is.

• We need to help corporate aviation concerns makesmarter decisions about their biggest capital invest-ments: the aircraft they buy or lease (fleet commonal-ity cuts logistics overhead and training costs dra-matically; viz. Southwest, America West, and othermajor air carriers that limit themselves to one or twomodels of aircraft).

• We need to make our voices heard more loudly inthe design of the infrastructure that underlies theaviation system, including such “big ticket” items asvoice and data communications, as well as in thehomelier, but no less critical tools of charts andinstrument approach procedures.(10)

• We need to expand our involvement in aircrewtraining well beyond the traditional “lessonslearned” and “crosstell” models into preventive,proactive measures like airfield familiarizationprograms and discussions of dissimilar aircraftproblems.

• We need to gain inroads into the daily operation ofaviation-oriented companies, marryingmanagement’s priorities, especially in schedulingand route planning, to solutions that enhance bothprofitability and safety.

• We need to pay more attention to the problems andchallenges of airport management, such as lighting,snow removal, pavement upkeep, crash/fire/rescue,and long-range gate and terminal improvementprojects, to help increase capacity (and revenue)without sacrificing safety.

Summary

Returning to the original problem stated at the start ofthis paper: how do we as air safety investigators, and byextension safety professionals, relate what we do to whatthe business world both wants and needs? The foregoingdiscussion should lead the reader to conclude that athree-pronged strategy will be required:

• We must educate management how systematicapplication of safety expertise to their normalaviation-related business operations can help their“bottom line,” while remaining aware of and respon-sive to management priorities.

• We must work to enhance management’s complianceefforts, both qualitatively and quantitatively, to freeup more resources for prevention, while keeping theresulting prevention activities effectively transparentto the public and without overly identifying our-selves with “compliance” per se.

• We must expand our own horizons to seek out thesynergies that we might realize through moreeffective interaction with other professionals; indoing so, we will undoubtedly find new ways ofgetting our message through to the decision-makers,as well as a new body of allies with a common cause.

Our challenge, as investigators and safety professionals,is to explain both the compliance and prevention aspectsof what we do, and then help management identify andrealize returns on its safety investment beyond the mereminimization of legal vulnerability. In so doing, we mustalso understand and be guided by the priorities andexpectations of both management and the customer.These center around (1) the uniqueness of aviationcapability, (2) the costliness of its associated resourcesand infrastructure, and (3) the concomitant need tooptimize them both.

From a management perspective, a truly successfulsafety program is one that materially contributes to thesuccess (read: profitability) of the operation with mini-mal intrusion into that operation. This goes back to theparadox touched upon earlier: how can safety personnelbe at once transparent to the daily operation but cel-ebrated for their contributions? Fundamentally, the“marketability” of safety relies upon:

• The attractiveness of the concept (what it offers forcompliance and prevention vs. what advantages itoffers with respect to aviation business priorities);

• The affordability of the proposal (costs vs. benefits); and

• The adaptability of the strategy (meeting both what wethink management needs vs. what managementdecides it wants).


Safety initiatives, like others that originate from sub-ordinate parts of an organization, must concentrate onenhancing the technological attributes that make aviationattractive to its commercial users (speed, mobility, andcapacity) while simultaneously seeking to improve theeconomy, efficiency, and convenience of the serviceprovided.

The “wild card” in selling safety as a business multiplierlies in the irrefutable fact that we are to a large extentvictims of our own success. The level of safety enjoyedby many segments of the aviation industry, and particu-larly those segments most commonly encountered by thegeneral public, is so high as to have fostered a prevalentsentiment that the only risk involved with flying comesfrom random, uncontrollable events—an attitude themajor air carriers have worked diligently to nurture.

For decades, the unspoken agreement among virtuallyall major air carriers, not to mention the major aircraftmanufacturers, has been to keep all reference to therelative safety of any one operator or airframe out of thepublic eye. This has occasionally backfired, such as in thecase of the Comet, when its many shortcomings becamepublic knowledge, and more currently in the case ofValuJet, as a host of unspoken misgivings on the part offederal regulators and safety inspectors suddenly foundan alarming, even alarmist public voice. Still, debateabout means of enhancing the safety of existing aircraftand systems has remained within the fraternity ofaviation professionals, far from the public eye.

Obviously, the stage is potentially set for confrontationswith the marketing end of the business (in the case ofpassenger-carrying operations) unless future safetystrategies are specifically designed for maximum trans-parency to consumers, both external and internal to thecorporate aviation entity. Our ideals are truly worthy,but the times ahead will call for the utmost in practicalgood sense in pursuing them.

Footnotes

(1) See Wells, Alexander T., Air Transportation: AManagement Perspective (Belmont, CA:Wadsworth Publishing Company, 1994), pp. 194-195 for a fuller discussion of this characteriza-tion.

(2) Beaty, David, The Naked Pilot: The Human Factorin Aircraft Accidents (Shrewsbury, England, UK:Airlife Publishing, Ltd., 1995), pp. 214-215.

(3) Wells, op. cit., pp. 366-367.

(4) Ibid., p. 292.

(5) Hamel, Gary and C.K. Prahalad, Competing forthe Future (Boston MA: Harvard Business SchoolPress, 1994), pp. 157-160 passim.

(6) Nader, Ralph and Wesley J. Smith, CollisionCourse: The Truth About Airline Safety (Blue RidgeSummit, NJ: TAB Books, 1994), p. xx.

(7) Beaty, op. cit., p. 222.

(8) American Society of Safety Engineers (ASSE)“Home Page” (http://www.asse.org), 1996.

(9) For an engaging discussion of ATC’s evolution,see Buck, Robert N., The Pilot’s Burden: FlightSafety and the Roots of Pilot Error (Ames, IA: IowaState University Press, 1994), pp. 9-18.

(10) Michael Overall, writing in Cathay Pacific’scompany safety magazine Kai Talk (“Newchallenges pressure safety management sys-tems,” July 1995, pp. 13-20), made a forcefulargument to the effect that the entire aviationsystem is at risk due to the reduction of its safetymargins and the growth of its technologicalcomplexity.

Thomas A. Farrier (B.S., US Air Force Academy; M.A.,Georgetown University) works in the Pentagon on the staff ofthe United States Air Force’s Chief of Safety as Chief, FlightSafety Issues. He is a frequent contributor to United StatesAir Force safety magazines, and has also appeared in the RoyalAir Force’s Air Clues. He has served as the United States AirForce’s representative to the Air Transport Association’s“Human Factors Task Force,” and is a former United StatesDelegate to the System of Cooperation Among the AmericanAir Force’s Accident Prevention Committee (SICOFAAPREVAC).


On October 31, 1994, at 1559 Central Standard Time, anAvions de Transport Regional, model 72-212 (ATR 72),registration number N401AM, leased to and operated bySimmons Airlines, Incorporated, and doing business as(d.b.a.) American Eagle flight 4184, crashed during arapid descent after an uncommanded roll excursion. Theairplane was in a holding pattern and was descending toa newly assigned altitude of 8,000 feet when the initialroll excursion occurred. The airplane was destroyed byimpact forces; the captain, first officer, 2 flight atten-dants, and 64 passengers received fatal injuries. Flight4184 was a regularly scheduled passenger flight beingconducted under 14 Code of Federal Regulations (CFR)Part 121, and an instrument flight rules flight plan hadbeen filed.

On July 9, 1996, the National Transportation SafetyBoard adopted the final report of facts (1), conditionsand circumstances, and determined that the probablecauses of this accident were the loss of control attributedto a sudden and unexpected aileron hinge momentreversal that occurred after a ridge of ice accretedbeyond the deice boots because:

• ATR failed to completely disclose to operators, andincorporate in the ATR 72 airplane flight manual,flightcrew operating manual and flightcrew trainingprograms, adequate information concerning previ-ously known effects of freezing precipitation on thestability and control characteristics, autopilot, andrelated operational procedures when the ATR 72 wasoperated in such conditions;

• the French Directorate General for Civil Aviation’s(DGAC’s) inadequate oversight of the ATR 42 and72, and its failure to take the necessary correctiveaction to ensure continued airworthiness in icingconditions; and

• DGAC’s failure to provide the Federal AviationAdministration (FAA) with timely airworthinessinformation developed from previous ATR incidentsand accidents in icing conditions, as specified underthe Bilateral Airworthiness Agreement and Annex 8of the International Civil Aviation Organization.

The Safety Board also found that contributing to theaccident were:

• FAA’s failure to ensure that aircraft icing certifica-tion requirements, operational requirements forflight into icing conditions, and FAA publishedaircraft icing information adequately accounted forthe hazards that can result from flight in freezingrain and other icing conditions not specified in 14CFR Part 25, Appendix C; and

• FAA’s inadequate oversight of the ATR 42 and 72 toensure continued airworthiness in icing conditions.

As a senior air safety investigator with the NationalTransportation Safety Board (NTSB) it is my responsibil-ity as investigator-in-charge (IIC) to ensure that theBoard’s specialty investigators, along with those desig-nated experts participating in the investigation, collectall, or as many as is reasonably possible, facts, condi-tions, and circumstances of a particular accident, identifyissues that are critical to flight safety, and developrecommendations that will prevent a reoccurrence of theaccident.

The investigation of the American Eagle ATR 72 in-volved numerous investigators spending hundreds ofman-hours identifying and analyzing thousands ofdocuments to identify the causes of this accident. To saythat after 20 months of investigating, the Safety Board’sfinal analysis and determination of probable cause wasnot readily accepted by all parties, organizations, orindividuals is a monumental understatement.

The Safety Board, as well as the numerous national andinternational organizations and independent investiga-tors, have a deep and sincere interest in preventingaircraft accidents worldwide. The responsibility that the“Air Safety Investigator” bears is great, because the livesof hundreds of thousands of passengers andcrewmembers depend on the thoroughness of theinvestigation and the objectivity that we maintain whenwe perform our duties as an “Air Safety Investigator.” Itake this responsibility seriously.

As we all know, our first concern is to determine thecause(s) of the accident and to identify the remedial

American Eagle Flight 4184“An Icing Crisis”

Gregory A. Feith, MO2342Senior Air Safety Investigator

National Transportation Safety BoardWashington, D.C. 20594


actions necessary to prevent their reoccurrence. How-ever, we also know that there are often serious politicaland economic considerations that can affect the investi-gative process and possibly the outcome of the investiga-tion. While these factors can’t be ignored, it is ourresponsibility as investigators to put those influencesaside, and perform our duties with the objectivity,diligence, and professionalism that we know will serveto enhance the safety of the aviation industry and notone particular individual, group, or organization.

Icing Information

The following is a summary of the issues that wereidentified during the investigation, and the Safety Boardconclusions that supported the subsequent recommenda-tions to the Federal Aviation Administration.

The Safety Board found that the crew of flight 4184 wasnot provided a complete weather package by the com-pany dispatcher prior to departure. Specifically, theinformation contained in AIRMETs [airman’s meteoro-logical information] “Zulu,” “Sierra,” and “Tango,” andUpdate 2 was available well in advance of flight 4184’sdeparture, and was pertinent to the planned route offlight. However, the investigation revealed that thisinformation was not, and typically would not be, in-cluded in the weather portion of the flight releaseprovided by Simmons Airlines/AMR Eagle. While itcould not be determined if the flightcrew had obtainedthe updated weather information via the hazardous in-flight weather advisory service (HIWAS) while en routeor prior to the recorded conversations on the cockpitvoice recorder (CVR), 14 CFR Part 121.601 (b) and (c)state, in part, respectively, that “…before beginning aflight the aircraft dispatcher shall provide the pilot incommand with all available weather reports and fore-casts of weather phenomena that may affect the safety offlight” and that during a flight the dispatcher shallprovide “any additional available information of meteo-rological conditions including adverse weather phenom-ena.” The Safety Board found that FAA Order 8400.10,paragraph 1423, (Operational Requirements -Flightcrews) requires that AIRMET information beconsidered in the preflight planning process; however,Center Weather Advisories (CWAs) are not required tobe included or considered. Simmons Airlines dispatchersreview the AIRMETs, but they do not typically includethem in the flight release package. CWAs are not in-cluded in the release packages because they are notrequired. The Safety Board was concerned that becauseSimmons Airlines dispatchers do not include AIRMETs(which include information regarding moderate icing)and CWA information, flightcrews may not be provided“all available weather reports and forecasts of weatherphenomena” necessary to make informed decisions.

This finding led the Safety Board to recommend to FAAthat the principal operations inspectors (POIs) should

ensure that all air carriers require their dispatchers toprovide pertinent information, including AIRMETs andCWAs, to flightcrews for preflight and in-flight planningpurposes. Further, FAA should Revise FAA Order8400.10, Chapter 7, Section 2, paragraph 1423 to specifythat CWAs be included and considered in theflightcrew’s preflight planning process.

The Safety Board also found that the icing definitionspublished in the Aeronautical Information Manual(AIM) did provide sufficient basis for assessing iceaccumulation in PIREPs; however, the definitions werevague and subjective, and of limited use to pilots ofdifferent aircraft types. For example, using these defini-tions, “light” icing for a B-727 could be “severe” icing foran ATR 72 or a Piper Malibu. The icing report providedby the captain of the A-320 Airbus that was holding at anintersection near Roselawn indicated that he observedabout 1 inch of ice accumulate rapidly on his aircraft’sicing probe. The captain provided a PIREP to ATC andreported the icing as “light rime.” However, after theaccident, the captain stated that the anti-ice equipmenton the airplane “handled the icing adequately,” and thathe believed the icing intensity was more accuratelydescribed as “light to moderate.”

The Safety Board concluded that icing reports based onthe current icing severity definitions may often bemisleading to pilots, especially to those pilots of aircraftthat may be more vulnerable to the effects of icingconditions than others. Therefore, the Safety Boardrecommended to FAA that new aircraft icing intensityreporting criteria be developed that are not subjectiveand are related to specific types of aircraft.

The investigation revealed a problem with the aviationcommunity’s general understanding of the phrase “icingin precipitation,” which is used by the US NationalWeather Service but is not defined in any aeronauticalpublications, including FAA advisory circulars, Part 1 ofthe Federal Aviation Regulations (Glossary of Terms), orAIM. This phrase is often contained in inflight weatheradvisories; however, it does not typically specify types ofprecipitation. According to NWS, this phrase is intendedto include freezing drizzle and freezing rain. The SafetyBoard concluded that defining “icing in precipitation” inthese publications would make pilots and dispatchersmore aware of the types of precipitation and icingconditions that are implied by this phrase. Accordingly,the Safety Board recommended to FAA that a definitionshould be provided for the phrase “icing in precipita-tion” in the appropriate aeronautical publications.

Icing Certification

Based on the evidence revealed in this accident, theSafety Board believes that the current methods of fore-casting icing conditions are of limited value because theytypically cover very large geographic areas, do not


provide specific information about liquid water content(LWC) or water drop sizes, and use only relative humid-ity and temperature. According to testimony from ascientist from the National Center for AtmosphericResearch (NCAR), it is not possible to infer the severityof icing using only temperature and humidity. Rather,the severity of the icing also depends on LWC and thesize of the water droplets, information that is not cur-rently identified and forecasted.

The Safety Board was concerned that there are noreliable methods for flightcrews to differentiate, in flight,between water drop sizes that are outside the 14 Code ofFederal Regulations (CFR) Part 25, Appendix C, (JAR)icing envelope and those within the envelope. Further,although side window icing was recognized as anindicator of ice accretions from freezing drizzle duringflight tests of an ATR 72 after the accident, the crew ofFlight 4184 could not have been expected to know thisvisual cue because its significance was unknown to theATR pilot community at the time. Moreover, in-serviceATR incidents and pilot reports have shown that sidewindow icing does not always accompany ice accretionsaft of the deice boots, which ATR has stated only occursin freezing drizzle and/or freezing rain.

The Safety Board acknowledged the efforts of atmo-spheric research in the meteorological community andhopes that its important findings will eventually providethe aviation industry with a better understanding of thefreezing drizzle/rain phenomenon. The Safety Boardconcluded that the continued development of atmo-spheric measuring and monitoring equipment, such asatmospheric profilers, use of the WSR-88D [weathersurveillance radar] and terminal Doppler weather radars,multispectral satellite data, aircraft-transmitted atmo-spheric reports, sophisticated mesoscale models, and thedevelopment of computer algorithms, such as thosecontained in the FAA’s Advanced Weather ProductsGenerator program to provide comprehensive aviationweather warnings, could permit forecasters to refine thedata sufficiently to produce more accurate icing forecastsand real-time warnings. Thus, the Safety Board recom-mended to the FAA that it continue to sponsor thedevelopment of methods to produce weather forecaststhat define very specific locations of potentially hazard-ous atmospheric icing conditions (including freezingdrizzle and freezing rain) and to produce short-rangeforecasts (“nowcasts”) that identify icing conditions for aspecific geographic area with a valid time of two hoursor less.

Although the Safety Board has found no evidence thatwould indicate that the ATR 42 and 72 airplanes werenot properly certificated for flight into icing conditions,the investigation did raise a number of concerns relatingto the process for certifying airplanes for flight into icingconditions. Among these concerns were the acceptanceby regulatory authorities of a limited number of icing

test data points, most of which were not near the bound-aries of the envelope; the limited range of conditions(LWC and MVD [median volumetric diameter] size)provided by the Appendix C icing certification envelope;the lack of standardized methods for processing LWCand MVD data; the implied authorization of flight intoconditions beyond the envelope; and the certification ofstall protection systems that are intended to preventexposure to undesirable (even dangerous) characteristicsof the airplane without a requirement for the manufac-turer to advise FAA, operators, and pilots of suchcharacteristics.

This investigation revealed that the ATR 42 and 72airplanes were not required to be tested throughout asignificant portion of the icing conditions that arecurrently specified in the Appendix C icing envelope.The limited number of test points accepted by FAA assufficiently comprehensive were well within the bound-aries of the envelope, but did not include the warmer,near freezing conditions at the upper boundary of theAppendix C envelope (the area in which run-back icingand asymmetric sliding/shedding are likely to occur).Thus, by allowing limited data that is well within theenvelope to suffice for certification purposes, FAAeffectively precluded any chance of identifying thephenomena that led to the ice ridge buildup that led tothe uncommanded aileron deflection and loss of controlby the crew of Flight 4184.

The Safety Board’s concern about the adequacy of theAppendix C criteria was heightened by the results of oneDecember 1994 ATR icing tanker test in which iceaccumulated behind the active portion of the ATR 72’sdeice boots during exposure to water droplet sizes ofonly 57 microns MVD, which is only slightly outside theAppendix C envelope. Further, data developed byNACA, the NASA predecessor, indicated in the 1950sthat MVDs of 70 microns or more could be encounteredin layer clouds at various altitudes. Flight in layer cloudsis not an unusual event in the United States, but flightinto layer clouds can result in encounters with icingconditions beyond those set forth in 14 CFR Part 25,Appendix C. This was evident in several ATR 42 icingincidents whereby ice accreted aft of the deicing boots inlayer clouds; this supports the conclusion that icingencounters in high altitude layer clouds can exceed thecapabilities of aircraft certified to the Appendix Cenvelope.

Thus, because the Appendix C envelope is limited anddoes not include larger water drop conditions, such asfreezing drizzle or freezing rain (conditions that can beroutinely encountered in winter operations throughoutmuch of the northern United States, and were most likelyencountered by flight 4184), the Safety Board concludedthat the current process by which aircraft are certified forflight in icing conditions using the Appendix C icingenvelope is inadequate and does not require manufactur-


ers to sufficiently demonstrate the airplane’s capabilitiesunder a sufficiently realistic range of icing conditions.

In addition, the lack of standardized methods for pro-cessing icing data to determine MVDs raises concern thatcertification icing tests may be conducted at actual MVDsbelow the calculated values. For example, during theseries of icing tanker tests at Edwards Air Force Base, itwas determined that two generally accepted methods ofcalculating MVD and LWC provided significantlydifferent results. Therefore, it is possible that airplanescertificated in accordance with Appendix C criteria maynot actually have been tested in the icing conditionsdescribed in the certification documentation. Thus, theSafety Board recommended to FAA that it revise theicing certification requirements and advisory material tospecify the numerical methods to be used in determiningMVD and LWC during certification tests.

Another significant Safety Board finding in this investi-gation is the belief that no airplane should be authorizedor certified for flight into icing conditions more severethan those to which the airplane was subjected in certifi-cation testing, unless the manufacturer can otherwisedemonstrate the safety of flight in such conditions.Although no aircraft are certified for flight into freezingdrizzle or freezing rain, the ATR 72 flight manual did notspecify the operational limits and capabilities of theairplane in conditions such as freezing drizzle andfreezing rain.

Currently, FAA’s ground deice and anti-ice programspermit operators to dispatch aircraft into freezing drizzleand light freezing rain provided they use Type II anti-icefluid and respect the specified holdover timetables.Specifically, Flight Standards Information Bulletin (FSIB)for Air Transport (FSAT), 95-29, dated October 25, 1995,states that Type II deicing fluid will be used when“operating during light freezing rain and freezing drizzleweather conditions” and that the “use of special proce-dures (i.e., visual inspections, remote deice capability) isrequired.” The Safety Board recognized that FAA’sintent of this FSAT was to provide operators with themeans to dispatch airplanes that will quickly depart andclimb through the freezing drizzle or light freezing rainconditions, and that FAA’s permission of limited opera-tions in freezing drizzle and light freezing rain is appar-ently based on the assumption that the airplane willdepart within the prescribed “holdover” time of the anti-ice fluid, and transit through the freezing drizzle/lightfreezing rain conditions with minimal exposure. How-ever, FSAT 95-29 does not specifically state that contin-ued flight in such conditions is prohibited. The SafetyBoard was concerned that in some situations it may benecessary to operate an airplane in such conditions foran extended period of time. One such situation is thefailure of an engine (in a multi-engine aircraft) shortlyafter takeoff, a situation that could require maneuveringfor an indeterminate period of time while returning to

the departure airport where freezing drizzle or lightfreezing rain conditions are known to exist.

Further, although it is known by many in the aviationcommunity that flight into freezing drizzle or freezingrain is not safe, the Safety Board is unaware of an explicitprovision in the Federal Aviation Regulations thatprohibits flight into freezing drizzle and freezing rain.Additionally, as was noted in the Safety Board’s 1981study on aircraft icing (2), airplanes certificated for flightinto known icing are authorized to fly into weatherconditions that produce “severe” icing under 14 CFRParts 91, 135 and 121. However, by definition, severeicing conditions result in a rate of ice accumulation thatexceeds the capabilities of the airplane deice/anti-icingsystem or that require immediate diversion from theplanned route of flight.

The Safety Board expressed its concerns that theseunclear and inconsistent messages to pilots about theoperation of aircraft that are certified for flight in icingconditions may create the misconception that flight infreezing drizzle and/or freezing rain is acceptable whenin actuality it is not.

The Safety Board is aware that, as a result of this acci-dent, FAA is currently planning a review of its icingcertification and operational regulations, including theicing severity definitions issue. The Safety Board issupportive of this action but believes that FAA shouldalso revise 14 CFR Parts 91.527 and 135.227 in a timelymanner to ensure that the regulations are compatiblewith the published definition of severe icing, and toeliminate the implied authorization of flight into severeicing conditions for aircraft certified for flight in suchconditions.

The Safety Board recognizes that the risk of another ATR42 or 72 accident resulting from an uncommandedaileron excursion in freezing drizzle/freezing rain hasbeen reduced by the addition of extended deice boots,improved operational procedures, extensive crewtraining, and heightened awareness by pilots. However,because wind tunnel and in-flight tanker tests have beenperformed for only a limited range of icing and flightconditions, the Safety Board remains concerned whether,even with the improvements to the ATR airplanes, theycan be controlled under all naturally occurring combina-tions of icing conditions. Moreover, the Safety Boardfound that ATR’s post-Roselawn brochure entitled,“ATR Icing Conditions Procedures,” still does notadequately address or clearly represent the exact natureof the ATR ice-induced aileron hinge moment reversal.

The Safety Board noted that Special Condition B6,developed by the French DGAC in the 1980s and initiallyapplied during the ATR 72 certification, includes a “zeroG” flight test maneuver (pushover) designed to identifyice-induced elevator hinge moment reversals. The Safety


Board understands that at least some manufacturers inthe world aviation community (including the UnitedStates) are concerned that Special Condition B6 is toodemanding, particularly the tailplane icing pushovertest. However, the Safety Board concluded that theaddition of a test procedure to determine the susceptibil-ity to aileron hinge moment reversals in both the cleanand iced-wing conditions could help to prevent accidentssuch as that involving Flight 4184. Thus, the Safety Boardrecommended to FAA that it develop a test proceduresimilar to the tailplane icing pushover test to determinethe susceptibility of airplanes to aileron hinge momentreversals in the clean and iced-wing conditions.

As part of the investigation, the Safety Board reviewedhistorical accident and incident data of similarturbopropeller aircraft. The data did not show otherairplane models to have a similar incident/accidenthistory involving uncommanded aileron excursions inthe presence of freezing drizzle/freezing rain. Onepossible reason for this is that other model aircraft usehydraulically powered ailerons, smaller mechanicalailerons with larger hydraulically powered spoilers, ordifferent balance/hinge moment control devices toprovide adequate roll control with less propensity foraileron hinge moment reversals. In a related area, theSafety Board was concerned that FAA and other airwor-thiness authorities permit airplane manufacturers to usestall protection systems (SPS) to prevent flightcrews fromexperiencing known undesirable flight characteristicsunique to their particular aircraft design without requir-ing the manufacturers to reveal these characteristics tothe airworthiness authorities, operators, and pilots.According to ATR, its use of an SPS to prevent, amongother things, aileron hinge moment reversals in the cleanand iced configurations was not explained to the airwor-thiness authorities or the operators because ATR was notrequired to do so. The Safety Board concluded that thefailure of DGAC and FAA to require that they be pro-vided with documentation of known undesirable post-SPS flight characteristics contributed to their failure toidentify and correct, or otherwise properly address, theabnormal aileron behavior early in the history of theATR icing incidents. Therefore, the Safety Board recom-mended that FAA require aircraft manufacturers toprovide, as part of the certification criteria, informationto FAA and operators about any known undesirableflight characteristics beyond SPS and related shaker/pusher flight regime.

Evidence from the investigation also revealed that theFAA’s Aircraft Evaluation Group (AEG) did not main-tain a data base of incident/accident information.Moreover, a communications deficiency resulted in thefailure of AEG to receive pertinent documentationregarding ATR icing incidents that could have been usedto monitor the continued airworthiness of the airplane.This is not the first time that the Safety Board has identi-fied problems with the timeliness and effectiveness of

FAA’s continuing airworthiness oversight of foreign-built aircraft. The Safety Board noted in its 1987 reporton the crash of a CASA C-212-CC that FAA’s monitoringof airworthiness issues relating to that aircraft wasinadequate. Specifically, that investigation revealed thatFAA delayed for more than 3 years taking actions tocorrect known issues of noncompliance with 14 CFR Part25, and that there “was an apparent lack of standardiza-tion and coordination” among various offices withinFAA.

Accordingly, the Safety Board concluded that the lack ofdefined lines of communication and adequate means toretrieve pertinent airworthiness information preventedAEG from effectively monitoring the continuing airwor-thiness of aircraft. Therefore, the Safety Board recom-mended to FAA that it develop an organizational struc-ture and communications system that will enable AEG toobtain and record all domestic and foreign aircraft andparts/systems manufacturers’ reports and analysesconcerning incidents and accidents involving aircrafttypes operated in the United States, and ensure that theinformation is collected in a timely manner for theeffective AEG monitoring of the continued airworthinessof aircraft.

The investigation revealed FAA had limited involvementduring the initial certification of the ATR 42 and 72. Forexample, there were several meetings in which only oneperson from FAA reviewed vast amounts of certificationdocumentation and had to resolve problems from manytechnical disciplines. Further, because FAA personnelwere either unavailable, or budget constraints restrictedtravel, issues involving noncompliance or other concernswere resolved through “issue papers.” An issue paper, ofwhich there were more than 90 for the ATR 42 and 17 forthe ATR 72, describes FAA’s position on a certificationissue and the method(s) necessary to achieve compliance.For ATR, FAA delegated the compliance oversight forthe issue papers to DGAC. In addition, also included inthe certification process is FAA review of test data,including data acquired from flight tests. According totestimony provided by FAA’s ATR certification teamleader, FAA did not flight test the aircraft; rather, itconducted “evaluation” flights for the purpose of“familiarity with airplane…and [to] determine suitabilityfor use in airline service…” FAA conducted about tenhours of evaluation flights on the ATR; however, none ofthese flights duplicated any tests required for certifica-tion, and none were conducted in icing conditions.

The Safety Board concluded that FAA’s limited involve-ment in the ATR 42 certification does not appear to haveresulted in an improperly certificated airplane (ATR 42/72). However, such excessive reliance on a foreignairworthiness authority could result in improper certifi-cation of an aircraft. Therefore, the Safety Board recom-mended to FAA that it review and revise, as necessary,the manner in which it monitors a foreign airworthiness


authority’s compliance with US type certification re-quirements under the Bilateral Airworthiness Agreement(BAA).

The Safety Board also expressed concerns about theprocess by which FAA ensures the continuing airworthi-ness of airplanes certificated under BAA. For example,FAA did not receive pertinent information about theairworthiness of the ATR 42 and 72 series airplanes,including ATR’s analyses of the icing-induced aileronhinge moment reversal incidents in 1991, and those in1993 and 1994. The Safety Board believes that FAA couldhave been more aggressive in requesting data fromDGAC following these previous incidents. However,DGAC should have, on its own accord, taken actions tomake sure that FAA was provided with all informationabout the ATR incidents to ensure FAA involvement inthe continuing airworthiness of the airplane.

The Safety Board concluded that FAA’s ability to moni-tor the continued airworthiness of the ATR airplanes hadbeen hampered by an insufficient flow of critical airwor-thiness information. DGAC’s apparent belief that suchinformation was not required to be provided under theterms of BAA raised concerns about the scope andeffectiveness of BAA. Thus, the Safety Board recom-mended to FAA that it establish policies and proceduresto ensure that all pertinent information is received,including the manufacturer’s analysis of incidents,accidents, or other airworthiness issues, from the export-ing country’s airworthiness authority so that it canmonitor and ensure the continued airworthiness ofairplanes certified under BAA.

Flightcrew Actions

The Safety Board found that the flightcrew did notindicate that it was concerned about holding in icingconditions; however, it was noted that there were somepotentially distracting events during the holding period.The cockpit voice recorder (CVR) recorded about 15minutes of personal conversation between a flightattendant and the captain from 1528:00 to 1542:38. CVRalso recorded music playing for about 18 minutes, aswell as the sounds of the captain’s departure from thecockpit for about 5 minutes to use the rest room.

According to 14 CFR Part 121.542 (the “sterile cockpit”rule) and FAA staff testimony at the Safety Board’spublic hearing on this accident, holding at 10,000 feet orabove is not considered to be a “critical” phase of flight.Although the presence of the flight attendant and themusic could have been a distraction to the flightcrew,both pilots appeared attentive to flight-related dutiesimmediately before, as well as during, the roll upset.Thus, the Safety Board concluded that neither the flightattendant’s presence in the cockpit nor the flightcrew’sconversations with her contributed to the accident. TheSafety Board also noted that the AMR Eagle ATR 72

flight manual gives the captain the authority to declare“any other phase of a particular flight” a critical phasedepending on the circumstances, thus permitting thecaptain to invoke the sterile cockpit rule. The SafetyBoard concluded that a sterile cockpit environmentwould have likely reduced flightcrew distractions andcould have heightened the flightcrew’s awareness to thepotentially hazardous environmental conditions inwhich the airplane was being operated. However, thesterile cockpit environment would not have increasedthe flightcrew’s understanding of the events that eventu-ally transpired. Nonetheless, the Safety Board recom-mended to FAA that it evaluate the need to require asterile cockpit environment for all air carriers holding incertain weather conditions, such as icing and convectiveactivity.

In this accident, the Safety Board attempted to determinewhy the crew of flight 4184 was unable to successfullyrecover the airplane and prevent the accident when theflightcrews of other ATR airplanes involved in the priorincidents were able to do so. At the time of the accident,the AMR Eagle pilot training program did not include an“unusual attitude” or “advanced maneuvers” segment(nor was such training required). During simulatortraining, AMR Eagle pilots were not exposed to aircraftattitudes that were typically beyond those used fornormal operations or considered unusual, and they onlyexperienced an abnormal pitch attitude when theypracticed emergency descents.

In four separate safety recommendations over the past 27years, the Safety Board has addressed the issue ofunusual attitude training. FAA’s unfavorable responsesand failure to require such training have resulted in theSafety Board classifying FAA’s past actions as “Unac-ceptable” in three of the four cases. In the fourth case,Safety Recommendation A-93-72, FAA’s actions topromulgate rules to bring most 14 CFR Part 135 sched-uled passenger operators under 14 CFR 121 trainingrequirements (which include the use of simulators) wasclassified “Closed–Acceptable Action” on August 29,1995. However, the Safety Board continues to expressconcern that this does not necessitate a requirement toprovide unusual event/attitude training.

Based on the circumstances of this accident, the historicaldata of similar accidents, and safety recommendationspreviously issued by the Safety Board, FAA, in August1995, in joint cooperation with the aviation industry,issued an FAA Inspector Handbook Bulletin detailing aprogram that encourages air carriers to implementadvanced maneuver/unusual attitude training in theirpilot training programs. AMR Eagle implemented anunusual attitude training curriculum into its pilottraining syllabus, action that the Safety Board supports.Additionally, while the Safety Board is encouraged byFAA’s latest position regarding unusual attitude/eventstraining, there remains a concern that the lack of a


required program might result in some carriers notproviding unusual attitude training, and that theirrespective training programs might be insufficient todemonstrate the cause for and the recovery from aircraftattitudes that are not considered to be “normal.” Thus,the Safety Board recommended to FAA that it amend theFederal Aviation Regulations to require air carriers toprovide standardized training that adequately addressesthe recovery from unusual events and attitudes, includ-ing extreme flight attitudes, in large, transport categoryairplanes.

Footnotes

(1) National Transportation Safety Board AircraftAccident Report, NTSB/AAR-96/02,DCA95MA001, In-Flight Icing Encounter and Lossof Control, Simmons Airlines, dba American EagleFlight 4184, Avions de Transport Regional (ATR)

Model 72-212, N401AM, Roselawn, Indiana,October 31, 1994.

(2) National Transportation Safety Board, SafetyReport, NTSB-SR-81-1, September 9, 1981.

Gregory A. Feith is a Senior Air Safety Investigator, MajorInvestigations Division in the Office of Aviation Safety,National Transportation Safety Board (NTSB). He has been anircraft accident investigator at the NTSB for over 16 years,and was the Investigator-In-Charge (IIC) of the followingrecent accidents: American International Airways, DouglasDC-8 at Guantanamo Bay, Cuba; USAir Flight 1016, DouglasDC-9 at Charlotte, North Carolina; American Eagle Flight4184, ATR -72 at Roselawn, Indiana; ValuJet Flight 592,Douglas DC-9 in the Florida Everglades. He is a co-author ofNTSB Safety Study - “Commuter Airline Safety.”


Introduction

Over the past thirty years, a long-term associationbetween a helicopter’s external load capability and theforest products industry has taken place throughout theworld. It is a growing industry. From its early days inCanada and the Northwestern United States, heli-logging is conducted now on every continent.

The expansion of helicopter long-line operations hasbeen due to several obvious reasons. As the demand forhigh quality, old-growth timber increases, easily acces-sible stocks of the resource decline. Conventional timberharvesting by surface road systems using ground ma-chinery to move the logs to a central loading area (toyard)(1), has, in some locations, been replaced to largedegree by helicopter long-lining. Beyond timber harvest,helicopter operations in the forest products industry nowroutinely include removal of paper and pulp wood inplantation growth softwood and hardwood, cedarshingle and shake products, and even Christmas trees inNorth America. Helicopters are routinely used todaythroughout Pacific Rim rain forests and South America,especially in the Amazon River drainage. Helicopterlogging is expanding accessible timber resources fromthe northern latitudes of Scandinavia to Chile, Australia,and New Zealand.

In some parts of the world, the pressure of environmen-tal restrictions has increased the use of helicopters whereconventional road system construction threatens water-courses, fishery resources, erosion-prone hillsides, andother delicate ecosystems. Public opinion in somepolitical regions has forced “clear-cut”(2) logging to behidden by margins of standing timber or areas awayfrom view.(3) In North America, helicopter loggingoperators commonly complain that service sites, sup-port-ship landing sites and refueling operations requireengineering standards driven by environmental require-ments, rather than by operational considerations. Envi-ronmental factors affecting service landings, drop andloading areas, and forest helipads(4) have become issuesaddressed by various federal, state, and local govern-ments.

As a result of six helicopter accidents in an 18-monthperiod between 1 January 1992 and 30 June 1993 by a

small number of helicopters and pilots in a limited forestproduct area, a US workers’ occupational safety agencycalled the first and second helicopter logging safetyconferences.(5) In a timber resource area of southeastAlaska producing 47 to 70 million board feet annually,(6)an estimated 25 logging helicopters (including supportships), operated by approximately 50 pilots, suffered anextraordinarily high accident rate in this limited statisti-cal model.(7)

A working group was formed by the National Instituteof Occupational Safety (NIOSH) late in 1993, bringingtogether government agency representatives and indus-try managers. In February 1995, the working groupsponsored the first helicopter logging safety workshop inKetchikan, Alaska, attended by most helicopter loggingindustry managers and helicopter manufacturers.

Between 30 June 1993 and 1 July 1996, the accident rate forhelicopter logging operations in the same area was zero.(8)

Timber Removal by Helicopter Long-line

Obviously, sustained accident-free logging by helicopterrequires carefully directed cooperative effort betweenprofessional forest workers and helicopter operators.Accident records and statements made at recent confer-ences of North American heli-logging operators suggesta relationship of accident rates to the “learning curve” ofstart-up companies.(9) Mishaps to “under-capitalized”companies (or logging divisions) may be much higherthan those well established with large maintenancefacilities and in-house training programs.(10) While farmore single-engine, light and medium lift helicopters arein service, more tonnage is transported by fewer, twin-engine heavy-lift helicopters. The distribution in accidentdata for 62 long-line accidents (United States 1980-1994)(11) indicates a far higher loss rate among single-enginehelicopters, yet engine failure/power loss were notnecessarily dominant causal factors. Sidestepping theargument between twin-engine heavy-lift advocates andmore numerous, single-engine medium lift machines, theundeniable issues in most all post-logging crash analysesmay be evenly balanced between human performanceissues and design/mechanical ones.

Helicopter Logging Mishaps:Applying Lessons In An Expanding Industry

Douglas R. Herlihy MO3194


Prior to actual lifting operations, timber cutters aretypically transported to remote areas by support helicop-ter(12), often clearing and constructing the first helipad.Timber in 3000 to 10,000 pound sections is felled and cutto transportable lengths in a remote area or unit(13) bythe forest workers. External load operation typicallyemploys a long-line detachable cable from a quick-release belly hook, extending 100 to 200 feet to a remote,electrically operated load hook. Short cables (chokers)are hooked to this remote hook. In setting up for theinitial timber removal, and at regular intervals thereafter,a number of loop-ended chokers(14) are transported tothe site, hanging from the helicopter’s remote hook.

The remote hook is electrically actuated from a cycliccontrol, and opens by gravity at the end of the long linecable. The belly hook, attached directly to the helicopterat its center of gravity(15), is generally electrically andmanually releasable by cyclic and floor mounted controlsrespectively. While the remote hook is free to rotate 360degrees at the end of the long-line cable, the belly hook isnot. Typical installations of belly hooks open toward thefront of the aircraft, an issue to be discussed further inthe paper. The logging pilot visually “flies the hook,”continuously leaning out of a bubble window, to posi-tion the hook or the load at the location of the groundpersonnel.

Single logs or multiples of logs, or “turns,” totaling the“target” weight(16) are selected by forest workers inradio communication with the helicopter crew. Chokercables are hooked to the log ends so as to avoid snaggingor other interference at the time of lift. The other end isattached to the remote hook with the helicopter hoveringoverhead in out-of-ground-effect hover, at the length ofthe long-line. The ground personnel(17) must be skilledin estimating the weight of the turns, as aborting the liftor releasing the load once hooked up is understandably adangerous event.(18) With a choker eye in the remotehook, the signal to lift is given by the hooker and theturn is lifted and flown to the log release area, either ondry land or into a water drop area. Individual weightsare typically recorded by the flight crew from readingson a load cell, or strain gauge attached electrically to thebelly hook. Operators’ revenues are generated as afunction of weight carried, and the pay and bonuses oftimber cutters (fallers) and hookers are likewise reflectedin these readings.

Working Environment, Terrain, Weather, and RemoteLocale Hazards to Long-line Helicopters and FlightCrews

As with most risk-management analysis, higher expo-sure to operational cycles equals higher potential loss.Fixed-wing aircraft naturally have higher accident ratesbeginning at top-of-descent than at cruise operation.Heli-logging operations normally expose pilots andaircraft to hundreds of lifting-transporting-unloading

evaluations over steep, forested hillsides or “logged-off”timberland, studded with destructive hazards to success-ful autorotation. Inasmuch as the timber is being ex-tracted by helicopter, the area is likely steep, undulating,inaccessible to ground vehicles, and complete with itsown area micro-climate of mechanical turbulence andup-slope condensation. Helicopter logging pilots rou-tinely spend seven to ten hours at altitudes of 250 feetand below, hovering or in slow transit. This aerodynamicdisadvantage (slow forward airspeed and out-of-ground-effect), of course, produces increased downwash veloci-ties (induced velocities) and increased power require-ments for any given weight of helicopter/load combina-tion. In many operations, logging pilots rarely findthemselves in “ground effect” or with the advantage of“translational lift.”(19) In addition to continuous highdemands on the logging helicopter’s engine(s), drivetrain and rotor systems, the pilots are faced with continu-ous high tasking as they operate the machine near thetop of the power-required vs. power-available curves.

To most helicopter pilots, operating within an envelopewithout sufficient altitude or sufficient airspeed tosuccessfully autorotate in the event of a power loss is acondition to be avoided, at least on a routine basis, or forlong periods of time. Helicopter pilots refer to thisheight/velocity envelope as a “dead man’s curve.”(20)High forward airspeed and/or sufficient altitude in theevent of engine failure is fundamental to safe emergencyoperation. Successful autorotation, from its aerodynamicparameters at entry to its selection of landing zone, maybe purely academic, given the nature of the terrain belowthe logging helicopter. The addition of a five ton externalload to a pilot’s emergency procedures multiplies thetasking considerably.

Where operators set no limitations other than certificatedmaximum gross weight, accident investigations revealthat helicopter/load combinations regularly exceed legallimits. Due to the dynamic loads in the initial lift, theoscillation of the load in transit, and the inertia of theload at delivery, many operators have set “target” limitsof approximately 80% of maximum allowable externalload in calculating the “turn” weights to be lifted.Routinely, logging pilots communicate initial targetweights after calculating density altitude, performance,and fuel load. The target weight of each turn can beincreased as fuel is burned throughout each operatingcycle.

Crash survivability depends on a number of factors,including, but not limited to, “maintaining a livablevolume, restraining the occupant, limiting the crashloads and providing a means and time to escape.”(21)Helicopter logging presents several disproportionatelyhazardous issues in the survival factors equation. Inaddition to the tall trees, stumpage, and steep terrainover which the aircraft operates in OGE hover, helicopterpilots normally fly with little or no upper torso/shoulder


restraint. Continually leaning into a bubble window tolook below, such restraint is severely limiting. Whenpower loss or mechanical malfunctions occur from deepwithin the hazardous limits of the height-velocityenvelope (“dead man’s curve”), impact forces can bebeyond non-injurious human tolerance.

Hazards to the Forest Workers

The exposure to traumatic injury in helicopter logging is,of course, not limited to that faced by flight crews.Besides the standard helicopter operating cautions,helicopter logging adds a level of risk that is likely notfound outside of combat situations. Because the greatpower requirements with heavy loading in a hoverproduce high downwash velocities, forest workers areexposed to considerable missile hazard in the form ofbroken branches, tree tops, or even high velocity woodchips made airborne by the helicopter. The possibility ofpremature load release over workers, or intentionalrelease due to insufficient power or engine failure, isever-present when logs are hoisted. In plantation loggingof paper and pulp timber, lightning strikes attracted bythe helicopter’s static potential or particle field have beenfatal to ground workers near the long-line cable.(22)

Support/Utility Helicopters; Overloading, ConfinedSpaces & Performance

Hazards to workers under, or near long-line helicopters,notwithstanding, considerable risks are encountered byground crews when transported by support helicopters.Light helicopters carry fallers (timber cutters), hookers,mechanics, forest officials, and equipment operators tolanding pads cut into steep hillsides and to platforms oftimber built on outcroppings and clearings in the forest.Special training and precautions must be undertaken tokeep workers from helicopter hazards such as reducedblade clearance uphill from landing zones, or egress orentry from a low hover, or from protected positionsbeside hillside platforms. Boarding from and egress to“crouch and wait” positions must be trained and briefedby the pilots. Protective equipment and individualcommunications are mandatory in many well-managedoperations.

Support helicopter pilots need to mission plan withextraordinary attention to performance, human andmachine. High density altitude, heavy loads of forestworkers and their equipment require cautious operatingtechnique to avoid dynamic roll-over accidents.(23)

Remote Locations, Distant from Medical Assistance

Traumatic injuries to helicopter flight crews or forestworkers regularly occur in remote locations. An exami-nation of “fatalities by occupation” in the state of Alaskabetween 1990 showed occupational injuries to forestworkers to rank among the top four most hazardous

occupations.(24) Helicopter accidents and forest workertraumatic injury are routinely exacerbated by the diffi-culties in reaching the injured, distance to medicalassistance and delays in communications. Many profes-sional helicopter logging operators insist on equippingevery worker with FM communications portable radios.Helicopter crews communicate with workers and servicelanding base stations on VHF-FM, utilizing VHF-AM forair traffic control only. Recommendations by US federaland some state agencies currently include medicaltraining for all forest workers, due to the record of delaysin medical response.

Flight Crew Duty Cycle and Tasking Requirements

While individual “turns” amount to a single round tripfrom the pick-up site to the release site and back, thelogging pilot (pilot flying) remains at the controls, withfixed concentration on the location of the remote hook,for an entire fuel load of the helicopter. The periodbetween refueling at the service landing is referred to asa single cycle. Typically, cycles last an hour to hour andfifteen minutes, and account for 40 to 60 turns. Withhelicopters capable of lifting 8000 to 10,000 pound turns,it is common for crews to transport upwards of 1000 tonsper day(25), all the time concentrating on the movementof the remote hook and load, more than 200 feet below.In single-pilot operations in light and medium lifthelicopters, the pilot must divide his attention for shortscans of instruments, though modifications to placecritical instruments near his vision at the side bubble arecommon. Also in single-pilot operations, the load tally isrecorded by selected ground personnel via radiocommunications from the logging pilot.

Under current or projected US Federal Air Regula-tions(26), no limits have been placed on flightcrewmembers’ flight and duty time. Investigationsfollowing helicopter logging mishaps have foundlogging pilots to routinely fly from seven to nine hoursdaily. However, flight time in excess of ten hours daily,while not common, has been recorded. Many pilotsreport duty cycles of ten days between days off.

“Flying the hook” requires the logging pilot to concen-trate on the movement of the remote hook while in anout-of-ground-effect (OGE) hover above trees, or in slowtransit to and from the loading and dump sites whiletrailing the long line. Routine monitoring of pressuresand temperatures is understandably less, and morereliance must be placed on audible and secondarycautionary panels and other audio/visual enunciators.

While no attempt can be made to measure or quantifylogging pilots’ in-flight tasking requirements or stressindicators in this paper, investigations following heli-logging mishaps and interviews of helicopter loggingpilots suggest high levels of tactile tasking and concen-tration for long periods of time that may easily exceed


that experienced routinely by any other pilot require-ment. The high level of concentration for extendedperiods of time on the location and movement of theexternal load hook below the helicopter suggests a stateof intense mental focus. Instrument scan and existentialawareness outside of the range of hook movement, bynecessity, has low priority.

It has been suggested, by at least one heavy-lift operator,that owing to this level of required pilot focus andattention, added emphasis is placed on emergencyprocedure review. Pilots of this company are urged toplace extra emphasis on memorizing “bold-face items”on emergency checklists, due to the “principles ofrecency and primacy.”(27) Obviously, any delays areunacceptable in recognizing malfunctions and reactingaccordingly, while focused outside in a 200 foot hover,out of ground effect.

Economic Pressures Influence Pilot and Forest WorkerDecision-Making

Sales of timber are calculated generally by board feet ofresource recovered from a given area. Timber removedfrom the forest by helicopter is valued (initially)(28) byits weight, and compensation for those who produce orcarry it in log form is calculated on that weight. Whilemany pilots and forest workers are paid a base rate, mostworkers benefit from high weight production. Theincentive to carry maximum loads, to work extendedhours, and to push the envelope of human and equip-ment capacity is obvious.

Within recent years, professional managers have recog-nized the advantage of setting “target” limits on logweights. While much of this information is proprietary,these self-imposed limitations reduce dynamic loads onrotor systems, gear boxes, transmissions (and humans).To encourage compliance with a target weight system,bonus incentives using various schemes have beendevised to reward “hookers” for selecting preciseallowable weights, penalizing them for overweight andunderweight turns as well. Continual communicationand feedback between flight crew and forest workersduring long-line operations can be seen as a naturalextension of proper “crew-resource-management”(CRM), contributing to the safety of workers and thereliability of the equipment.

Training, Currency, and Testing Requirements

At this point in time, within the US helicopter loggingindustry, proficiency and competency standards arenearly all industry driven. Major operators have gone onrecord in public forum to resist regulatory oversight oftheir training, checking, or testing by the federal govern-ment.(29) US Federal Air Regulations(30) require demon-strated knowledge and skill with respect to the rotor-craft/load combinations Class A, B, and C(31), specifi-

cally only once in a career, if currency is maintained inan aircraft of the same type within the past 12 months.The demonstration of knowledge and skill, in the case ofa sole proprietor/operator, is presented to a representa-tive of the (US) FAA Administrator. However, in the caseof larger companies (2 or more pilots), the demonstrationof knowledge and skill to authorities is only required ofthe operator’s chief pilot. Pilots in these larger companiesmay demonstrate their competency to the chief pilot orhis assistant, except in the case of Class D operationswherein personnel are hoisted.(32)

At present, other than those training requirementsimposed by the inspector assigned to the particularoperator, no US regulation currently requires initial orrecurrent or upgrade training of pilots in specific rotor-craft/load combinations. Likewise, Canadian operatorsreport a similar approach has been taken by TransportCanada. Quoting from a letter from that Canadianagency at a US national forum, a Canadian operatorstated, in part,

It could be concluded that the safety responsibil-ity of Transport Canada should be biasedtowards large passenger air operations, and thatother activities must be evaluated for the level ofrisk using the probability of failure and impactassessment approach…In view of the above, andthe fact that no passengers or innocent bystand-ers were killed or injured, this type of operation(helicopter logging) must be rated as a mediumrisk, but low impact. As a consequence, the useof lengthy, expensive, and involved certificationevaluation is a difficult exercise to justify on cost-effective grounds.(33)

Accident investigations by the (US) National Transporta-tion Safety Board following helicopter long-line acci-dents in the period 1986 to 1993(34), gave no indicationof finding any company training records (nor were theyrequired) beyond letters on file indicating proficiency,competency, and currency as required by FAA under thelimited guidelines of FAR part 133.(35)

External Load Helicopter Accidents (U.S. data)1980-1994

Data from the 15-year period ending in FY1994 records147 United States external-load mishaps, classified asaccidents. 73 (50%) of these accidents/events occurred inthe forest industry, either in log lifting or in forestproduct related work.(36) Of these 73 accidents orevents, 62 were related directly to log-lifting or wereotherwise logging-related, 31 crashes (50%) involved atleast one fatality or serious non-fatal injury.(37) Inaddition to the higher logging-related crash rate, theindustry had a fatal/serious injury rate considerablyexceeding those sustained in non-logging external-loadcrashes.


During the same 15-year period, the combined fatal/serious injury crash rate for external-load helicopters innon-logging operations, was 25 total crashes involving afatality and 30 crashes involving serious injury.

Case Histories as Examples of Pilot and MachineOverload

Case 1: Economic Influence, Pilot(s) Decision-making,Overload, Regulatory oversight, and the belly hook (38)

On 19 February 1993, a Bell 214B, operating near the startof its fifth hour of external-load logging on Prince ofWales Island, southeast Alaska, crashed near the logdrop point. Witnesses told investigators that the helicop-ter had been hovering at 200 feet as it set down the turnof two logs at the drop zone. The helicopter began erraticpitching and yawing about all three axes, followed by arapid descent in a right turn. The long line was notreleased, and approximately 6000 pounds of logsdragged a short distance before snagging on largestumps. Still attached to the belly hook, the long-line wasseen to pull taut and the helicopter pitched forwardabout the attachment point and crashed on the face of a60-inch diameter stump. The entire front of the machinewas compromised and the accident was fatal to bothpilots. There was no indication of fire.

At the scene, investigators found that the right cyclicactuator extension rod separated as a pre-impact event,five inches below the attaching point on the main rotorhead. Multiple “peening” marks on the side of thetransmission housing matched the blunted edges of thenewly separated actuator rod. Laboratory tear-down ofthe actuator found binding in the internal sphericalbearing (bearing to race) of the actuator, through whichthe piston rides. The bearing normally affords limitedlateral movement to the piston to allow for side to sideflexibility during pitch changes. The binding limited the

lateral movement and fractures originated from normalstress risers at the threaded portion of the rod, onceoutside of the actuator housing. While the laboratoryteardown identified the “proximate cause” of the acci-dent, the operational factors revealed a more completepicture.

Influence, Decision-making, OverloadLoad tally sheets were found in the wreckage thatrecorded the “turn” weights for the past five hours ofoperation. Both pilots, as co-captains, alternated piloting(from the left bubble window) upon each 70 minutecycle. Based on the number of turns per cycle, round-trips were taking approximately two minutes betweenthe pickup and release points. The pilots were salaried,with bonus pay based on total weight lifted during theiroperating period.

The aircraft’s certificated weight and balance data werecompared with the load sheets. The limiting externalload weight was calculated to have been 5700 pounds atthe beginning of each cycle, increasing in linear measureas the fuel burned off, to 6900 pounds allowable at theend of each cycle. The load sheets listed 20 percent of theturn weights to have been between 6400 pounds early inthe cycle to over 8000 pounds late in each cycle. In thefatal fifth cycle, with only five turns (10 minutes) of fuelburn-down, one of the single log turn weights wasrecorded at 7000 pounds. In each of the more than 140turns recorded, the entry showed a cardinal number, adecimal point and a number to the 10th’s place. Forexample, log entries of 7000 were indicated by an entryof 7.0. The last line showed an entry of merely “6,”suggesting that the entry was being made at the time ofthe delivery hover when the failure occurred. The logsattached to the long-line cable at the crash site wereestimated to have weighed 6400 pounds.

Logging

50%

General

7%

Hi rise

4%

Powerline

8%

Seismic

8%

Firefighting

5%

Construction

18%

External Load Helicopter Accidents by ActivityUnited States 1980-1994


Regulatory oversightUnder US law, the operator of this heli-logging companyoperated with the authorization of FAA under Part 133(Rotorcraft External-Load Operations). Principal Opera-tions and Principal Airworthiness Inspectors (POI andPMI) were assigned to the certificate to provide surveil-lance and oversight. The operator’s home base, andcertificated maintenance facility, were located approxi-mately 2000 miles south of the logging activity, inCalifornia. The investigation revealed that neither POInor PMI had ever inspected a logging operation con-ducted by their operator in Alaska, or elsewhere, thoughthe operator had been operating in that area utilizingmultiple helicopters and a 10,000 square foot mainte-nance facility for two years. In response to inquiry, FAAinspectors told NTSB that the lack of government budgetand time resources prevented this level of surveillance.

The belly hookOne issue in this accident remains unresolved. The 150foot long-line cable, the electrically-actuated remotehook, and two twenty-five foot (25) choker cablesremained at the crash site, connected to the logs andstretched from the log drop area to the wreckage. Theloop-eyed end of the long-line cable was found intactand unattached to the helicopter as it lay resting invertedat the location of impact. The loop end was sound. Thebelly hook was closed. Witnesses reported that the cableremained attached to the helicopter throughout itsdescent, and it is unknown whether the pilots hadattempted to jettison the long-line either electrically ormanually by the foot pedal near the feet of the right-seatpilot (non-flying pilot). Undoubtedly, the eye of the long-line was released at or just before impact. What is notknown is whether the hook was open as the helicopterdescended, or opened and closed at impact.

The belly hook is of a type in common use throughoutthe world on many different external load helicop-ters.(39) The hook is hinged at the rear so as to be loadedin the closed position from the front end. A “keeper” orsnap clip prevents the loop of the cable from falling outof a closed hook. When opened electrically or manually,the hook swings downward to the rear, limited toapproximately 80 degrees of movement by a tang andbumper at the rear of the assembly so as to cushion theshock and limit the swing of the hook when a load isreleased. Shallow wear on the hook to a depth of 1/16thof an inch was found, forming a small groove in thebottom of the hook. That wear was found to be commonin most hooks examined. What was revealed in thecourse of recreating the accident helicopter’s flight pathand location of the dragging logs was that, even if thehook had been in the open position, the limit of thehook’s opening would have likely prevented the releaseof the loop. The design of the hook facilitates release of anear-vertical load; however, it is not designed to releasea load if the helicopter is in forward flight under trailingload conditions.

The belly hook in this accident was removed and testedunder static load conditions, both electrically andmanually; it functioned normally. Due to the condition ofthe wreckage, no continuity could be established be-tween the electrical or manual cockpit release controlsand the belly hook.

Case 2: Economic Influence, Pilot Decision-Making,Overload & Military Surplus Parts (40)

Pilot Decision-making, Economic pressuresOn 2 May, 1993, in steep, forested terrain near CopperHarbor, Alaska, a normal category commercial Bell 204B,in an external-load logging operation, crashed from ahover above its service landing. It had been operating forapproximately five one-hour cycles, carrying logs fromthe 1200 foot hillside elevation in a steep descent to thewater’s edge at sea level. A descending flight path wasreported to be steeply to the water, requiring 11 or 12seconds to carry the log. Round trips between the pickuppoints and the drop zone were recorded to be less than aminute long: 60 turns per hour. Witnesses told investiga-tors that, upon lifting a log at the 1200 foot elevation,forest workers communicated a warning to the pilot thathis “tail rotor was smoking.” The pilot reportedlyacknowledged by saying that he would make “a precau-tionary landing” at the service landing (at sea level). Theload was jettisoned from the remote hook and the pilot,trailing the 150 foot long-line cable, descended to theservice landing. Upon arriving at the service landing, thepilot reportedly entered an out-of-ground-effect hover at150 feet, and began descending vertically to coil or layout the cable below him. At an altitude of 75 feet aboveground level (agl), a major portion of the 90 degree tailrotor gear casing departed the ship with a loud report,and the helicopter entered three rapid turns, impactingnearly vertically. The pilot received fatal injuries.

Interviews with ground personnel revealed that eco-nomic pressures may have played a role in the accident.Reportedly, the pilot had been recently told by manage-ment to adopt a more direct path from the pickup zoneto the drop zone. Previous flight paths used by thelogging pilot were reported to be longer, arcing descent,increasing the “turn” times substantially. Ground crewreported the pilot to have held reservations about “beingable to do it” (the new direct procedure) at dinner on theprevious evening. The pilot and ground crew weresalaried and received bonus payments based on totalweight carried by the helicopter.

Overload, static and dynamicThe certificated maximum gross weight of the helicopter,8500 pounds, allowed external loads to be carriedranging from 3400 pounds at the beginning of each hourto 4000 pounds at the end. Fuel burn rate was calculatedat 600 pounds per hour, and the hook’s limiting weightwas 4000 pounds. Records kept by ground personnel, astransmitted by radio from the pilot, indicated that turn


weights routinely exceeded the 3400 pound limits withinthe first few minutes of each hour. Some of the logweights exceeded the limits by 1000 pounds within 15minutes of each hour, exceeding the hook limit as well.

Laboratory analysis of the 90 degree gear box compo-nents revealed that an input pinion had overheated, theouter race and balls of that bearing had failed, and thecase studs had ultimately failed in shear at the gearboxflange. The order of the failure was not determined,though stud failure origins were low cycle fatigue andoverload at normal stress riser locations at the flangeentry point.

Component originInvestigators reviewed the helicopter’s airframe log bookand data plates for 41 of its drive train components. Nine(9) were found to be manufactured and delivered to theUS military for use on a UH-1 helicopter, two (2) compo-nents were manufactured for use on a commercial 204B,and thirty (30) parts could not be verified as to origin ormanufacturer. Research by NTSB and Bell Helicoptercould not find delivery history on the 30 part-number/serial-number combinations. The 90 degree gear box, thefocus of the crash investigation on scene, had a serialnumber originating with the US Army, but that recordended in 1986 as “removed from service, scrap, excessivewear.” New component cards showing the gearboxoverhaul were found in the operator’s records. Thoserecords indicated that the gearbox had been overhauledin the operator’s own FAA-approved maintenancefacility.(41) The overhaul work order showed that thegearbox had been “disassembled, visually inspected,refitted with seals, painted and placed in service.”

Mixed regulatory signals and difficult solutionsDifferences between US FAA local authorities presentedcomplications for resolution of the component originproblems. The FAA’s Helicopter Directorate, overseeingdesign, certification, and subsequent modifications,sided with Bell-Textron Helicopter in their opinion thatonly factory-approved facilities could overhaul Bellcomponents, and these must be carried out in accordancewith the manufacturer’s overhaul and part manuals. Theoverhaul facility, located in a distant FAA region, did notmeet Bell criteria for a factory-approved facility. More-over, the local Flight Standards District Office (andassigned inspectors) were located in that distant region.That FAA regional office, having jurisdiction over thelocal FAA office and the operator’s overhaul facility aswell, rejected the contention that the overhaul wasimproper. The accident occurred in yet a third region(Alaska) and was investigated by the Alaskan Region’sinspectors. According to Alaskan FAA inspectors,violation reports were filed following their examinationof records and the operator’s practices. These violationscited the operator for using “unapproved parts,” im-proper overhaul, and improper record keeping. Seriousand substantial penalty recommendations for the viola-

tions were forwarded to the FAA region of the operator’shome office. That region’s sanction of the operator waslimited, and the case was closed.

Primary Lessons From Helicopter Logging Mishaps toReduce Frequency and to Improve Survivability

• Flight crew preflight preparation and in-flightdecision-making must be extraordinarily profes-sional by pilots for both long-line external-loadoperations, as well as confined-space operations ofsupport ships. In the case of external loads, given thehighly repetitive, maximum effort operations, pilotsmust consider human limitation as a self-critical facetof their planning.

• Operators must adopt and enforce realistic targetweights for external loads, well short of maximumcertificated operating weight limitations. Overloadsof target weights must be viewed as unacceptable,and overloads of legal limits must be consideredviolations of company policy. A commitment bymanagement must be to limit airframe loading toextend reliability of time-limited parts.

• Recognizing that highly repetitive operations areconducted at the design limits of helicopters oftenoriginally designed for utility and cargo duties,operators should review original overhaul andreplacement intervals with manufacturers to assignincreased frequency of inspection (or replacement) ofrotor systems and critical drive train components.

• Operators need to establish reasonable flight andduty time limits.

• Operators should develop standardized and profes-sional training programs for flight crews and forestworkers engaged in helicopter logging operations.Recurrent and upgrade training should be carefullymonitored to avoid the development of hazardousoperating procedures. Chief pilots should fly withlogging helicopter captains on a regular basis.

• Emergency medical training, equipment usage, andcommunications should be part of all crew trainingin helicopter logging operations.

• Maintenance managers must recognize the hazardsand limitations of using unapproved parts, or thoseparts without complete maintenance histories.

Future Accident Analysis and Areas of Interest inHelicopter Logging

• The level of required concentration and exactingperformance for long-line helicopter pilots engagedin a hoist operation is undeniably high. When theseprecise maneuvers become highly repetitive, there


may be a tendency to allow concentration on thehook below to override normal wider focus andcognitive function. Some anecdotal reports suggest astate of “mesmerizing” focus is assumed by loggingpilots as they concentrate for hours at a time on thelocation and movement of their long-line hook.Reaction times and procedural responses to emer-gencies deteriorate after periods of intense concen-tration. Accident investigators should be alert to thepossibility of such phenomena when examininghuman performance factors in helicopter loggingaccidents. The issue should be a candidate for morescientific and verifiable examination.

• Future survivability studies should include effectiveand pilot-usable upper torso restraint for loggingpilots, recognizing the ergonomic requirements ofthe bubble window viewing position.

Conclusion

Logging by helicopter long-line need not, and shouldnot, be particularly hazardous in any climate or region.However, helicopter logging operations are not “get-rich-quick” endeavors. They are marginally profitableunless performance and safety are optimized. Accidentrecords consistently underscore the necessity for supe-rior planning by experienced operational management,pilots, and ground personnel. First class maintenanceand equipment, as reflected in both logging and supportaircraft, ground support gear, logging tackle, communi-cations, and emergency equipment, must not be compro-mised at either initial capital outlay or periodic support.Management commitment to safety as the only accept-able key to profitability must be the fundamental operat-ing guide.

Managers must recognize that long-line logging onlyoperates within a delicate envelope of machine andhuman capability, given the enormous loads and poten-tial for disastrous consequences. At this time most facetsof aviation, through new technology, attempt to reducethe criticality of the human system in its relationship toother on-board systems. Many times in recent monthsaccidents have reminded us that the human remains theprimary consideration. In long-line logging, the demandson the flying pilot likely exceed the performance re-quired of most other pilots.

Glossary of terms for investigators at logging sites(42)

Abort Stop or reject a lift of a turn due tooverweight or other problem

Beads Another name for chokers

Belly Hook The helicopter’s hook attacheddirectly to the airframe

Bench A level place on mountainsidechosen for helicopter pad

Bight Any dangerous loop of line thatmay close under tension

Binder Cable used to secure logs to a truckto prevent spilling

Bridle Choking a large log with twochokers

Bubble Logging pilot’s window

Buck To cut logs to length

Bunk Heavy steel frame or cradle for logs,usually on log trucks

Bull Buck Lead or foreman of woodcutters,also known as “woods boss”

Calks Short spikes screwed into the soles& heels of boots to improve footing

Corks NW US name for “calks”

Chase Unhook logs at the landing

Choker A cable with loops on the ends for“choking” logs

Choker Setter A rigging crew member who setschokers

Cull Lowest quality log, or to rejectunwanted material

Cycle Flight time between refueling whilelifting turns

Deck A pile of logs

Donkey A machine for pulling in logs(known as yarding)

Eye A loop splice in the end of a wirerope or cable

Faller Timber Cutter

Felled and Bucked Down timber, cut and ready foryarding

Ground Lead Pulling logs (yarding) to a landingalong the ground

Haulback Line or rigging carrying themainline away from the donkey


Hooker Rigger who chokes logs and hookschokers to helicopter load hook

Hot refueling Refueling the helicopter while theengine is running

High lead Cable system attached to a spar treeto elevate logs being yarded

Knot Bumper Wood cutter who trims limbs fromlogs, works at the landing

Landing Area where loader and/or yarder islocated, log drop area

Loader Any machine that loads logs

Load hook Remote hook on the end of ahelicopter long-line

Long-line Helicopter load line between bellyhook and load (remote) hook

Mainline Cable used to yard logs, either onground lead or high lead

Max turn A turn of one or more logs near thehelicopters maximum capacity

Necktie Choker

Old Growth Virgin timber

Pad Any helicopter landing zone

Peaker Top log on a load

Peeler Best quality log, suitable forplywood manufacture

Ripper A log so large as to need sawing(lengthwise) for acceptable liftweight

Sandbag An inexperienced helicopter copilot,on board for regulations only

Skidder A large wheel or tracked vehicle forpulling logs over ground

Strip runner A forest worker assisting a hookersetting chokers

Turn A log or logs carried on a single tripfrom pickup to dropoff

Turn time Time to fly each round trip

Widowmaker A rotten or broken high limb or top,potential hazard

Footnotes

(1) Removal of felled and cut timber to a centralloading location by several means including cablesattached to spar trees or mechanical spar poles, orby ground haulers or “skidders” or other methods.Over the years, water flumes and helium balloonshave been used with varying success.

(2) A complete harvesting of the timber resource,versus selective cutting of market size timber

(3) Helicopter logging is often conducted in the south-east Alaska archipelagos often out of view frompassing tourist ships, where timber industry inter-ests clash with tourist industry interests.

(4) Landing sites cut out of the forest to accept supportship operations carrying loggers, cutters, fallers,choker-setters and hookers (see Glossary of Terms).Mountainside flat areas, known as “benches,” formnatural landing sites.

(5) National Institute of Occupational Safety & Health(NIOSH); Helicopter Logging Safety Workshops,Ketchikan, Alaska

(6) U.S. Forest Service estimate by report to NIOSH, ofpublic & private land production, for approximately1000 square miles of archipelago.

(7) NIOSH, CDC, Mortality & Morbidity Weekly Report 8July 94 Vol. 43, No 26; “Risk for Traumatic Injuriesfrom Helicopter Crashes During Logging Opera-tions”

(8) On 13 July 96, a CH54 helicopter engaged in exter-nal load logging, crashed near Shelter Cove, Alaska.NTSB Preliminary Report ANC-96-F-A098, fatal toone, serious injury to one, helicopter destroyed.

(9) Proceedings, Helicopter Logging Safety Workshop, USNIOSH Ketchikan Alaska 1-2 March 1995

(10) North American comparisons 1992-1994, using FAArecords of CRI, Rocky Mountain Helicopters,Carson Services, Columbia Helicopters, Silver BayAviation and Erickson Skycrane

(11) NTSB data 1995, 56 Single-engine accidents (90.3%),6 Twin-engine accidents (9.7%)

(12) Notwithstanding US restrictions (14 C.F.R. 133.35)against passenger transport by helicopters withlong-lines attached, most operators of large ma-chines employ smaller service helicopters for


personnel transport due to helipad size and confine-ment, and cost of operation. Most heavy lift helicop-ters are landed only at the designated service pad,designed to accommodate the long-line machine formaintenance and fueling.

(13) US term for timber sale area

(14) Chokers cables and slings are attached to single ormultiple logs for each evolution.

(15) In single-main rotor helicopters, generally, hardmounted to the transmission above.

(16) Individual lifts and transports referred to as “turns”are estimated by ground personnel so as not toexceed the external load limit of the aircraft (or thehook).

(17) The heli-logging industry term for choker-setters isa “hooker.”

(18) Investigator’s interviews with logging pilots revealthat they would be reluctant to release even anoverweight log, once fully aloft, given the dangersto the forest workers below.

(19) Ground effect at approximately 1/2 rotor diameter,retraining rotor downwash, reducing power-required, and translational lift, where larger massesof air per unit of time, in forward flight, reducesdownwash velocity and the induced power require-ment. J. Montgomery, Sikorsky Helicopter FlightTheory, Sikorsky Aircraft, 1964

(20) Additional height/velocity curve explanations; H.E.Rowland, J.R. Detweiler, Fundamentals of Fixed &Rotary Wing Aerodynamics, Univ. Southern Califor-nia, 1967

(21) Roy G. Fox, Bell Helicopter Textron, Inc. Proceed-ings, 23rd Seminar of ISASI 1992, “Helicopter CrashSurvival Investigation”

(22) Flot v. Scott Paper Co., Mobile Cir Ct. AL CV94-2419, Lighting strike on or near HH43 helicopter inlogging operation was fatal to ground worker. 22July 1992. Helicopter flight continued.

(23) As shown in crash of N750LT, logging supportBell206 at 6500’ level, fatal to forest worker passen-ger, destroying the helicopter, 24 July 1996, WarrenID. NTSB SEA-96-L-A165

(24) Of 365 occupational fatalities in a five (5) yearperiod, the fishing industry lead with 140, followedby pilots with 40 and loggers with 22. NIOSH/CDCdata, Anchorage 1995.

(25) Mark Lindamood, Carson Helicopters Inc., stated inaddress to 1st Annual Helicopter Safety Workshop,1 March 95, “SK61 average 1000 tons a day, six daysa week.”

(26) (US) 14 CFR Part 133 et seq.

(27) Mark Lindamood, VP Logging Division, CarsonServices Inc. Jacksonville, OR, USA

(28) Mill evaluation is often in board-feet as calculatedby certified “scalers.”

(29) George Warren, Chief Pilot, Columbia Helicopters,Inc.; Paul Mavrinac, President, Canadian Aircrane;Jim Neal, Safety Manager, Aerial Forest Manage-ment Foundation; Tim Harper, Erickson Aircrane;Proceedings, Helicopter Logging Safety Workshop, USNIOSH Ketchikan Alaska 1-2 March 1995

(30) (US) 14 CFR Part 133.37

(31) (US) Class A, external load, generally fixed to thehelicopter, not long-line configured; Class B,external load long-line, such as described in helicop-ter logging, or other typical external load construc-tion and equipment installation; Class C, externalload operations such as found in cable laying orwire-stretching.

(32) (US) Class D operations involve personnel hoisting,and require somewhat more stringent testing, initialand recurrent training as defined by the FAAinspector having surveillance responsibility.

(33) Paul Mavrinac, Pres., Canadian Aircrane Ltd.,Proceedings, Helicopter Logging Safety Workshop, USNIOSH Ketchikan Alaska 1-2 March 1995, pp. 100-101

(34) No records of external-load logging helicopter in1994 were found in US NTSB database. Records for1995 are incomplete.

(35) Hobart Bay, Alaska, 23 Feb. 92, 5 fatalities, 5 seriousinjuries, destroyed, ANC-92-F-A040

Ketchikan, Alaska 3 Mar. 92, 2 serious injuries,destroyed, ANC92-F-A044

Elf Point, Alaska, 10 Nov. 92, substantial helicopterdamage, ANC-93-L-A015

Prince of Wales Isl., Alaska, 19 Feb. 93, 2 fatal,destroyed, ANC-93-F-A033

Copper Harbor, Alaska, 2 May 93, 1 fatal, de-stroyed, ANC-93-F-A056


Thorne Bay, Alaska, 8 May 93, 2 minor injuries,substantially damaged, ANC-93-F-061

(36) National Transportation Safety Board, 1995. Nineaccidents (6%) of the 147 total accidents occurred inthe Christmas tree industry, and are included in theforest products accident data.

(37) Combined data from NTSB 1995 and NationalInstitute of Occupational Safety, Proceedings, Heli-copter Logging Safety Workshop, US NIOSH KetchikanAlaska 1-2 March 1995

(38) NTSB case number ANC-93-F-A033

(39) The FAA-Approved hook is a manually and electri-cally operated hook model with a load capacity of8000 pounds.

(40) NTSB case number ANC-93-F-A056

(41) While this particular facility was approved tooverhaul Bell Helicopter components by FAAregion having oversight, Bell-Textron did notextend such authorization to that facility. TheFAA’s Helicopter Directorate in Fort Worth re-mained at odds with the FAA region, supportingBell Helicopter in denying manufacturer’s approvalof that facility for overhaul.

(42) Sources: Workers’ Compensation Board of BritishColumbia, booklet, “Helicopter Operations in theForest Industry” Richmond, B.C. 1995, and “Yard-ing and Loading Handbook” Vancouver, B.C. 1981.

Doug Herlihy is a partner in the firm of Herlihy & Leonard.A former NTSB Operations Group Chairman on the Wash-ington “Go-Team;” as an Anchorage field investigator, heserved as Investigator-in-Charge on accidents highlighted inthis paper. Prior to the NTSB, Commander Herlihy completeda Coast Guard career as Chief, Search & Rescue, Atlantic. Heis an aviator with 16000 hours.


I want to draw attention to a number of Incidents thathave fired my interest in Human Factors in Engineering.

I am not a psychologist or a physiologist and cannotclaim to be a ‘Human Factors Expert.’ However, I amand have been for a number of years a government airaccident investigator, regularly called upon to fulfill therole of Investigator-in-Charge of a wide variety ofaccident and incident investigations.

So why does the Human Factors investigation concernme so?

As an industry we are inclined to boast about our safetyrecord, and in many respects history supports ourclaims. This record has been achieved through closeattention to detail in design, manufacture, maintenance,and operating standards, where practices have oftenbeen characterised by conservative factors of safety.When it has gone wrong and an accident has resulted,meticulous investigation has often identified a visiblecause and the lesson has been learnt. By this processmany types of accidents have been ‘squeezed’ out of thesystem. Having eliminated some of the more obviouscauses of accidents, we are left with those causes that aremore difficult to identify and address. Consequently, inrecent years the accident rate has remained reasonablyconstant. It appears to be generally accepted that 70-80%of accidents are now attributable directly to humanfailing in the operation of the aircraft.

The Department of Transport 1991 Air Traffic forecastsfor the United Kingdom include forecasts of annualtraffic growth rates up to the year 2005.

Air traffic passenger movement at UK airports isexpected to increase by between 75% in the loweconomic growth scenario and 145% in the higheconomic growth scenario over the period. Thesefigures are consistent with an annual growth rateover the period 1989 to 2005 of between 3.5% and5.8%.

These forecasts agree well with a paper produced by theBoeing Airplane Co. and published in Aerospace, thejournal of the Royal Aeronautical Society, in July 1990,which stated

The world-wide fleet of transport aircraft (ex-cluding the USSR) is expected to handle apassenger growth rate of 5.5% a year. By the year2005 the existing fleet of over 8,200 aircraft willgrow to over 14,700. If the accident rates for thelast 20 years are used to forecast the future,annual hull loss could increase from 14 to 20 orthere will be a hull loss about every 20 daysinstead of the current 28 days.

The combination of a constant accident rate and a steadyincrease in air traffic movements inevitably results in anincreasing number of accidents to be investigated.

A study of the total number of civil aircraft accidentsoccurring in the UK during the last 10 years showsannual rates in the range 180 to 307. Although there areconsiderable annual fluctuations, the trend is for a steadyand significant increase in the number of reportableaccidents. 1990 was a record year with the 300 markbeing passed for the first time; since then we haveconsistently passed that figure! If, as currently, up to 80%of the accidents are directly attributable to humanfailing—this is why the Human Factors investigationconcerns me so!

Why engineering!

The June edition of the RAeS Journal Aerospace statedthat the number of “Maintenance Concern” accidents ison the increase and that over the preceding 10 yearswhilst the number of flights had increased by 55% thenumber of “Maintenance Concern” accidents hadincreased by 100%.

Human factor related causes to accidents are not re-stricted to the flight-deck. I have heard the argument thatit is only in the cockpit that actions and results areclosely linked in ‘real time’ and that, as a result, humanfactors are of little or no consequence elsewhere. If amechanic completes a task operating alone and is del-egated the authority to ‘sign off’ the work, against abackground of time pressure with minimum resources oftooling and supplies and in a physically uncomfortableenvironment, he is unlikely to give his best. If the resultsof his work then go without functional or independentinspection until the aircraft is airborne, any error can

Incidents: the Route to Human Factors in Engineering?

David King M03393Principal Inspector of Air Accidents (Engineering)

Air Accidents Investigation Branch, United Kingdom


result in an in-flight incident or worse. Does it matterthat his actions and the final consequences are separatedin time by hours, even days, if in that intervening periodthere was no attempt or opportunity to discover themistake? Time was real enough throughout the task forthe individual and only a sterile period separates causeand effect.

An acceptance that human factors have relevance outsidethe flight-deck has led to expansion of the ConfidentialHuman Factors Incident Reporting Programme (CHIRP)in the UK to accept reports from air traffic controllersand there are/have been proposals to include engineers.However, this remains an issue of considerable debatewith some significant opponents.

At the start of this paper I referred to a number ofincidents that had fired and which continue to stimulatemy interest in this area. I outline three of them below:

Accident Involving BAC One-Eleven, G-BJRT, whichoccurred over Didcot, Oxfordshire on 10 June 1990

The accident happened when the aircraft was climbingthrough 17,300 feet on departure from BirminghamInternational Airport enroute to Malaga, Spain. The leftwindscreen, which had been replaced prior to the flight,was blown out under effects of the cabin pressure whenit overcame the retention of the securing bolts, 84 ofwhich, out of a total of 90, were of smaller than specifieddiameter. The commander was sucked halfway out ofthe windscreen aperture and was restrained by cabincrew whilst the copilot flew the aircraft to a safe landingat Southampton Airport.

The following factors contributed to the loss of thewindscreen:

• A safety critical task, not identified as a ‘Vital Point,’was undertaken by one individual who also carriedtotal responsibility for the quality achieved, and theinstallation was not tested until the aircraft wasairborne on a passenger-carrying flight.

• The Shift Maintenance Manager’s potential toachieve quality in the windscreen fitting process waseroded by his inadequate care, poor trade practices,failure to adhere to company standards, and use ofunsuitable equipment, which were judged symptom-atic of a longer term failure by him to observe thepromulgated procedures.

• The British Airways local management, ProductSamples and Quality Audits, had not detected theexistence of inadequate standards employed by theShift Maintenance Manager because it did notmonitor directly the working practices of ShiftMaintenance Managers.

Features of the windscreen change:

• Short staffing — Night shift of 7 down by 2.

• Shift Manager does job himself and alone (10 yearsRAF, 23 years BA — exemplary record).

• The A/C was remote and took the Shift Manageraway from the location of his other duties.

• Time pressures — the morning shift was shortstaffed — aircraft was programmed for a wash.

• The task was conducted between 0300-0500 hrs. — atime of Circadian lows.

• Shift Manager was on his first night work for fiveweeks.

• The Maintenance Manual was only used to confirmthat the Job was ‘straight forward.’

• The IPC was not used — the IPC was misleading.

• Shift Manager assumed the bolts fitted were cor-rect—incorrect bolts fitted 4 years before.

• Shift Manager chose bolts by physical matching —main stores below minimum stock level.

• Shift Manager ignored the advice of the storeman onbolt size.

• Shift Manager got bolts from uncontrolled AGSCarousel with faded labels in dark corner.

• Shift Manager did not use his reading glasses at anytime.

• Shift Manager arbitrarily increased the torque from15 pound-foot to 20 pound-foot.

• Shift Manager didn’t notice excessive countersinkingor that next window was different.

• The safety raiser used provided poor access.

• Shift Manager failed to recognise difference in torquewhen fitting the corner fairing.

• Shift Manager rationalised the use of different boltsthe next night when doing a similar job.

Was this just one reckless individual?

What had happened to QA? What was the organisationalCulture? What was the effect of internal and CAAaudits?


Incident to Airbus A320-212, G-KMAM, at LondonGatwick Airport on 26 August 1993

The incident occurred when, during its first flight after aflap change, the aircraft exhibited an undemanded roll tothe right on takeoff, a condition that persisted until theaircraft landed back at London Gatwick Airport 37minutes later. Control of the aircraft required significantleft sidestick at all times, and the flight control systemwas degraded by the loss of spoiler control.

The investigation identified the following causal factors:

• During the flap change, compliance with the re-quirements of the Maintenance Manual was notachieved in a number of directly relevant areas:

During the flap removal, the spoilers wereplaced in maintenance mode and moved usingan incomplete procedure: specifically the collarsand flags were not fitted.

The re-instatement and functional check of thespoilers after flap fitment were not carried out.

• A rigorously procedural approach to workingpractices and total compliance with the MaintenanceManual were not enforced by local line management.

• The purpose of the collars and the way in which thespoilers functioned were not fully understood by theengineers. This misunderstanding was due in part tofamiliarity with other aircraft, and contributed to alack of adequate briefing on the status of the spoilersduring the shift handovers.

• During the independent functional check of theflying controls, the failure of spoilers 2 to 5 on theright wing to respond to right roll demands was notnoticed by the pilots.

• The operator had not specified to its pilots anappropriate procedure for checking the flight con-trols.

Features of the Flap change

• LAE and team were new to the task.

• LAE was A320 authorised, but the aircraft wererarely seen; this was third party work.

• Planning was limited to a job card, change the flap,and provision of some special tooling.

• Maintenance Manual, A/C Maintenance Taskorientated support system (AMTOSS) format.

• Tooling supplied was deficient or incorrect; therewere no collars for locking the spoiler.

• The LAE requested additional experienced help:none was available.

• Other tasks were tackled during tooling delays, andthere were changes in task allocation.

• The task was carried out during the early hours, atime of Circadian lows.

• The team attempted to remove the flap withoutdisabling the spoilers, but couldn’t.

• The spoilers were disabled without collars or flags, adeviation from the Maintenance Manual.

• The shift hand-over was verbal and the paperworkwas incomplete, leading to the misunderstandingover the spoilers.

• The spoilers were pushed down during the flaprigging.

• The team was more familiar with Boeing aircraftwhere the spoilers auto reset.

• Flaps were functioned, the spoilers were not — adeviation from the Maintenance Manual.

• Duplicates were lead by the day shift engineer.

• The Maintenance Manual was not followed.

• During the flight crew walkaround, there wasnothing amiss to see.

• During the preflight check, three seconds mismatchcontrol/surface position are required to generate awarning.

• Engineers demonstrated a willingness to workaround problems without reference to designauthority — including deviations from MaintenanceManual.

Incident Involving Boeing 737-400, G-OBMM, Over-head Daventry on 23 February 1995

The incident occurred when the aircraft was climbing tocruise altitude after a departure from East MidlandsAirport enroute to Lanzarote Airport in the CanaryIslands. Following an indicated loss of oil quantity andsubsequently oil pressure on both engines, the crewdiverted to Luton Airport; both engines were shut downduring the landing roll. The aircraft had been subject toBorescope Inspections on both engines during the nightprior to the incident flight. The High Pressure (HP) rotor


drive covers, one on each engine, had not been refitted,resulting in the loss of almost all the oil from bothengines during flight. There were no injuries to any crewor passengers. The aircraft was undamaged; both en-gines were removed and examined as a precautionarymeasure.

The investigation identified the following causal factors:

• The aircraft was presented for service followingBorescope Inspections of both engines, which hadbeen signed off as complete in the Aircraft TechnicalLog, although the HP rotor drive covers had notbeen refitted.

• During the Borescope Inspections, compliance withthe requirements of the Aircraft MaintenanceManual was not achieved in a number of areas: mostimportantly the HP rotor cover drive covers were notrefitted and ground idle engine runs were notconducted after the inspections.

• The Operator’s Quality Assurance Department hadnot identified the non-procedural conduct ofBorescope Inspections prevalent amongst Companyengineers over a significant period of time.

• The Civil Aviation Authority, during their reviews ofthe ‘Company Procedures’ for JAR-145 approval,had detected limitations in some aspects of theOperator’s Quality Assurance system, includingprocedural monitoring, but had not withheld thatapproval, having been satisfied that those limitationswere being addressed.

Features of the Borescope Inspection

• The Borescope Inspections were not carried out inaccordance with the procedures detailed in themanufacturer’s Task Cards and the Aircraft Mainte-nance Manual. Specifically:

The two HP rotor drive covers, one on eachengine, had not been refitted after the BorescopeInspections.

A post inspection ground idle engine test hadnot been conducted.

The entry in the aircraft Technical Log relating toBorescope Inspections had wrongly been signedas having been completed in accordance with theAircraft Maintenance Manual.

• The work originally planned for Line was trans-ferred to base.

• There were Line and Base staff shortages, includingthe absence of three Base supervisors.

• There was minimal preplanned paperwork consis-tent with Line Maintenance.

• In order to retain his Borescope authorisation, theBase Controller performed the inspections.

• A/C was remote and took the Base Controller awayfrom the location of his other duties.

• The Line Engineer gave a verbal handover to theBase Maintenance Controller.

• There was inadequate reference to the MaintenanceManual.

• An unapproved reference source was used.

• The lighting conditions were poor.

• There were many interruptions.

• The inspections were done in the early hours ofmorning, a time of Circadian lows.

• There were no post inspection engine runs; this wasa deviation from the Maintenance Manual.

• There had been nine previous occurrences.

• The staff had regularly completed Borescope Inspec-tions in a non procedural manner, failing to replacethe HP rotor drive cover O-rings or to conduct anidle engine run, both specifically required by theAircraft Maintenance Manual.

• The operator’s Quality Assurance system had notidentified frequent deviations from a proceduralapproach and failure to observe the requirements ofthe AMM over a considerable period of time.

• The regulator’s monitoring system had been inef-fective in identifying and making the operatorcorrect the same procedural lapses.

Common Features

• Night shift engineers operating at their Circadianlows. Most Maintenance at night.

• Supervisors tackling long duration, hands-on in-volved tasks.

• Interruptions.

• Failure to use the Maintenance Manual — IPC.

• Confusing, misleading, difficult manuals.


• Shift handovers — poor briefing — lack of com-prehensive stage sheets.

• Time pressures.

• Limited preplanning paperwork, equipment, spares.

• Staff shortages.

• Determination to cope with all challenges.

Many ingredients came together to create these inci-dents, but what if some are there all the time?

Conclusions

The only object of identifying the causal factors andcontributing features of an accident/incident for thegovernment investigator is accident/incident prevention.This means that once a cause has been identified it mustbe accepted by the industry and change implemented toavoid a repeat. If nothing changes, the most elegant ofinvestigations is for nought, a waste of time and effort.

The first hurdle to the implementation of change toaddress a human factor cause is acceptance of the findingthat someone or some organisation failed to performadequately. This involves one or more individuals — aflight crew, a design team, a maintenance crew, a man-agement — accepting that their performance on the dayor over a period of time, perhaps for reasons outside oftheir control, fell below par.

If the subject of the Investigation is an incident withoutinjury or damage, there is generally more of a will-ingness on the part of all parties to the investigation toaccept the findings. When a human factor cause is cited,the burden of coming to terms with the realisation that,for an individual or an organisation, performance hasbeen substandard can usually be accommodated, even ifwith some discomfort. None of us find it easy to accom-modate responsibility for our actions when they lead toan incident or, much worse, an accident, so incidentsoffer us a route to human factors in engineering.

Even when the investigation is of an incident, mypersonal experience is that the collection and analysis ofevidence to produce an acceptable conclusion is verychallenging. Making the connection between individualperformance on a specific task and a more generalconclusion about the personnel or the organisation andits culture is a difficult step. Considering the organisa-tion’s performance within the context of the Regulationsand the role of the Regulator in monitoring compliance isa further step away from the individual occurrence.However, if the causes are systemic, these links are coreto understanding the real causal factors and makingeffective changes.

In the investigation of human factors, the evidence isoften circumstantial, subjective, and sometimes easy tocollect but often impossible to corroborate. ShouldInvestigators, pursuing such an investigation, be con-strained to achieve proof of their findings to meet somelegal definition? Can Human Factors be dissected andanalysed in such a way to provide such proof in mostcircumstances? I believe not.

I believe that the incidents cited in this paper, along withothers that I have investigated, indicate that many of thefactors that came together to contribute to their causesare with us most, if not all, of the time. The developmentof maintenance practices over several generations ofaircraft types has delivered us to where we are today.Are the processes appropriate to today’s high technologyaircraft that we operate in a high pressure, fiercelycompetitive, operational climate? The volume of materialthat the engineer is required to have available andaccessible to perform his task on the aircraft is enormous:is it really presented in such a way that he can be awareof all of its significance? Is information in a large numberof volumes on the shelf or on a microfilm reader readilyavailable and usable by the engineer trying to meet tightoperational deadlines? In moving from Quality Controlto Quality Assurance, are we monitoring the administra-tion of the task and not the quality of the engineeringproduct? Have commercial pressures resulted in mini-mal staff allocations to the task, allocations which rarelymaterialise due to absences for leave, sickness, or train-ing?

Up to the time of the above incidents, all the individualsinvolved were considered to be well qualified, compe-tent, reliable employees selected for management roles.Immediately afterwards they were shocked at what hadhappened and would be condemned by many; but howhad they suddenly changed during the few hours of thetask? The answer, of course, is that they had not! Theindividual must shoulder some responsibility, but thereal causal factors are systemic and do not stop at theindividual, but reside within the culture of theorganisation — an organisation approved by the Regula-tor.

The significance of incidents as rehearsals for catas-trophic accidents is sometimes recognised all too late;these three incidents have identified a wide range ofcommon features conspiring to undermine the pursuit ofquality in aircraft maintenance. What these investi-gations indicate is that there is a need for an independentreview of the way we regulate, conduct, and deliverassured quality in aircraft maintenance.

I believe that incident investigation is the route to humanfactors in engineering. This route through these investi-gations is already telling us something. Are we going tolisten? Are we going to do something?


References

1. Aircraft Accident Report 1/92. Report on the accidentto BAC One-Eleven, G-BJRT, over Didcot,Oxfordshire, on 10 June 1990. HMSO

2. Aircraft Accident Report 2/95. Report on the incidentto Airbus A320-212, G-KMAM, at London GatwickAirport on 26 August 1993. HMSO

3. Aircraft Accident Report 3/96. Report on the incidentto Boeing 737-400, G-OBMM, overhead Daventry on23 February 1995. HMSO

David King is a Principal Inspector of Air Accidents with theUK Air Accidents Investigation Branch. He apprenticed withHawker Siddeley Aviation, during which he obtained aBachelor of Science Degree in Aeronautical Engineering; hethen worked in the Future Projects section. Mr. King joinedAAIB in 1972 and since 1984 has acted as Investigator-in-Charge of a number of large public transport accidents andincidents. He obtained a Master of Business Administration(MBA) from The City University in 1991.


Acknowledgments

The author wishes to acknowledge the technical assis-tance of Raymond Taggart, Ph.D., recently retired fromthe University of Washington School of Engineering. Dr.Taggart’s enthusiasm and persistence were invaluable insolving the metallurgical mysteries. It is Dr. Taggart’selectron microscopic analysis that appears herein. Specialthanks also go to Don Hamill for his insights into thehistory behind the component in question, and to theHonorable Rosanne Buckner, Superior Court Judge, whocompelled the disclosure of critical facts.

The author also wishes to thank his associate counsel,whose patience allowed this investigation to be com-pleted: Francis Fleming of New York; Jean Chevrier ofParis; and Roberto Fuentes of Panama.

Background

At the first international seminar of ISASI hosted inSeattle in 1979, the author co-chaired the event withPrater Hogue of Boeing. The topic of discussion, whichpermeated the gathering, was the potential maintenanceproblem posed by the aging US jet fleet being resold andkept in service by the developing nations and privateleasing companies. The example of the DC-3 (C-47) beingusable indefinitely seemed unmatchable by Boeing 707sand Douglas DC-8s.

Like TV networks looking to profit from residuals of aprime-time series, the major manufacturers were un-willing to set a sunset period for their products. Thegovernment agencies, likewise, held out the hope that aslong as an aircraft was properly maintained, it could beoperated indefinitely. The United States Congress, in theGeneral Aviation Revitalization Act of 1994 (Public Law103-298; see Appendix A) established what might becalled a useful safe life for general aviation aircraft(fewer than 20 passengers) of 18 years. There is no suchuseful safe life for commercial or military aircraft.

This leaves the task to “properly maintain” the com-mercial fleet to the operators and government inspectors,a task that is difficult enough if the part numbers on acomponent really identify the source and stand for thequality required for its certification. This paper tracks the

investigation of the probable cause of a crash off theisland of Contadora, near the western shore of Panama,that killed 22 people. Its purpose is to point out to theinvestigator, the industry, and the government agentsthe need to enforce intellectual property rights, i.e.,ownership of trademarks and part numbers, to createmarking systems that make counterfeiting difficult, andto verify the source of replacement parts—i.e., boughtfrom the manufacturer, not from a secondary source. Thefew additional dollars spent for the replacement partswould be repaid in safe operations. Even if a part failedbecause of some defect, the operator, or the operator’sinsurer, would then find a stand-up entity behind thepart, not a will-o’-the-wisp that left the operator holdingthe bag for a non-existent warranty or an unfortunate airdisaster.

Bogus

“Bogus” is defined in Webster’s New World Dictionary,College Edition, as “Not genuine; spurious; counterfeit.”

On repeated occasions, the existence of “bogus” partsrises to the surface, such as during the 1995 criminalprosecution, by the United States Attorney for theWestern District of Washington, of a Boeing componentsupplier for falsifying Boeing inspection stamps on itsproducts. The public position is always that the parts inquestion are not critical to flight, nor were they installedin production aircraft. The issue is made to appear asmerely a sub-component supplier problem, or a war-ranty problem in which the public has no interest. Thelatter problem is exemplified by Pratt & Whitney discov-ering warranty claims on 707 replacement engines itnever manufactured. A third world manufacturer hadbeen putting Pratt & Whitney identification plates andpart numbers on engines that were sold by a well-knownsupplier.

There have not been any major air disasters in the recentpast attributed publicly to “bogus” parts. The followingcase study demonstrates that the potential does exist.[NOTE: The underlying litigation that prompted thisanalysis has been settled by all parties. Thus, there is nojury verdict determining the cause of the crash. What isset out herein is the analysis and findings of the authorand his team. The investigation by the Panamanian

Bogus Parts—Myth or Fact?

James F. Leggett, Attorney at Law MO0847Managing Partner, Leggett & Kram


government had closed prior to the identification of theproblem with the fuel nozzles.]

Photo 1 shows the major aircraft wreckage after it wasrecovered from the ocean. The witnesses reported thatthe twin engine turbo-prop aircraft had a longer thanusual take-off roll, and that fire came from the rightengine. The cockpit inspection confirmed witness reportsthat the right engine was feathered at or immediatelyafter lift-off, the right fire bottle having been activated.The aircraft entered a slow right rolling turn, whichincreased until it dropped into the sea.

The hull and engines were recovered within a few days,but not before salt water began its work. The engineswere washed and crated. They were shipped to themanufacturer for tear-down under the auspices of thePanamanian government. The damage to the fuelnozzles from the right engine was presumed to havebeen caused by exposure to the salt water, and furtherinspection and analysis were not believed to be neces-sary. The preliminary probable cause of the engine firewas laid to the operator being late on a fuel flow check ofthe nozzles, resulting in carbon build-up and hot-streaking of the fuel flow into the hot section.

There was, and is, no dispute that the probable cause ofthis crash was a fire in the right engine on take-off,generated by un-atomized fuel coming from a highpressure fuel nozzle. The blow-torch effect of this fuel isseen on the guide vanes shown in photos 2 and 3. Thisflame pattern is inconsistent with carbon build-up on thenozzle, as that would burn off rather than block thenozzle outlet, and the nozzle that failed was an over-hauled nozzle recently installed in the engine by anoperator in California before the aircraft was leased to

the Panamanian operator. Further, there was no evidenceof fuel stain on the nozzle faces.

Photo 4 shows the fuel nozzles with the shields andmanifolds from the right engine. Photo 5 shows theinternal components of a couple of the nozzles (note theXA on one of the nozzle bodies).

Figure 1 is a diagram of the fuel nozzle components. Thefuel enters the nozzle assembly from the manifold,passes through a strainer, and is accelerated into awhirling pattern by the distributor so that it atomizes asit escapes through the nozzle tip into a predictablepattern (see Figure 2).

The questions were 1) what caused this hole; and 2)whether this hole existed before the crash. A hole such asthis would cause the fire and loss of the right enginewithin seconds. Therefore, the answer to the first ques-tion would answer the second. Was the hole caused bysalt water corrosion so that it eroded from the outside in,or was it caused by fuel erosion from the inside out?

The nozzle assemblies in the right engine were alloverhauled as shown by the letter codings on the side—ZA, NR, XA. These letter codes indicated batch numbers,and ZA was December 1986. The part number on thenozzle body, 3013635-E, was the part number of theengine manufacturer. Photos 6 & 7 show two fuel nozzlebodies bearing the ZA identifier. Photos 8 & 9 show twofuel nozzle bodies bearing the XA identifier. Photo 10shows a fuel nozzle body bearing the NR identifier. Onlythe nozzle bodies designated ZA showed any degree ofpitting (the orange deposits on the nozzles are from theerosion of other components and do not relate to theissue here).

Photo 1


Photo 2

Photo 3

Photo 4 Photo 5


Figure 1

Figure 2


Photo 6 Photo 7

Photo 8Photo 9

Photo 10


As all nozzles were exposed to the same conditions, thisfinding indicated there was something different aboutthe ZA nozzle bodies. The metallurgical analysis was inaccord with Wrought Stainless Steels by George F. VanderVoort and Hughston M. James (Carpenter TechnologyCorporation, Vol. 9 ASM Metals Handbook, 1985); andMetallographic Technique for Wrought Stainless Steels (Vol.8 ASM Metals Handbook, 1973). There was also some-thing particularly different about the ZA fuel nozzlebody that had the extra hole through its face, as no otherZA nozzle body had anywhere close to the depth ofpitting found on it.

The analysis of the ZA fuel nozzle body was a three-stepprocess. First, an EDAX analysis was done of the ZAnozzle, and of an exemplar nozzle purchased from theengine manufacturer. Then a micro-etchant glyceregiatest was completed on the ZA fuel nozzle body (photo11) and on the exemplar. A Rockwell hardness test wasdone on the ZA fuel nozzle body and on the exemplar.Finally, a borescope exam was done to the inside of theZA fuel nozzle body. The results are consistent withSteels: Heat Treatment and Processing Principles (Krause p.382), “Martensitic Stainless Steels”; and Stainless SteelsHandbook (Allegheny Ludlum p. 33).

The exemplar fuel nozzle body was analyzed by theelectron microscope as being 410 stainless steel and, after

Photo 11

Photo 12


Figure 3

INTE - % - ZAF:LABEL = NOZZLE - E {ZA}13 - MAY - 94 11:58:31

16.891 LIVE SECONDSKV = 20.0 TILT = 30. TKOFF = 22.

ZAF CORRECTION

ELEM K Z A FMOL 0.2144 0.927 0.856 1.002CRK 0.2244 1.024 0.936 1.054MNK 0.1239 1.007 0.955 1.000FEK 0.3582 1.028 0.918 1.000

ELEM CPS WT% ELEMMO L 676.3419 26.97CR K 972.6559 22.21MN K 470.0766 12.87FE K 1209.1743 37.95TOTAL 100.00

Figure 4


being etched with the glyceregia, showed a temperedmartensitic structure indicating that it had been properlyheat treated. This was consistent with the manufacturer’sspecifications that the nozzle body should be made of410 or 416 stainless steel.

The suspect fuel nozzle body, which had the ZA iden-tifier and the extra hole from the fuel source side to theoutlet side, showed that it was 416 stainless steel.

However, after the glyceregia etch was performed, theanalysis of the percentage breakdown of the elements ofthe metal (see figures 3 & 4, 13 May 1994, 11:25:49 and11:58:31) showed that it contained numerous metallicinclusions, and the areas of the metal between theinclusions had the chrome leached away (photo 12).

This formation would be unable to resist being brokendown by the erosive action of the fuel forced against it atup to 800 psi when the engine was in operation. Theextra hole was probably formed by the erosive action ofpressurized fuel linking inclusions and depleted matrixto form a tunnel in the metal. Inadequate heat treatmentat the time of manufacture of the fuel nozzle body mostlikely caused these inclusions.

The mechanism of erosion of the extra hole would nothave been discoverable upon inspection nor upon fieldflow test. Until the hole wore completely through, thenozzle would exhibit nothing out of the ordinary. Oncethe hole was through, engine failure would have oc-curred within seconds. There was nothing the operatorcould have done. The defective heat treatment wouldalso explain why the ZA fuel nozzles were more sus-ceptible to pitting from exposure to salt water thannozzles with the NR and XA identifiers.

This extra hole could not have been caused by exposureto the salt water, as there was no extensive pitting insidethe nozzle body (probably because fuel remains in thefuel nozzles after engine shut-down), and only this singleZA nozzle had a hole through the face. The location ofthe inside of the extra hole coincides with the area ofhighest pressure coming from the distributor. Theexternal corrosion on the face of the nozzle clogged thedesigned hole in the nozzle body faces as well as theextra hole in the face of the ZA nozzle, so the extra holewas not discovered during the Panamanian investigation(Figure 5).

The investigation revealed that the allegedly defectivenozzle had been manufactured in the United States bythe same manufacturer that manufactures fuel nozzleassemblies for the engine manufacturer. This sub-component manufacturer also possessed a Parts Manu-facturing Authority from the Federal Aviation Adminis-tration to manufacture and sell nozzle assemblies underits own part number. According to its vice president ofengineering and new product development, when fuel

nozzle assemblies are sent in for overhaul and a nozzlebody is replaced with a new one, the enginemanufacturer’s part number is affixed, rather than thesub-component manufacturer’s PMA number, albeit thepart does not go through the engine manufacturer’squality program. The only indication that it is not theengine manufacturer’s part is the two-letter code, i.e.,ZA.

Conclusion

The retail price of the overhauled fuel nozzle assemblywas $75.38, while that of the new assembly was $257.40.The total assembly is 3/4" x 3/4" at its maximum dimen-sions. Because each engine has 14 of these fuel nozzleassemblies, there is a significant incentive to cut costs bybuying overhauled nozzles, as did the California opera-tor that installed the ZA nozzles in this engine.

Because the dimensions are small, it is challenging todevelop unique identification means to prevent confus-ing customers about the source of parts. The UnitedStates government has just changed the design of its $100bill to foil state-of-the-art counterfeiters. It is time formanufacturers to meet the challenges of identifying theirproducts more specifically to avoid consumer confusion.

Hopefully, this case study will not only alert investi-gators and operators to the real threat of “bogus” parts,but will also encourage manufacturers to developmodern identification procedures for replacementparts—even for older aircraft—such as bar coding,

Figure 5

hole due to erosion


invisible dyes, permanent dyes, etc., and to continuetheir support of government efforts to eliminate the saleof counterfeit parts. This problem will not disappear anymore than will the knock-off Levi jeans, but we can limitits impact by recognizing that it exists, and by takingsteps to combat it.

airworthiness of, an aircraft or a compo-nent, system, subassembly, or other partof an aircraft knowingly misrepresentedto the Federal Aviation Administration,or concealed or withheld from theFederal Aviation Administration,required information that is material andrelevant to the performance or themaintenance or operation of suchaircraft, or the component, system,subassembly, or other part, that iscausally related to the harm which theclaimant allegedly suffered;

(2) if the person for whose injury or deaththe claim is being made is a passengerfor purposes of receiving treatment for amedical or other emergency;

(3) if the person for whose injury or deaththe claim is being made was not aboardthe aircraft at the time of the accident; or

(4) to an action brought under a writtenwarranty enforceable under law but forthe operation of this Act.

(c) General aviation aircraft defined.—For thepurposes of this Act, the term ‘generalaviation aircraft’ means any aircraft forwhich a type certificate or an airworthinesscertificate has been issued by the Adminis-trator of the Federal Aviation Administra-tion, which, at the time such certificate wasoriginally issued, had a maximum seatingcapacity of fewer than 20 passengers, andwhich was not, at the time of the accident,engaged in scheduled passenger-carryingoperations as defined under regulations ineffect under the Federal Aviation Act of 1958(49 U.S.C.App. 1301 et seq.) [former section1301 et seq. of Appendix to this title, fordisposition of which subsequent to repeal byPub.L. 103-272 § 7(b), July 5, 1994, 108 Stat.1379, see Table at beginning of this title] atthe time of the accident.

(d) Relationship to other laws.—This sectionsupersedes any State law to the extent thatsuch law permits a civil action described insubsection (a) to be brought after the appli-

James F. Leggett had his first encounter with crash inves-tigation while on active duty as a fighter pilot with the UnitedStates Air Force. He completed his MS in Aerospace SafetyEngineering at the University of Southern California and hisLaw Degree at the University of Washington. He is admittedto practice before the US Patent & Trademark Office and theUnited States Supreme Court.

USCA 49 § 40101.General Aviation Revitalization Act of 1994Pub.L. 103-298, Aug. 17, 1994, 108 Stat. 1552, providedthat:

Section 1. Short title.This Act may be cited as the ‘General Aviation Revital-ization Act of 1994’.

Sec. 2. Time limitations on civil actions againstaircraft manufacturers.

(a) In general.—Except as provided in subsection(b), no civil action for damages for death orinjury to persons or damage to propertyarising out of an accident involving a generalaviation aircraft may be brought against themanufacturer of the aircraft or the manufac-turer of any new component, system, sub-assembly, or other part of the aircraft, in itscapacity as a manufacturer if the accidentoccurred—(1) after the applicable limitation period

beginning on—(A) the date of delivery of the aircraft

to its first purchaser or lessee, ifdelivered directly from the manu-facturer; or

(B) the date of first delivery of theaircraft to a person engaged in thebusiness of selling or leasing suchaircraft; or

(2) with respect to any new component,system, subassembly, or other partwhich replaced another component,system, subassembly, or other partoriginally in, or which was added to, theaircraft, and which is alleged to havecaused such death, injury, or damage,after the applicable limitation periodbeginning on the date of completion ofthe replacement or addition.

(b) Exceptions.—Subsection (a) does not apply—(1) if the claimant pleads with specificity the

facts necessary to prove, and proves, thatthe manufacturer with respect to a typecertificate or airworthiness certificate for,or obligations with respect to continuing

Appendix A


cable limitation period for such civil actionestablished by subsection (a).

Sec. 3. Other definitions.For purposes of this Act—(1) the term ‘aircraft’ has the meaning given

such term in section 101(5) of the FederalAviation Act of 1958 (49 U.S.C. 1301(5))[former section 1301(5) of Appendix tothis title];

(2) the term ‘airworthiness certificate’means an airworthiness certificate issuedunder section 603(c) of the FederalAviation Act of 1958 (49 U.S.C. 1423(c))[former section 1423(c) of Appendix tothis title] or under any predecessorFederal statute;

(3) the term ‘limitation period’ means 18years with respect to general aviationaircraft and the components, systems,subassemblies, and other parts of suchaircraft; and

(4) the term ‘type certificate’ means a typecertificate issued under section 603(a) ofthe Federal Aviation Act of 1958 (49U.S.C. 1423(a)) [former section 1423(a) ofAppendix to this title] or under anypredecessor Federal statute.

Sec. 4. Effective date; application of Act.(a) Effective date.—Except as provided in

subsection (b), this Act shall take effecton the date of the enactment of this Act[Aug. 17, 1994].

(b) Application of Act.—This Act shall notapply with respect to civil actionscommenced before the date of theenactment of this Act [Aug. 17, 1994].

Legislative HistoryFor legislative history and purpose of Pub.L. 103-272, see1994 U.S. Code Cong. and Adm. News, p. 818.


Introduction

As the co-chair of the ISASI Flight Recorder WorkingGroup (FRWG) for the past few years, I have been asked,on the Group’s behalf, to present some of the issuesregarding the use of advanced computer graphics foroccurrence investigation. The topic is certainly not newand a number of papers have been given at ISASISeminars over the years as this technology has beenevolving rapidly. I will try to consolidate some of theviews that have been expressed at past ISASI Seminars

as well as give some of the experiences of the Transpor-tation Safety Board of Canada (TSB) with respect to thissubject. I will also review an example animation in somedetail. Many of the issues surrounding computeranimations merit the attention of all accident investiga-tors.

What is driving the technology?

The conventional written accident report is essentially anattempt by the investigating body to communicate the

Accident Analysis with Advanced Graphics

Michael R. Poole, P.Eng., M03278Chief, Recorder Analysis & PerformanceTransportation Safety Board of Canada

Figure 1. TSB computer animation of a mid-air collision between a Metroliner and aNavajo, discussed later in this paper.


results of an investigation, in terms of a presenta-tion of factual information, analysis, conclusions,and recommendations for preventing a recur-rence. As desktop publishing becomes feasiblefor the average person, more and more picturesor ‘graphics’ are being added to the accidentreport in an attempt to more readily communicateinformation—‘a picture is worth a thousandwords.’ Computer animations are the next logicalstep, as they are simply a series of pictures or‘graphics’ that form a moving picture whencombined. To date, the generation of computeranimations has been somewhat specialized due tothe expensive and complicated systems requiredto generate professional looking images, muchlike the generation of individual report ‘graphics’was more specialized years ago but somewhatcommonplace today. As the technology im-proves, more and more people will have theability to readily and easily generate animations.Some animations will be based on recorded flightdata, others will essentially allow investigators toportray the image they have in their mind of theoccurrence or sequence. While the TSB exten-sively utilizes the former, we do not currentlypractice the latter. The media often extrapolates ‘eyewitness’ testimony into graphics images, and morerecently have created animations of accidents, oftenwithin hours of the occurrence. To date, mediaanimations have been somewhat unsophisticated inpresentation and detail, and therefore tend to be ac-cepted as computer generated ‘artist renditions.’ As thetools improve, it will become more difficult to differenti-ate between ‘artist renditions’ and ‘virtual reality,’something that should concern us all a great deal.

While these factors are driving the technology, usingcomputer animations to communicate results is quitedifferent than using animations to analyse the occurrenceto understand what happened and why. At the TSB todate, we have used animations primarily to analyse dataas opposed to communicate the results.

There is no doubt that the compelling and visuallystimulating images that can be generated with sophisti-cated computer graphics is here to stay and has almostbecome the ‘norm’ in major investigations in both theUnited States and Canada, as well as other countries.The media now expects the investigation authorities, atleast in North America, to produce an animation. Unfor-tunately, like so many things in life, for everythingpositive there is the other side of the coin. The use ofsuch technology has some potential pitfalls, especially ifnot used cautiously, objectively, and honestly. Fortu-nately, the benefits of the technology likely outweigh thepitfalls, and the pitfalls can be reduced through educa-tion and exchange between those utilizing the technol-ogy.

The Advantages - Effective means of understanding anddisseminating complex information to a wide range ofaudiences

The main attribute of computer animations rests in theirstimulating and effective way of assimilating a largeamount of data and a complex sequence of eventsquickly, both to the investigating authorities themselvesfor analysis, and subsequently, to a wide range ofaudiences. At the TSB, we tend to generate very detailedanimations for analysis purposes with lots of informationon the computer screen. The final video produced forpublic dissemination typically contains a lot less infor-mation because of the limited resolution of video (sub-stantially less than our computers) and the fact that mostpeople are not interested in as much detail, but onlywant the main points.

A major shortcoming of the accident investigationprocess around the world is the limited communicationor dissemination of the results. Detailed written reportsfrom sources around the world are normally not read bythe majority of line pilots, arguably those who often needto be most aware of the information. Unfortunately, itseems as though it sometimes takes two or three similaroccurrences these days to prevent a recurrence despiteoften thorough investigations by the relevant authority.The NTSB gave a paper at the Ottawa ISASI whichdescribed several examples of similar occurrences thatalso seemed to take more than one to get the messageout. If the aim of accident investigation is to preventrecurrence, internationally we sometimes fail in ourmission (as evidenced by repeat accidents). Computer

Figure 2. DC-10 runway overrun animation. Ground trackdetermined by double integration of acceleratio data.


animations, by their powerful and compellingnature, could certainly prove very effective indisseminating the results of many of our investiga-tions in a stimulating and even entertainingfashion. A familiar recommendation after manyaccidents is immediate and recurrent training toflight crews on the hazard of attempting to performa certain procedure. The use of computeranimations could probably be put to efficient use insupport of this type of training recommendation.

The Pitfalls—Misuse, misleading, giving thewrong messages:

‘Seeing is believing’ and the prettiest picture tendsto garner the most credibility. This is sadly trueregardless of the source of the data, analysis, anddata reduction methods. This paper will highlite afew examples, but not go into all of the limitationsof animating flight data, as they have been dis-cussed in the past in various papers and the pilots’unions have been pointing some of them out forsome time. Suffice to say that there is often signifi-cant operational and engineering analysis required todevelop a good objective animation of a typical aircraftoccurrence.

For example, TSB recently generated a computer anima-tion of a DC-10 runway overrun in which the pilotrejected the take-off within a few seconds of the V1 call,after a loud bang occurred (later determined to be anengine power loss). The ground track took several daysof detailed engineering analysis to develop, and theaircraft never even got airborne (which makes it evenmore difficult to determine actual position in space). Theground track was determined by double integration ofthe accelerometer data. Most people who work exten-sively with accelerometer data are aware that there isusually a bias; e.g., when the aircraft is level and at rest,the accelerometer doesn’t read exactly zero. A small biasof even a few milli-g will make a big difference incalculated distance, since integration errors build withtime. During the efforts to develop an accurate groundtrack, it was determined that the accelerometer had adifferent bias on the acceleration portion of the takeoffroll than the deceleration portion. The difference in biaswas attributed to a hysteresis effect on the accelerometer.After we did some checking, we discovered that theaircraft manufacturer as well as other flight recorder labshad experienced this phenomenon as well. The bias wasdetermined by essentially matching the slope of theinertial speed (single integration of the accelerometer)with the slope of the recorded airspeed. Only after aniterative process of trial and error several times were weable to come up with an accurate ground track thatreasonably matched the physical tire marks. If the datawere given to a ‘computer animator’ as opposed to aninvestigation laboratory, the ground tracks, and conse-

quently the investigation conclusions, could be quitedifferent.

The most common potential problem in developinganimations that I have seen is the tendency to ‘make-up’data that doesn’t exist. A recent computer animation ofthe OJ Simpson murder case showed in great detail howthe murder might have occurred. These types ofanimations without data are being sold as a visualizationof ‘expert testimony.’ Unless the murderer and victimswere sporting the latest in data recorder apparel, theanimation is essentially fiction. The professional lookinganimation, no doubt created with considerable resources,may have a tendency to become fact by many peoplewho view it because; ‘seeing is believing.’

Adding realism to the picture is a natural desire toincrease the professional look, but often requires theaddition of subjective information. In some cases, thiscan do a disservice to the investigation process. Addingsubjective information into the animation such as fog,rain showers, clouds, etc., is easy and tempting to do. Atthe TSB, we try to constantly remind people that whatthey are viewing is a computer animation and not thereal world or an attempt to depict the real world. Howmany liberties should you take in making the animation?Some purists would say none, while others would go tothe other extreme. It is common for the lay person whoviews an animation to immediately link the picture tothe real world and conclude ‘this is what it must havelooked like’ or ‘this is what the pilot saw.’

An example of subjective information that has beendiscussed at past ISASI Seminars is the Delta Flight 191

Figure 3. DC-8 animation with taxi speeds estimated usingFDR time and physical distance of the taxiways. Could bemisleading if interpreted as taxiing too fast around thecorners.

windshear accident at Dallas, which was animated,apparently under contract by the US Justice Departmentfor a lawsuit. The animation depicted storm cells asdiscrete, dark cylinders in an otherwise clear blue sky.As has been pointed out by ALPA, the layman’s interpre-tation of such an animation can do a disservice to pilots.The animation leaves the impression that the pilot flewright into the middle of that big dark blue cylinder, andhe obviously should have flown around it. In reality,picking out a thunderstorm on an overcast stormy day isnot as easy as the animation might lead one to believe.Replicating the second by second atmospheric conditionsthat might have existed is simply not possible withtoday’s data.

Other potential pitfalls include realistic EFIS displays,which are simulated even though the FDR did not recordsufficient information to drive the display and/or theFDR information had not undergone the same process-

ing algorithms as the real EFIS information displayed tothe crew. This is one reason why it is apparently nowIFALPA policy to video-record EFIS displays in additionto flight data.

Sometimes, the person requesting the animation andspecifying its scope is not aware of these issues and isunaware that he is setting the stage for a potentiallymisleading result. It is imperative that all assumptionsand the methods used to develop an animation are wellunderstood by those who intend to draw conclusionsfrom it. In 1991, a DC-8 crashed in Jeddah, Saudi Arabia.The TSB participated in the investigation at the requestof the Saudi Arabian government, including performingthe flight data and cockpit voice recorder playback andanalysis in Ottawa. The investigation determined thattires blew during the beginning of the takeoff roll, whicheventually led to an on-board fire. To complete theground taxi portion of the animation, we estimated taxi

Figure 4. Estimated field of view looking from the Metroliner at the Navajo. 100%on the y-axis corresponds to the collision point.


speeds (IAS did not record below 40 KIAS) basedon the length of the taxiways and the time base onthe FDR. One of the investigators on the team, whosubsequently examined the animation, unaware ofhow the animation was developed, concluded thatthe crew taxied too fast around the corners. Whathe didn’t know was that we could have taxied theaircraft at any speed around the corners. This is agood example of how an animation could lead tofalse conclusions if the persons developing theconclusions are not aware of the limitations of thedata.

A Recent Animation Example

There is often a fine line as to how far to go inmaking an animation more realistic, and there issometimes value in pushing that fine line, particu-larly in a training environment. The following is arecent example of an animation done by the TSB inwhich we were walking close to that ‘fine line,’ butfelt the merits of producing the animation out-weighed the negatives. It is important to note thatthe decision to animate in this case was not to perform theanalysis but to demonstrate the results. Generally speaking,when demonstrating the results, it may be more accept-able to take a few liberties with the animation in terms ofsubjective information.

The animation example involved a Fairchild Metro 23and a Piper Navajo which collided head-on at 4500 feetASL near the Sioux Lookout airport in northern Ontario.The Metro was inbound from Red Lake descendingthrough 4500 feet. The Navajo was outbound in level

Figure 5. View from the Metro one half second prior tocollision.

Figure 6. View from the Metro one tenth of a second prior tocollision. These pictures do not necessarily represent whatthe pilots might have seen, but illustrate the relative size ofthe aircraft in the moments before collision.

flight at 4500 feet. The Metro 23’s flight recordersrevealed that the crew was not aware of the Piper’sexistence and, in fact, never even knew what hit them.There were no recorders on the Piper, but witness markson the wreckage of both aircraft indicated a relative bankangle of 45 to 60 degrees. Since the Metro’s recordersrevealed that aircraft in a wings level descent, it wasconcluded that the Piper must have banked sharply tothe left. There was a solid overcast layer of cloud at 6500feet and the Metro was waiting to clear cloud beforecontacting FSS (on the mandatory frequency) for an

airport advisory. The Piper had just left the zonemoments before the Metro’s call to FSS. Thecollision occurred during the response from FSS tothe Metro.

The occurrence aircraft were operating in VMCconditions and were therefore required to main-tain visual separation using the See and Avoidconcept. The view of the Navajo from theMetroliner would have placed the Navajo againsta relatively complicated backdrop of the surround-ing terrain features and colours. The view of theMetroliner from the Navajo would have placed theMetroliner against less complex sky and cloudcover as a backdrop. The See and Avoid conceptrequires that visual contact be established earlyenough to allow evasive manoeuvring if on acollision course. The ability to visually spot anaircraft is dependant on many factors includingrelative movement, visibility, cockpit ergonomics,and prominence of the object with respect to thebackground complexity, to name a few. Generallyspeaking, relative movement is regarded as one ofthe more significant factors, as it is well knownthat the human eye can more easily detect move-


ment of an object as opposed to picking out a stationaryobject against a backdrop. With a head-on type collision,the object is fixed on the windscreen because there is norelative movement. The only movement excepted wouldbe in terms of growth in size of the converging aircraftwithin the field of view. The percentage field of view ofthe Navajo from the Metro’s perspective is depicted inFigure 4 as a function of time to impact for severalclosure rates. This simplified analysis examined the fieldof view as a function of distance and wingspan of theconverging aircraft (see Figure 4). The curves in Figure 4are largely asymptotic (no growth) until near the time ofimpact, and indicate that as the closure rate increases, thetime at which significant growth occurs prior to impactdecreases. At the estimated closure rate of 430 knots, thegrowth of the Navajo’s width within the Metro’s field ofview occurred approximately four seconds prior toimpact. Prior to this, the aircraft was essentially a ‘dot’on the horizon.

The animation of the collision was produced based onthe Metro’s FDR data and assumptions for the Navajo,which included flight at 4,500 ft ASL on the same airwayat normal cruising speed, and with an evasive manoeu-vre at the maximum roll rate to match the witness marks.The flight animation is not intended to depict the realworld or what the pilots saw or didn’t see. It is simplyintended to demonstrate that the aircraft are very smallwithin the field of view, and that significant growthoccurs only in the final seconds. All the other factorsthat affect someone’s ability to detect an object arebeyond the scope of the animation and this analysis.

The animation was done with both a white aircraft on ablack background and with the actual terrain data fromthe Defence Mapping Agency with simulated colours.We took some liberties in presentation (sky colour andground colour); however, the computer animation isquite revealing in that, even when viewed in black onwhite, it suggests that it is unlikely that either aircrafthad the opportunity to see each other in sufficient timefor successful evasive action, even if all other factors wereoptimal, because of the high rate of closure and the lack ofrelative movement.

Cockpit Voice Recorder Issues

The increased use of animations will no doubt bring newinternational pressures on how voice recordings (CVRs)

are used. Adding the acoustic environment to theanimation compliments the presentation, and a powerfuland compelling result can be achieved suitable for bothanalysis as well as dissemination. At the TSB, weroutinely combine the audio with the animation, butstrictly for internal analysis purposes only. We have notreleased an animation with the CVR audio to date. Inrare cases, when the Board has determined it to be in thebest interests of safety, a partial written transcript hasbeen released and included as text with the animation.We can include ATC audio on animations since commu-nications transmitted over the airwaves can be picked upby anyone, and are typically not of a sensitive or privi-leged nature.

Summary

This paper has highlighted some of the ongoing issuesconcerning the use of advanced computer graphicanimations in accident investigation. Animations aregoing to become more and more sophisticated and moreaccessible to average users as time goes by. With openand ongoing dialogue with respect to each other’sexperiences around the world, they hopefully willbecome a valuable tool for both analysing and dissemi-nating the results of investigations.

Mike Poole graduated as a professional engineer from CarletonUniversity in 1981. After graduation, he was a key member ofa special engineering team that designed and developed a 20foot radio controlled model of a lighter-than-air aircraftprototype. He worked at Pratt & Whitney Aircraft of Canadain experimental engine test. He left Pratt after winning aresearch contract entitled ‘Light Bulb Filament ImpactDynamics’ in aid of aircraft accident investigation. He washired by the TSB/C in 1986, and was tasked with developing aflight recorder playback and analysis facility. The systemdeveloped by TSB is now in use by many of the worlds leadinginvestigation authorities. He is currently the Chief, RecorderAnalysis & Performance for the TSB and acts as the FlightRecorders Group Chairman on major investigations. He alsoholds a current pilot’s licence.


Aircraft accident research and investigation has shownthat many accidents “are no accident.” That is, in thelead-up to many accidents are factors that can predisposea situation toward a human error or system or compo-nent failure, and these can directly contribute to thecause of the accident. Examples abound: training defi-ciencies yield incorrect decisions; poor maintenancepolicies can lead to system failures; documentationerrors promote air traffic control mistakes. (1)(2)(3)

Accident Prevention, versus Accident Investigation

For the purposes of accident or incident investigationthere is a reasonably well developed set of techniquesand practices that involve piecing together what hap-pened and why. This knowledge certainly advances thestate of knowledge of accident causation, and thereby theavoidance of future accidents. There seems, however, tobe an inexhaustible supply of new and different waysthat things can go wrong, in aviation and other modes oftransportation. The accident prevention task involvesidentifying and mitigating these different ways beforethey go wrong, and herein lies the problem: there is nostraight forward procedure or road map for uncoveringall the possible ways that things can go awry.

This is especially true as operational procedures andsystems evolve and change. As economic constraints,technological opportunities, and environmental concernsare mixed together in the same cauldron, there areincreasingly frequent changes to the way our aviationsystem works. It becomes increasingly important to buildin as much accident avoidance as is possible in thedesign and pre-operational planning stages. There arerobust sets of guidelines and good practices for suchendeavors, and these are constantly being informed andimproved by the results of ongoing accident and incidentinvestigation and research. Yet there are too often seriousincidents, and accidents, associated with the introductionand evolution of new systems, technologies, and opera-tional practices.

There is an evident need for realistic means of inspectingpre-operational designs and plans for potential hazards,and for assessing risk potentials associated with thosehazards, before actual operations are begun or changed.When the operational environment has changed, the

problem changes from determining what happened, thetraditional province of the investigator, to assessing whatmight happen. It is insufficient to fall back on one’soperational experience, when the operational environ-ment has changed.

Tools for Identifying Hazards Before They Materialize

The essential difficulty in assessing pre-operationaldesigns and plans, and the reason why even extensiveexperience is not wholly adequate, is that the number ofways that something can go wrong grows combinatori-ally large, as designs or plans become more detailed andrealistic, and hence more complex. There are so manypossible ways for something to go awry that we cannothope to identify all of them. A “tree” of possibilities andsub-possibilities tends to grow without bound. Most ofthe “branches” of such trees represent possibilities thatin most situations are remote and unlikely. However,there are no infallible techniques for “pruning the tree”to eliminate the unlikely occurrences a priori. We mustuse our experience, the results of past accident andincident investigations, and conceptual models of howfailures and errors can occur to focus in on the morelikely and more critical eventualities.

This is not a wholly new idea; the need for better pre-operational assessment methods has been recognized,and many contributions have been made. The organiza-tional failure models of Dr. James Reason of ManchesterUniversity in the UK(4), the concept of a safety net oftechnical, procedural, and human backup contingencies,the quality management paradigm of the ISO 9000 familyof standards, and engineering techniques such as failuremode effects analysis, are examples. (5)(6) The purposeof these models is to focus our attention on situationsand conditions where things are more likely to gowrong, since the level of effort required to inspect theentire aviation/ATC operational process exhaustively isnot feasible.

I won’t attempt to make a comprehensive list of themethods and approaches available, but a few briefexamples may be helpful.

Reason’s approach asserts that the conditions that maylay the groundwork for an operational error or failure

Applying Perrow’s Complexity Model to Aviation Hazard Identification

Robert S. Ratner (M03703)


can be hidden quite far, in functional terms, from theactual work-face. These conditions, termed latent defects,are embedded in organizational structure, corporatestrategy, policy, management practice and style, humanrelations policies, training, and supervision. They are thepre-conditions to unsafe acts, in Reason’s terms. Reasondescribes in some detail the aspects of organizational andmanagement systems where these latent defects canexist, and describes ways of identifying them.

A very different approach is the Quality Managementparadigm of the ISO 9000 family of standards. Whilemany people view ISO 9000-9004 as merely a bureau-cratic set of rules for documenting work processes, theparadigm leads to a useful technique for examiningoperational plans, tasks, and procedures to identifypotential hazards they may contain. (7)

Each addition to the list adds to the aviation safety toolkit. Each adds value; none can cover all the necessaryground by itself.

Professor Perrow’s Contribution

In recent years another important model of systemfailure has been articulated by Dr. Charles Perrow ofYale University in the United States. (8)(9) In his bookNormal Accidents, Dr Perrow develops a model of systemcomplexity that has immediate applicability to theidentification and assessment of aviation hazards. Heuses the term “normal accidents” to denote the untowardresults of complex interactions in a system, but doesn’tobject to my calling his concepts “complexity theory.”His essential insight involves recognizing that all largesystems are subject to failures and errors, which aregenerally defended against by traditional means—training and supervision, engineering standards andpractices, and redundancy. However, he argues, systemcomplexities of several particular sorts can overwhelmthese defenses, and lead quickly to serious incidents andaccidents.

The first sort of complexity is when events are too tightlycoupled in time for human judgment to recognize aproblem and deal with it (“tight coupling”). Pilots recog-nize this phenomenon as “not flying ahead of the air-plane,” and air traffic controllers often call it “goingdown the chute.” Perrow observes that, notwithstandingchecklists and emergency procedures, many accidentsare avoided by timely real-time ad hoc problem-solving,and one wants to ensure that there is enough time forthis to happen. For example, terminal area designs thatseparate arrival and departure airspace are effective inensuring that closing speeds between aircraft are keptsmall, so that there is time to deal with a situation if oneaircraft deviates from its assigned flight path.

The second sort of complexity occurs when some of theinformation an operator needs to deal with a situation is

hidden from him or her (“hidden dynamics”). For ex-ample, it is well known that, in many terminal area airtraffic control situations, simply knowing the position ofan aircraft is insufficient for separation assurance. Speedand heading information are needed. Serious incidentshave occurred when this information has became un-available or inaccurate.

A third sort of complexity occurs when subsystems orparts of an operation are supposed to be separate andindependent, but in fact are not (“common modes”). Sometime ago, overheating in the radio communicationscabinet in a major US airport control tower caused thetower to fill with smoke, and the tower was evacuated.The need for hand-held radios to maintain communica-tions in such situations was foreseen. But the radios werestored in a cabinet adjacent to the other communicationsgear, and were unreachable due to the smoke for asignificant length of time. When the effectiveness of abackup system or procedure is damaged by the sameerror or failure that made it necessary to go to thebackup, we have a common modes problem.

Perrow’s model gives us three particular circumstancesto look for>

TIGHT COUPLING: Situations where there isvery little time between the ability to detect anerror or failure and the untoward consequence ofthe error or failure.

HIDDEN DYNAMICS: Situations whereimportant information about what is going on insome part of the system, or what is going to behappening, is hidden or can become unavailableto a pilot or controller.

COMMON MODES: Situations where an erroror failure can make a planned backup procedureor system unavailable, or when one supposedly-independent action affects another.

Application to SIMOPS Procedures

Let’s see what happens when we apply Perrow’s focus toassessing procedures for simultaneous independentoperations on intersecting runways (SIMOPS). SIMOPSprocedures, in one form or another, and under variousnames, are used in a number of locations throughout theworld to improve throughput at congested airports. Theairport configurations where SIMOPS procedures areused involve runways that cross, but that cross farenough along one of them that operations can beplanned so as not to involve the intersection or runwayportion beyond it. Occasionally SIMOPS proceduresinvolve runways that do not physically intersect, butwhose extended centerlines intersect, with operationalcapability of avoiding the extended intersection. The ideais that under certain circumstances both runways can be


used simultaneously and independently, with one orboth of each runway’s aircraft agreeing to hold short orturn away before the intersection.

Properly planned and correctly executed in compliancewith appropriate control clearances, SIMOPS enablesomewhat increased runway throughput at an airport.However, most pilots and controllers will admit to somedegree of unease or heightened concern while involvedin SIMOPS procedures. While fortunately there havebeen no real catastrophes attributable to SIMOPS proce-dures themselves, there have been a number of ex-tremely serious incidents. (10)

Perrow’s model can help us understand the causes of ourconcerns with SIMOPS, show where error- and fault-tolerant procedure design can reduce potential hazards,and indicate the limits of such hazard mitigation. We’llconsider a particular instance, a serious incident thatoccurred using SIMOPS procedures at SydneyKingsford-Smith airport some years ago.(11) [SIMOPSprocedures are no longer used at Sydney.]

A Thai Airways DC-10 was on final for Sydneyrunway 34 on 12 August 1991. SIMOPS were ineffect, and an Ansett Australia Airbus A320 wason a short final for runway 25. SIMOPS landinginstructions included a requirement for the DC-10 to hold short of the 34/25 intersection. AQantas Airways Boeing 747 was holding on theground near the intersection. A total of 667 soulswere rapidly approaching each other.

Eighteen seconds after the DC-10 landed, theA320 initiated its landing flare. Within fiveseconds after that the controller assessed that theDC-10 was approaching the intersection at anexcessive speed. Believing that it would not stopprior to the intersection he transmitted “Thai 485stop immediately! Stop immediately!” The DC-10 then applied heavy braking. At about thesame time, the captain of the A320 sensed thecollision risk and commenced a go-around froma height of 2 feet AGL. As the A320 passedthrough the intersection at a radio altitude of 52feet, the DC10 had almost stopped, with the noseof the aircraft approximately 40 +/- 20 metersfrom the crossing runway centerline.

What is the Perrow interpretation of these events?

HIDDEN DYNAMICS: The controller knew hehad given a “hold short” clearance to the DC-10and received an acknowledgment. However, thecockpit “dynamics” were hidden from him. Hehad no way of knowing what was going throughthe pilot’s mind, or transpiring in the cockpit. Hehad no information to alert him to possible

hazard until he observed the aircraft’s speed anddeceleration rate well down the runway.

TIGHT COUPLING: From the time when thecontroller sensed the hazard developing, to thepoint when the DC-10 entered the intersection,only a few seconds elapsed. Not enough time tothink out the best way to resolve the situation;just enough time to reactively shout “stop.” Onlybecause the Ansett pilot was concerned enoughabout SIMOPS to watch the DC-10 all the waydown—not a part of the procedure—did he seethe hazard in enough time to abort his landing.This is not separation assurance; it is separationby chance. The event “failure to decelerate” wastoo tightly coupled in time to the event “intersec-tion penetration.”

COMMON MODES: Although not a factor inthis incident, there is a common mode issue, andit still exists at a number of airports whereSIMOPS are used today. The issue concerns theability of flight crew or ATC staff to provide animmediate alert if it appeared that the SIMOPSintersection hold-short requirement would notbe complied with. The primary communicationmeans, radio, was also the backup alertingmeans. In a situation where fractional secondscount, there was no SIMOPS procedure forhaving a light gun ready, nor were there specialrunway signing or lighting devices for alerting.

Another incident further illustrates the hidden dynamicsissue:

A Cathay Pacific Airways Boeing 747-300 wascleared to commence take-off on runway 34 atSydney (Kingsford Smith) Airport on Tuesday 11September 1990 on a scheduled flight to HongKong. (12) Thirty-six seconds later, a QantasAirways Boeing 747-300, under tow, was clearedto cross the same runway. With the undividedattention of all participants, the Cathay aircraftrotated, became airborne, and passed lowoverhead the towed aircraft straddling therunway. While at first look it appeared that theADC had simply neglected to activate the“Runway in use” indicator to alert all towerpersonnel that the Cathay aircraft was departingfrom runway 34 and would require the entirerunway, it emerged that procedures for ensuringunambiguous runway jurisdiction were notconsensually understood.

The ‘’dynamics’’ of who was controlling movement nearthe active runway were hidden from the tower localcontroller.


Using the Perrow model, one first needs to identify theessential safety assurance information. In the SIMOPSexample this consists in knowing whether a hold-shortinstruction is going to be complied with, early enough tointervene if it isn’t. Searching for ways this informationcan be unavailable is greatly aided by Perrow’s way ofthinking. Simulation of procedures, using event sequenc-ing techniques, is often useful. Searching for situationswhere there is too little time to take effective actionrequires experience as well as the Perrow point of view.Effective hazard assessment is often done best with ateam approach, combining people with experience in theoperational area under assessment, and those withanalytical skills.

Having helped us identify hazards, can the Perrowmodel help us mitigate them? Well, yes and no. In manysituations it has turned out that once the hazard wasidentified, fixing it was relatively painless. Sometimessimply better design of procedures was all that wasnecessary. The SIMOPS situation is not so easy; theproximity of large aircraft at high closing rates is inher-ent in the operation.

However, commercial pressures for SIMOPS are difficultto resist. For example, European airport planners arenow investigating increased use of SIMOPS to reduceairport congestion on the continent. What has been doneto reduce SIMOPS hazards is to establish specific train-ing programs and operational procedures that recognizethat cause and effect are very tightly coupled in SIMOPSand attempt to increase vigilance, and to define runwaysigning and lighting requirements for the same purposes.No effective technological solutions have been proven,and SlMOPS today remains a source of what Bob Dodd,of the Australian Civil Aviation Safety Authority, calls“chronic unease.”

References

(1) National Transportation Safety Board, RunwayCollision 1 February 1991, Report AAR-91/08,Washington, 1991.

(2) National Transportation Safety Board, ControlledCollision with Terrain 1 December 1993, ReportAAR-94/05, Washington, 1994.

(3) Review of the Australian Air Traffic Services System.Ratner Associates Inc., April 1992.

(4) J. Reason, Human Error. Cambridge University Press,New York, 1990.

(5) D. Petersen, Techniques of Safetv Management: ASvstems Approach, Aloray Inc., New York, 1989.

(6) R. Ratner and J. Guselli, “Using the Enhanced SafetyNet Method for ATC Quality Improvement,” TheJoumal of Air Traffic Control, Vol. 36, No. 3, July-September 1994.

(7) ISO Quality Standards Collection, Global ProfessionalPublications, Irvine, California, 1991.

(8) C. Perrow, Normal Accidents: Living with High RiskSystems. Basic Books, New York, 1984

(9) C. Perrow, “Accidents in High Risk Systems,”Technology Studies. Vol. 1, No. 1, 1993.

(10)R. Ratner and J. Guselli, “Techniques for IdentifyingIndirect Causal Factors in Aviation Incidents andAccidents,” The ISASI Forum, Vol. 28, No. 2, June1995.

(11)Bureau of Air Safety Investigation, [SIMOPS] Inci-dent B/916/3032 at Sydney Kingsford-Smith Air-port, Canberra, 1991.

(12)Bureau of Air Safety Investigation, [Towed Aircraft]Incident B/902/3307 at SydneyKingsford-SmithAirport, Canberra, 1991.

Dr. Ratner’s aviation consulting practice specializes in hazardidentification and safety assessment. Emphasizing human,managerial, and institutional factors in aviation safety, he hascontributed both to identification and assessment of ATCsystemic hazards and to safety system design, in the UnitedStates and Australia. He led early research efforts that quanti-fied the relationships between ATC attention requirement,traffic load, and system capabilities, and has conductedaviation and ATC hazard assessments leading to improve-ments in procedures, training, supervision, and management.


Summary

Cockpit audio/video recorders (CAVRs) were usedduring certification flight testing of the MD-11 and MD-90, and will be used during certification of the MD-95.Although the purpose of the flight test CAVR installationwas (and will continue to be) the monitoring of avionicsdisplays rather than the provision of crash-protecteddata, some useful lessons were learned that may be ofinterest to those responsible to their organizations for theconduct and quality of accident and incident investiga-tions. Since investigators are familiar with audio record-ers, in the form of cockpit voice recorders, this paper willconcentrate on the “video” portion of the audio/videoinstallation.

Introduction

Prior to flight testing the MD-11 and MD-90 aircraft, theDouglas Aircraft Company determined that the installa-tion of a cockpit audio/video recording system (usingoff-the-shelf components), in combination with an on-board “repeater” station of the main avionics displays(also video-taped with recorders trained on the repeaterinstallation), would provide a cost-effective means tosupport the certification of the respective aircraft’scomplex avionics systems. The primary benefit to the testprogram was expected to be the early identification andresolution of avionics problems by:

• Evaluation of the avionics system performancewithout needing to recreate or duplicate the “glitch”on ground test equipment;

• Independence from pilot or flight test engineermemory of an event;

• Easy observation of avionics displays without crowd-ing around the cockpit doorway and intruding uponthe flight deck crew (permitting avionics engineers the“luxury” of being seated with seat belts fastenedduring takeoffs, landings, and maneuvering flight);and

• Viewing displays that are otherwise clearly accessibleonly from a flight crew position.

Additionally, with the above-described systems in place,it was expected that:

• There would be fewer personnel required on theaircraft to monitor avionics tests;

• There would be reduced time spent investigating non-problems;

• Engineers could more clearly understand any dis-crepancies discovered during a flight and couldtherefore request the minimum parameters and time“slice” needed for digital data review; and

• There would be a decrease in the required amount ofdigital data system (DDS) instrumentation tied intothe flight test aircraft’s avionics systems (and thus areduction in potential error paths as well as a reduc-tion in expense).

As described above, the purpose of the video system wasto save time and money during certification testingrather than providing data for an accident or incidentinvestigation. The system performed well in its intendedpurpose and, fortunately, the latter use never arose.

Systems Description

CamerasIt was evident from quite early in the MD-11 programthat a single cockpit camera could not record all thevarious displays and indicators at the required resolu-tion (1), let alone all flight deck pilot activity. As a result,on our flight test aircraft, a camera (two on the MD-11)was mounted above, behind, and inboard of the pilotcrew positions for overall cockpit views. An additionalcamera was mounted in the cockpit for some flights,focusing on the Flight Guidance Control Panel (FGCP),the Multifunction Control Display Units (MCDUs), or asrequired. However, additional cockpit-mounted cameraswere not always a realistic solution because somedisplays could only be seen clearly from a pilot position.For example, because the Navigation Display (ND) in theMD-90 aircraft is mounted just forward of the controlwheel and column, cockpit video of this display, at therequired level of detail and resolution, was not consideredfeasible.

Flight Test Experience With Cockpit Audio/Video Recorders (CAVRs)

William C. Steelhammer MO2912Douglas Aircraft Company CP0066


Three additional cameras and recorders were installed atthe repeater station in the cabin. The items recorded bythese cameras are described below.

Monitors/RecordersThe installation of a repeater station in the test aircraft’scabin made it possible to obtain a real-time, unobstructedview of the pilots’ displays via remotely mountedindicators that, in turn, were videotaped with additionalcameras and recorders. In our MD-90 test aircraft, therepeater station consisted of:

• A monitor displaying the video from the secondcockpit camera/recorder (usually installed to monitorand record FGCP inputs);

• One Flight Mode Annunciator (FMA) panel;• Two Primary Flight Displays (PFDs) (one for each

independent flight director);• Two NDs, (one for each independent Flight Manage-

ment System (FMS));• Two MCDUs (one for each FMS), with the keyboards

de-activated;• Displays for airspeed, vertical speed, and altitude;• A seat for the test engineer; and• Cameras mounted aft of the test engineer’s seat to

record the repeater displays. One camera recordedPFD1, PFD2, FMA panel, airspeed, altitude, andvertical speed. The second camera recorded ND1 andMCDU1; the third camera recorded ND2 and MCDU2.The time of day, synchronized with the test aircraft’sdata system, was recorded on all videos.

The off-the-shelf cameras, lenses, and recording devicesused were: Toshiba 3 CCD cameras, with Cannon MacroTV Zoom (12x) lenses, and Sony ED BETA recordingunits (ED beta format).

Experience

Certification flight testing for the MD-11 consumedapproximately 2300 total flight hours, of which ap-proximately 1100 hours were dedicated to avionicstesting. For the MD-90, the numbers were approximately1280 hours total, with more than 500 hours dedicated toavionics systems. Between the two programs, there weremore than 1600 flight hours dedicated to avionics testingin which all or portions of the cockpit and repeater videosystems were in use.

Camera PlacementBased on experience from the MD-11 and MD-90 flighttest programs, a generic definition of optimal location(s)for single or even multiple CAVR(s) may be quitedifficult, if for no other reason than the available cockpitspace. For example, on the MD-90 there is only one jumpseat available and there is a bulkhead directly behind thepilots’ seats, limiting the area of the flight deck andconsequently the area to be viewed by the camera(s).Since the choices of possible camera locations in the MD-

90 were fewer than on the MD-11, the MD-90 installationwas the easier of the two. In contrast, the MD-11 has avery large and roomy flight deck with up to threeobservers’ seats available and with the bulkheadsseparating the cabin from the flight deck roughly 10 feetaft of the instrument panel. Therefore, there were morefactors to consider on the installation of the MD-11system and bulkhead mounting was not consideredfeasible. As the accompanying video illustrates, on theMD-90 the general-purpose CAVR was located on anunoccupied area of the bulkhead just aft, above andinboard of the captain’s seat; for the MD-11 video clip,the camera was located in the cockpit overhead, just aftof the overhead switch panel area. The video clipsillustrate the differences in the instrument panel andcockpit areas recorded by the two installations. The MD-90 installation provided a much better overall view butwith less detail; the MD-11 installation provided greaterdetail but could only cover approximately three of the 6cathode ray tube (CRT) displays. As a result of the abovelimitations, we have incorporated a two-camera cross-cockpit system for most video recording of the MD-11cockpit displays (see Figure 1).

Lighting EffectsPerhaps the next most noteworthy difficulty encounteredduring flight test was the effect of ambient light on thequality of the cockpit recordings. Too much ambientlight resulted in nearly a video “whiteout;” too littleresulted in a video “blackout.” Additionally, the color ofthe clothing worn by the flight crews, and the color oftheir skin, had a huge impact on the quality of therecording under certain lighting conditions. Brightclothes and white skin had a tendency to amplify theambient light to the detriment of the recording and alsoinitiated reflections off some of the glass faces of theinstruments. Cameras with automatically controlledirises, and post-exposure video processing technologiessuch as histogram equalization, can minimize theseambient light effects.

Other Difficulties• Pilots or other personnel could often block the

camera’s view of the instrument panel by movingaround in their seats, stretching, reaching across thecockpit, etc., which resulted in events not beingrecorded.

• We found that electromagnetic interference (EMI)could at times adversely affect video recordings,although this was most notable on cameras mountedexternal to the aircraft for control surface monitoringand not so much on cockpit and repeater station videocameras. Relocating the camera usually solved theproblem.

• The installations on the flight test aircraft were crudecompared to a similar system on a production aircraftand, as a result, pilots and test engineers wouldoccasionally bang their heads on the camera(s),resulting in the camera losing focus and/or alignment.


• Vibration during the takeoff or landing roll couldadversely affect the quality of the recording.

• Due to the differing colors and types of displays, focuscould be problematic; for example, focus on the NDthat would provide a clear recording could result inblurred recording of the adjacent MCDU.

Even with the above described “difficulties,” the cockpitand repeater videos were judged by Douglas engineeringand flight test personnel to be at least as important as themore traditional digital data in analyzing avionicssystem performance. That is, even with the various“difficulties,” the system provided the expected informa-tion and more than met our requirements.

Although flight test is usually a non-stressful environ-ment, task saturation could set in and adversely affectcrew recall. Replaying the videos provided a quickverification of what actually occurred during a particulartest.

Similar system(s) will be used in the MD-95 when itbegins its flight test program. Our flight test engineerswill be evaluating state-of-the-art cameras and recordingsystems for improvements over our current system, suchas vibration isolation, automatic light compensation,power zoom, etc.

Lessons Learned Summary

• A single camera could not record all cockpit activityand/or displays.

• Some desired cockpit displays or indicators may notbe readily available for video recording due to cockpitlayout constraints.

• Specific requirements for any cockpit video recordersystem need to be clearly established in order toproperly design and install the appropriate system fora particular aircraft.

• Ambient light made a huge difference in the quality ofour video recordings.

• The camera will occasionally be blocked by flightcrews.

• Beware of EMI.• The installation needs to be non-hazardous to the

crew.• Vibration during takeoff and landing can adversely

affect video quality.• Focus adequate for one display may not be adequate

for the adjacent display.• Video replay was very successful at verifying recalled

pilot actions.

Conclusions

Clearly, the benefits to flight test described in this papercould be directly applied to accident and incidentinvestigations. Additionally, given that there may bequestions about what actually happened in the cockpitthat the CVR and FDR could not reconcile, the ability toclearly see what the flight crew did or did not do, andwhat the pilots were seeing in terms of instrumentdisplays, would be very useful in determining thesequence of events leading to an accident or incident (2).

Assuming that other issues associated with cockpitaudio/video recorders are resolved, such as securityfrom unauthorized or public disclosure (equal to orgreater than the current Federal laws protecting cockpitvoice recordings), pilot-specific issues, favorable cost-

MD-11 MD-90

Figure 1


benefit analysis, etc., how can we best implement such asystem for investigation purposes?

First, a definition of audio/video recording parameterand field of view requirements for accident and incidentinvestigation is a must. Will the CAVRs be installed torecord the instrument panel display indications only, orwill they be positioned to record flight crew gestures,actions, and control inputs in addition to the displays?These issues need to be resolved, and definitions andrequirements established, in order to proceed withCAVR design and installation. EUROCAE is in theprocess of developing the technical specifications andICAO is evaluating international requirements.

Second, if we want to capture as much cockpit displaydata as possible, and if we can not afford to establish“repeater” stations on production aircraft for recordingpurposes, non-video flight recorder(s) may need to beimproved to record the otherwise missing data. Suchdata could include the signals feeding the primarycockpit displays and/or the signals coming from theMCDUs. With these data, replays of the specific cockpitindications can be accomplished with duplicate displayunits, and/or flight crew inputs to the FMS or autoflightsystems can be recreated. These data could be especiallycritical because of the various and sometimes subtleand/or small display indications available in current andfuture aircraft, such as color changes in airspeed tapes orannunciations; small symbols of various shapes andcolors; symbols or annunciations changing shape orcolor, or moving to a different display, as a result of pilotactions; etc. And, with glass cockpits, investigators nolonger have the “luxury” of instrument needle impactmarks on dial faces to determine what an instrument wasindicating to the pilot when the crash occurred.

Third, the step towards implementation of cockpitaudio/video recording should be viewed as the next step(albeit a large one) in the evolution of crash-protectedflight recorders, much as the DFDR was an evolutionarystep up from etched foil devices, and as solid-statedevices were a step up from mylar or metalized mag-netic tape recorders.

We believe that the lessons learned from our experiencewill be useful to the expected “end users” of the data,those individuals who are responsible for the successfuloutcome of an accident or incident investigation. Duringthe ISASI Flight Recorder Working Group conference inSanta Monica earlier this year, these “end users” werenotable in their absence, and the author believes thattheir active participation and input is critical to thesuccessful implementation of cockpit video recordingdevices in future aircraft.

Footnotes

(1) The recorders needed to capture all the various (andsometimes subtle or small) color and mode changes,annunciations, and indications of the cockpitdisplays.

(2) Cockpit video recordings could have been quiteuseful for the following accident investigations(partial list only): ValuJet DC-9 near Miami, 5/11/96; USAF T-43 (B-737) at Dubrovnik, 4/3/96;Birgenair B-757, Dominican Republic, 2/6/96;American B-757 near Cali, Colombia, 12/20/95;USAir B-737 at Pittsburgh, 9/8/94; USAir DC-9 atCharlotte, 7/2/94; Continental MD-80 at LaGuardia, New York, 3/2/94; United B-737 atColorado Springs, 3/3/91; ATI DC-8, Swanton,Ohio, 2/15/92; and the Eastern L-1011 near Miami,12/29/92.

William C. (Bill) Steelhammer is currently employed by theDouglas Aircraft Company as a senior accident investigator,and was the Douglas Party Coordinator for the recent ValuJetaccident near Miami. Bill has been with Douglas since 1984and has been a full-time investigator since 1987. Prior tojoining Douglas, he was a flight instructor, charter pilot, andcorporate pilot in the Pacific Northwest region of the UnitedStates. Bill has a Master of Science in Safety from the Univer-sity of Southern California and a B.A. in Business Adminis-tration from the University of Washington.


Historical Perspective

The first powered flight of an airplane was at KittyHawk, North Carolina, with the Wright Brothers’ origi-nal Wright Flyer on December 17, 1903. Eighty-threeyears later (December 14 through 23, 1986), the world’sfirst nonstop circumnavigation without refueling wasachieved by Jeana Yeager and Dick Rutan in the two-engine composite Voyager. In virtually every sense, bothaircraft were amateur-built experimental airplanes.

While Voyager may be the pinnacle of the amateur-builtmovement, amateur-built experimental aircraft (alsoknown as homebuilts) have had a long-enduring tradi-tion and evolution since the Wright Brothers built theirfirst Wright Flyer. Amateur builders and experimenterswho followed in the steps of the Wright Brothers, andthe demands of the first World War, advanced aircraftdesign and materials technology. After World War I,airplanes built for the US civil marketplace were initiallybuilt and sold without Federal standards or regulation.When, due to the anticipated growth of commercialaviation activity, continued lack of regulation appearedimpractical or counterproductive, the US Congress actedto pass legislation for the regulation of air commerce.The Air Commerce Act of 1926 required certification andtesting of aircraft manufactured for sale to the public,which—when successfully accomplished—culminated inthe issuance of Approved Type Certificates (ATCs). Thefirst aircraft to be licensed under the act was a deHavilland DH-4B, and the first ATC was issued March29, 1927, for the Buhl-Verville J4 Airster, an open-cockpittandem two-seat biplane.

Although the Air Commerce Act of 1926 addressedregulatory standards for aircraft offered for sale to thepublic, the building of unregulated aircraft not involvedin air commerce persisted through the 1920s and 1930s.For example, aircraft neither licensed nor certificated bythe Civil Aeronautics Authority (CAA, antecedent ofFAA) but licensed by the state of Oregon could legally beflown until the outbreak of World War II, when theFederal government placed restrictions on all civilaircraft operations. After the war, George Bogardus, whoowned a pre-World War II amateur-built (Little Gee Bee, a

65-horsepower, low-wing airplane), repeatedly flew theairplane from Beaverton, Oregon to Washington, D.C. (in1947, 1949 and 1951) to petition CAA for a regulatorychange that would again allow individuals to build andoperate homebuilt aircraft. The petition was eventuallyaccepted by CAA and in September 1952 provision wasmade in the regulations for the certification of exper-imental category amateur-built aircraft.

If the Bogardus petition was the impetus for preservingthe amateur-built aircraft tradition, the tenacity andenthusiasm of Paul Poberezny, who founded the Experi-mental Aircraft Association (EAA) in 1953, can clearly becredited with much of its growth. EAA, headquarteredin Oshkosh, Wisconsin, currently has over 151,000 dues-paying members, and is arguably one of civil aviation’smost active and effective interest groups. For example,EAA was instrumental in seeking FAA’s recent approvalof primary aircraft certification (an effort aimed atsimplifying the certification process for smaller generalaviation aircraft) and for auto-gas supplemental typecertificates (STCs), which provide for use of automotivefuels in certificated aircraft powerplants. Additionally,EAA works outside the regulatory arena, having devel-oped a Technical Counselor’s program as a resource foramateur aircraft builders, and having recently created aFlight Advisor’s program to provide a similar resourcefor pilots to help them evaluate their qualifications andproficiency for test flying their amateur-built aircraft.

By the mid-1960s, EAA had identified and advised FAAof the potential for “kit” aircraft, which “could be madeavailable in a semi-fabricated kit form even though notholding a type certificate.” Evidence clearly suggests thatthe potential became a thriving reality. While many ofthe earliest kit and plan offerings relied upon 1930s and1940s materials and methodology, more recent designsprovide aerodynamic refinements and materials technol-ogy generally unavailable in factory-built generalaviation aircraft. Today, the US amateur-built aircraftmarketplace has over 200 firms serving the domestic andoverseas marketplace with plans and kit sales for aircraftas diverse as gyroplanes and two-seat transonic jetairplanes, with many other firms supplying materials,powerplants, propellers, and other components.

Accident Investigation and Amateur-Built Experimental Aircraft

Michael L. StockhillSenior Air Safety Investigator, NTSB


Groups of Amateur-built Aircraft

There are four distinct iterations of amateur-built ex-perimental aircraft:

• The one-of-a-kind aircraft designed and built byan individual or group.

• An aircraft built using plans provided by adesigner. The raw material is obtained indepen-dently by the builder. The designer has cus-tomarily built and flown a prototype.

• An aircraft built using plans and an associatedraw materials package provided by a designer orspecialty supplier. Again, the designer hascustomarily built and flown a prototype.

• A kit-built aircraft assembled from a package ofmaterials and pre-formed and machined compo-nents. Variations, from comparatively basic kitsto so-called “fast-build” kits, are marketed andpriced at correspondingly higher levels.

For each of these iterations, in the United States thebuilder (who is considered the manufacturer) hascomplete latitude to modify or “customize” his creation.For better or worse, even the most elaborate of kit-builtaircraft are subject to builder modifications and changes.As one kit manufacturer observed while being inter-viewed, “Each of our kits becomes a prototype—much toour distress.”

Marketplace for Amateur-built Aircraft

General aviation—that part of the aviation environmentnot encompassed by military, public use, or air carrieroperations—continues in a state of transition in theUnited States that started with a precipitous decline inaircraft production from a peak of 17,000 piston air-planes in 1979 to 499 in 1994. Industry observers suggestthat production declined for numerous reasons: factoriesoverproduced and overpriced their products; theregulatory climate changed and product liability becameonerous; favorable tax laws were changed; discretion-ary incomes declined; operating and maintenance costsbecame excessive; and the products being marketed didnot reflect changes in technology.

During this period, the amateur-built experimentalaircraft market has, by comparison, flourished. Since1982, the FAA registry has increased an average of about840 amateur-built aircraft each year. While many of thegeneral aviation aircraft manufacturers have closed theirdoors or have lingered on both sides of bankruptcy, theDecember 1994 issue of Kitplanes, a consumer magazine,compiled a directory of 509 different designs of aircraft(including gliders, helicopters, airplanes, and gy-roplanes, as well as 30 ultralights) marketed to the public

as plans, kits, or materials packages. The Popular Rotor-craft Association (PRA) has compiled a separate direc-tory with 43 gyroplane designs and seven helicopterdesigns.

Currently in the United States, the greatest obstacles aprospective designer or manufacturer of certificated lightaircraft encounter—besides finding a viable market forits products—are the certification process and productliability constraints. The builder of type-certificatedaircraft must effectively prove a design to the certificat-ing authorities, successfully complete flight and statictesting, and otherwise satisfy regulatory demands. Thisrequires a greater commitment and investment, whichmust be amortized over a relatively low volume produc-tion run, than is required to enter the kit aircraft market-place. The kit manufacturer, on the other hand, avoidscertification expenses and—to some extent—the expenseof product liability coverage, which is reputedly as muchas 20 percent of the purchase price of a type-certificatedaircraft. Some kit aircraft manufacturers state that theycarry little or no product liability coverage, relying ondisclaimers, the notion that the aircraft is experimental,and the conspicuous absence of their company’s name onthe aircraft manufacturer’s data plate (registration plate)to limit their liability exposure.

From a business perspective, limiting the disincentives ofcertification, liability, and high capitalization costsassociated with certificated aircraft eases access to the kitaircraft marketplace and its associated business activity.Simply by making some plans or drawings, building aprototype (not all do), filling the garage with raw materi-als, and buying some advertising, anyone can become akit aircraft manufacturer in the United States. Easy accessto the marketplace does not, however, assure success tothose who are undercapitalized or whose products areill-conceived or don’t catch the market’s fancy.

Regulatory Provisions

FAR Part 21.191(g) provides a mechanism to issueexperimental airworthiness certification for amateur-built experimental aircraft if the aircraft is built solely foreducational and recreational purposes and if the majorportion of fabrication and assembly is undertaken andcompleted by the amateur builder(s). Commerciallyproduced components and parts that are normallypurchased for use in aircraft may be used, including suchitems as engines and engine accessories, propellers, tires,standard aircraft hardware, wheels and brakes, andmain-rotor and tail-rotor blades. FAA provides guidancefor the amateur-builder in the form of an advisorycircular, AC20-27D, Certification and Operation ofAmateur-built Aircraft. A synopsis of the building andcertification process follows.

FAA allows builders to select or create their own design.FAA does not formally approve these designs, and


maintains that it is impractical to develop design stan-dards for the multitude of unique design configurationsgenerated by kit manufacturers and amateur builders.FAA advises prospective builders that construction kitscontaining raw materials and some prefabricated compo-nents may be used in building an amateur-built aircraft.Aircraft assembled entirely from prefinished parts andcomponents are not considered to be eligible, since themajor portion of the aircraft would not have beenfabricated and assembled by the builder. FAA notes thatvarious materials and parts kits are made available tobuilders, and cautions prospects to obtain verificationfrom local FAA offices as to the eligibility for certifica-tion as amateur-built aircraft of such kits. FAA does notcertify aircraft kits or designs or approve kit manufactur-ers. It does perform evaluations of kits that have poten-tial for national sales interest, but only to determine if theaircraft built from such kits will meet the criteria of thebuilder completing the major proportion of the aircraft’smanufacture.

Prior to 1983, FAA conducted “precover inspections,”inspecting amateur-built aircraft at several stages duringconstruction and making a final inspection upon comple-tion, before issuance of a temporary special airworthi-ness certificate (FAA form 8130-7). Additionally, FAAmade another inspection after flight test limitations weresatisfied before issuance of an unlimited duration specialairworthiness certificate. Since 1983, FAA determinedthat a single inspection of amateur-built aircraft (which islimited to ensuring the use of acceptable workmanshipmethods, techniques, practices, and issuing operatinglimitations necessary to protect persons and property notinvolved in amateur-built experimental aircraft activity)can meet safety objectives—if the builder has hadknowledgeable persons such as FAA certificated me-chanics or EAA Technical Counselors perform precoverand other appropriate inspections. FAA has designatedsome private persons as Designated AirworthinessRepresentatives (DARs) to act on its behalf in the inspec-tion and issuance of airworthiness certificates for ama-teur-built aircraft. DARs are authorized to charge fortheir services; those fees are not regulated by FAA.

The builder is expected to document the construction ofthe aircraft with photographs before covering the struc-ture (or structural components), and to maintain abuilder’s log or other construction records. That docu-mentation is provided to the FAA inspector or DARduring the certification process. The amateur buildermust certify that he or she built the major portion of theaircraft to comply with the so-called “51 percent” or“major-portion” rule. In doing so, the amateur builderbecomes the manufacturer of the aircraft for registrationand certification purposes. Additionally, prior to inspec-tion, the aircraft must have been registered with FAA bythe owner, and appropriate identification and registra-tion marks must have been attached to the aircraft.

Once the aircraft, builder’s log, and other documentationhave been satisfactorily inspected by either an FAAinspector or DAR, either a temporary or unlimitedduration special airworthiness certificate may be issued,together with appropriate operating limitations assignedby the FAA inspector or DAR. Additionally, the builderof the aircraft (or a single member of a group that hasbuilt an aircraft) is entitled to an FAA repairman’scertificate, which vests the authority to conduct mainte-nance and condition inspections solely upon the aircraftthat was built.

Normally, amateur-built airplanes or rotorcraft areinitially restricted to operation within an assigned flighttest area for at least 25 hours when a type-certificated(FAA approved) engine and propeller combination isinstalled, or 40 hours when a noncertificated engine (i.e.,modified type-certificated engine, snowmobile engine, orautomobile engine) and propeller combination is in-stalled. The carrying of passengers is not permitted whilethe aircraft is restricted to the flight test area. Pilots mustbe appropriately rated or endorsed for the type ofaircraft under FAR Part 61 rules. This requires them, forexample, to have seaplane ratings before carryingpassengers in seaplanes or high-performance endorse-ments if the aircraft meets the definition of a complex orhigh-performance airplane.

Certain operating limitations are routinely issued withall amateur-built experimental aircraft including thefollowing:

No person may operate the aircraft for carryingpersons or property for compensation or hire.

Passengers must be advised of the experimentalnature of the aircraft.

The aircraft cannot be operated for glider towing orparachute jumping operations unless so equippedand authorized.

The aircraft must have been found to be in a condi-tion for safe operation by a condition inspection(conducted by either the builder, if he/she holds arepairman’s certificate for the aircraft, or by anAircraft and Powerplant mechanic within the 12calendar months preceding flight).

The aircraft may not be operated over denselypopulated areas or in congested airways, except forpurposes of takeoff and landings.

There are no FAA regulatory standards for amateur-builtexperimental aircraft for structural or systems testing,control response and harmony, flight stability, designconformity, flight testing to any specific parameters, orcrashworthiness design. FAA Advisory Circular AC20-27D does, however, recommend that the design of the


cockpit or cabin of the aircraft should avoid (or providefor padding on) sharp corners or edges, protrusions,knobs and similar objects that may cause injury to thepilot or passengers in the event of an accident. It isrecommended that Technical Standard Order (TSO)approved or equivalent seat belts be installed along withapproved shoulder harnesses. AC20-27D also recom-mends that an engine installation ensure that adequatefuel is supplied to the engine in all anticipated flightattitudes. Additionally, a suitable means, consistent withthe size and complexity of the aircraft, should be pro-vided to reduce fire hazard wherever possible, includinga fireproof firewall between the engine compartment andthe cabin. When applicable, a carburetor heat systemshould also be provided to minimize the possibility ofcarburetor icing. The builder is encouraged to drawupon standard industry methods and practices outlinedin other advisory circulars.

Additionally, AC20-27D urges the builder to coordinateany proposed design changes with a qualified aeronauti-cal engineer or with the designer, should the aircraft bebuilt from plans or construction kits.

Differences between Type Certificated Aircraft andNon-Type Certificated Aircraft

There are a number of differences between type certifi-cated aircraft and amateur-built experimental aircraft.Several examples follow:

Unlike production general aviation airplanes, whichare manufactured under the design and manufactur-ing constraints of FAR Parts 21 and 23 or their CivilAir Regulation (CAR) antecedents, few—if any—kitairplane designs meet or are tested to FAR Part 23standards.

No kit manufacturer, by definition, meets the FARPart 21 manufacturing requirements associated witheither a type certificate or production certificate,which require materials quality control and confor-mity inspections. However, some of the larger kitmanufacturers have implemented similar in-housestandards for materials handling.

The certification standards for small airplanes,specifically FAR 23.49, specify a maximum power-offstall speed at gross weight of 61 knots for single-engine aircraft. Several kit airplanes being marketedexceed that stall speed. For example, one pressurizedhigh-performance airplane kit, the Lancair IV, with awing loading of 35.6 pounds per square foot andpower loading of 9.1 pounds per horsepower has a75 knot power-off stall speed.

While many kit manufacturers’ designs have inno-vative applications of contemporary materials suchas carbon fiber, Nomex, and Kevlar, many of those

materials and manufacturing processes are not yetroutinely accepted for primary structural applica-tions by FAA for certificated aircraft, due to concernssuch as useful life span, the effects of weathering, orother long term environmental factors, as well aslack of established maintenance and inspectionprocedures.

Many amateur-built designs have canopies insteadof cabin doors, with potentially limited egress after aturnover. Moreover, there is no requirement foramateur-built aircraft to provide structural pro-tection to protect the occupants during a turnover, asis required for type certificated airplanes.

Many amateur-built designs of two or more seatshave a single exit door. Unlike current certificationrequirements for type certificated airplanes, there isno requirement for amateur-built aircraft to providean emergency exit on the side of the aircraft oppositethe main cabin door.

Although AC20-27D encourages amateur-builtaircraft to be so equipped, there is no currentrequirement for shoulder harnesses to be part ofoccupant restraint systems, unlike newly manufac-tured type-certificated small aircraft, which nowrequire shoulder harnesses for all seats. Perhaps asimportant as shoulder harnesses and seat restraintsis the need for them to have appropriately designedand tested attachment points. (The author is familiarwith one case where the shoulder harness wasattached with some wood screws to a piece ofplywood behind the seat.)

Fuel system designs of amateur-built aircraft do notnecessarily meet FAR 23 or other recognized stan-dards. For example, many amateur-built aircraft fueltank installations have fuselage-mounted fuel tanks,with fuel supplies between the engine firewall andthe instrument panel. Those installations may or maynot conform with current regulations for typecertificated aircraft, which require such fuel tanks tobe isolated from personnel compartments by a fume-proof and fuel-proof enclosure that is vented anddrained to the exterior of the airplane.

Flight characteristics, controllability and maneu-verability, trim, stability, and stall and spin charac-teristics of amateur-built aircraft may or may notmeet FAR 23 or other recognized standards.

Many amateur-built aircraft—particularly those withless than 100 horsepower—are powered by non-certificated engines. (In the December 1994 Kitplanesdirectory, 179 designs of airplanes being marketedwere designed for the use of type certificated aircraftengines. 220 airplanes were designed for use withnon-type certificated engines. Ultralights, gliders,


gyroplanes, and helicopters make up the balance ofthe 509 listings.)

To summarize the distinctions between type certificatedand amateur-built aircraft, it is important to recognizethat the current regulations for design and certificationof production aircraft codify the rationale and experienceof an entire industry over time. Because of the certifica-tion process, the public is assured that an airplanemeeting the minimum standards set out by FAR 23 willbe safe. Currently, a similar systemic assurance is notprovided to the public by those marketing amateur-builtexperimental aircraft.

Crashworthiness of Amateur-built Aircraft

The National Transportation Safety Board (NTSB) hasnot specifically evaluated crashworthiness of amateur-built aircraft. However, NTSB evaluated crashworthinessof general aviation aircraft in a safety report publishedDecember 17, 1980 (The Status of General AviationAircraft Crashworthiness, NTSB-SR-80-2). At that time,NTSB expressed its belief that “when the crash forcestransmitted to occupants through properly designedseats and restraint systems do not exceed the limits ofhuman tolerance to abrupt decelerations, and when thecabin structure remains sufficiently intact to provide alivable space immediately around the occupants, theyshould survive the accident without serious injury.”NTSB made recommendations to FAA concerningshoulder harness installations, standards for cabin“delethalization,” dynamic testing standards, and seatand restraint system crashworthiness standards.

The crashworthiness safety report noted that all crewseats on US general aviation aircraft manufactured afterJuly 18, 1978, were required to be equipped with shoul-der harnesses, and crewmembers were required to wearavailable shoulder harnesses during takeoff and landing.Shoulder harnesses were not required for passengerseats, and there was no regulatory requirement forpassengers to wear them, even if the aircraft was appro-priately equipped. The report stated that improved seatdesigns and standards had been developed by somegeneral aviation aircraft manufacturers, but those seatswere not required by FAA standards. Also noted was thestatement that required design standards for seats andoccupant restraint systems in general aviation aircraftwere far below those for the family automobile.

After the crashworthiness safety report was published,FAA changed the certification requirements for new FARPart 23 designs, and implemented special retroactivecertification standards and requirements for shoulderharness and restraint systems for aircraft of new manu-facture that were to be built after December 12, 1988,under existing type certificates.

The issues addressed by NTSB’s crashworthiness safetyreport appear to be currently covered by FARs. Forexample, FAR 23 addresses emergency landing dynamicconditions, including such requirements as tests withanthropomorphic test dummies of seat and restraintsystem loads, and cabin crushing. FAR 23 spells outother seat, berth, litter, safety belt and shoulder harnessstandards, which were most recently amended Decem-ber 11, 1989. The rule also addresses cabin area occupantprotection.

During the author’s research, no evidence was found of asystemic application by kit manufacturers of the findingsof NTSB’s crashworthiness safety report or the ensuingregulations and design standards of type certificatedaircraft to amateur-built aircraft. So far, no US kit manu-facturers have performed dedicated crashworthinesstests on their kits. Some, however, say that they haveperformed analytical calculations to determine structuralstrength in certain areas of their design, as well asfirewall temperature tolerances. One kit manufacturerdeclared (before going bankrupt) an intent to performswing tests on its fuselage at a NASA facility, where thefuselage is attached to a swing-arm and released toswing in an arc and impact the surface at a controlledacute angle, in order to study design crashworthiness.

Some kits have a “roll bar” configuration over the cabinarea to protect the occupants in case of a roll over. Somedesigns isolate fuel supplies from occupants. Anotherdesign allows for a “crumple zone” forward of theoccupants. Most kits come standard with aircraft seatrestraints that meet FAA TSO standards; of note, one kitmanufacturer lists seat belts as an option.

The author believes a case could be made that thefindings of NTSB’s 1980 crashworthiness safety reportare not unique to certificated aircraft and that crashdynamics and kinematics of non-certificated aircraftdiffer little from those encountered by certificatedaircraft during crashes. In essence, the existence or lackthereof of an FAA type certificate is not an anticipatedvariable when evaluating the energy dissipated by twoaircraft of similar weight and similar speed at impactduring crashes.

Unlike aircraft certificated under FAR 23, FAA imposesno design standards upon amateur-built aircraft, andlimits its guidance to that earlier cited from AC20-27D,recommending cockpit or cabin detail design thatprovides for padding and the avoidance of sharp cornersor edges, protrusions, knobs and similar objects that maycause injury to a pilot or passengers in the event of anaccident. The advisory circular also recommends installa-tion of approved seat belts and shoulder harnesses, butdoes not provide guidance or recommendations for seatdesigns or restraint system attachment designs or testing.


It is beyond the scope of this paper to evaluate amateuraircraft builders’ motivations for wishing to buildaircraft, or for their specific selections of designs. How-ever, it seems reasonable to state that one attraction tomany amateur-built designs and kits is their compara-tively high performance for a given horsepower. (Perfor-mance, as used here, includes, as well as high cruisingspeeds, aerobatic capability, short or soft-field takeoffand landing capability, and other measures of perfor-mance.) Extracting optimum cruise performance fre-quently requires comparatively small fuselage crosssections, with minimal cockpit or cabin volume, andcomparatively high wing loadings (32.7 pounds persquare foot on one homebuilt design), which—all elsebeing equal—result in higher stall speeds, and highertakeoff and landing speeds. The effects of limited cabinvolume and higher stall speeds upon crashworthiness,and the potential for serious injury during forced land-ings and other mishaps, are well documented.

Other amateur-built aircraft designs are optimized fortakeoff and climb performance. Still others are mini-malist—with generous wing areas and light-weightstructures so that performance can be adequate withminimum horsepower powerplants. Low-poweredairplanes typically require low wing loadings of aroundsix to eight pounds per square foot of wing area in orderto provide adequate performance, with correspondinglylow stall and landing speeds. The comparatively lowenergy of these aircraft during forced landings, runwayexcursions, and other mishaps has the potential toreduce the extent of injuries that may be sustainedduring some incidents and accidents.

Currently in the United States, few amateur-built aircraftdesigns have more than two seats, and almost all arecomparatively compact; pilots and passengers are seatedvery close to the floor with legs extended and littlestructure beneath their seats. In recent years, the configu-ration of many low-powered airplanes has evolved into ahigh-wing open framework pusher design, with tandemseating and partial, complete, or no cabin enclosures. Inmany instances on designs such as these, the cabinenclosures are principally fairings and windscreens, withlittle value as primary structure or load-bearing compo-nents, subsequently providing correspondingly modestcrashworthiness protection.

The following are several areas of concern that haveevolved through the author’s observations from accidentinvestigations.

Restraint Systems

On April 19, 1994, a Cirrus VK-30 single-engine, ama-teur-built, experimental airplane collided with terrainwhile on visual approach to the Lake in the Hills Airport,Lake in the Hills, Illinois, after descent from flight level190 (about 19,000 feet above sea level). The pilot and his

son were both fatally injured. The aircraft was exten-sively equipped with cameras, had a builder-modifiedfuel system of increased capacity, and was routinelyused for commercial high-altitude aerial photography.NTSB determined that the probable cause of this accidentwas fuel starvation resulting from modification of thefuel system.

During the on-scene investigation, it was determinedthat the pilot had been wearing a shoulder harness, andhis son had occupied a seating position, located behindthe camera viewing mechanism, that was not providedwith a shoulder harness. The pilot’s shoulder harnessattachment fitting was improperly bonded to the com-posite fuselage and had separated, allowing the pilot tosustain massive head injuries in the crash. Thepassenger’s upper torso was not restrained from hittingthe camera mechanism during the crash.

Also of note, it appeared that the Cirrus was routinelyoperated at weights above the kit manufacturer’s designgross weight, and the airplane was being operated forcommercial purposes, contrary to the intent of theamateur-built regulations. The performance specifica-tions of the Cirrus, as provided to the builder by the kitmanufacturer, listed a maximum gross weight of 3,600pounds and a standard empty weight of 2,500 pounds,with 106 gallons of usable fuel. A post-accident review ofthe available weight and balance documents indicatedthat the maximum gross weight of the aircraft had beenincreased to 4,400 pounds, and the fuel system had beenmodified so that the aircraft had 140 gallons total fuelcapacity. Documentation for weight and balance of theaircraft was ambiguous, with two dated and two un-dated center-of-gravity calculations found. No evidenceof a current weight and balance document that reflectedthe aircraft’s installed equipment at the time of theaccident was found, but investigators weighed thecamera equipment and oxygen bottles and other equip-ment, and calculated the aircraft empty weight to be 3128pounds. Factoring in pilot and passenger weights, alongwith approximately 40 gallons of fuel that were found onboard the aircraft, it was determined that the aircraft wasapproximately 100 pounds over the kit manufacturer’spublished gross weight at the time of the accident.

Operating an aircraft above design gross weight signif-icantly and adversely affects stall speed and takeoff andclimb performance, and reduces the margin of safetybuilt into the aircraft’s structural limit loads. Note,however, that kit aircraft builders in the United States,like the builders of self-designed aircraft, have thelatitude to establish any maximum gross weight figuredesired, rather than using the kit manufacturer’s recom-mendations.


Rollover/turnover and Egress Issues

On April 2, 1995, an amateur-built RV-6A airplane wasdestroyed when it turned over during an attempt torecover from a bounced landing at Renton, Washington.The private pilot, who had built the aircraft, and hispassenger were seriously injured. The RV-6A is a low-wing side-by-side two-seat aircraft of all-metal alumi-num sheet monocoque construction, and is marketed inkit form by a kit manufacturer. The RV-6A has fixedtricycle landing gear and a formed clear plastic canopythat hinges forward of the instrument panel (in thestandard installation) or a sliding plastic canopy (in theoptional installation), and is normally built by amateur-builders with a Lycoming powerplant of 150 to 180horsepower. The airplane involved in the accident hadbeen built with the optional sliding canopy design. Kitssold with the standard canopy design incorporate arollover structure immediately behind the occupants; thesliding canopy design incorporates a rollover structurethat is immediately ahead of the occupants.

During final approach, the aircraft was seen with its nosepitching up and down. As it crossed the airport perim-eter, the nose pitched up, and the aircraft landed hard,bounced, and began to drift away from the runway. Theaircraft impacted the runway a second time; the enginewas heard to rev up and the airplane pitched up, bankedto the left, descended left wing first into the ground,cartwheeled off the edge of the runway, and slid to astop on a parking ramp. Fire department personnelfoamed on and around the aircraft due to leaking fuel.Rescuers were able to lift the tail of the aircraft so thetrapped persons could be extricated for transport to thehospital.

NTSB determined that the probable cause of this accidentwas the pilot’s failure to recover from a bounced landing,and his failure to maintain control of the airplane.Investigators found the aircraft inverted, about 275 feetto the left of the runway. The vertical stabilizer andcanopy structure were crushed, and the aircraft wassupported inverted on its engine and upper enginecowling, the remaining vertical fin and rudder, and theroll bar structure in front of the occupants’ seats. Seatbelts and shoulder harnesses were available and werebeing used by both occupants.

Fuel System Crashworthiness

One example of an inadequate fuel system design wasrelated by a pilot who was extensively burned aftercrashing in an amateur-built airplane. When interviewedby NTSB investigators, he observed that people who siton their gas tanks should wear Nomex protective cloth-ing. Perhaps, this writer believes, a more poignantconcern would be to wonder what design constraintsrequired the designer to locate fuel tanks beneath theairplane seats.

On October 1, 1995, a Kitfox Model 2 was destroyed byfire after it collided with terrain following an inflight lossof control during climb-out from Goheen Airport, nearBattle Ground, Washington. The private pilot was fatallyinjured by smoke inhalation and fire, and his passengersustained serious burn injuries.

Witnesses said they observed the airplane take off to theNorth and climb out steeply. During the climb, theairplane dropped off on its right wing and spun in,impacting the ground at a steep angle. According towitnesses, the engine sounded as if it was operatingproperly and not malfunctioning during the event. Theaircraft crashed into trees and was destroyed by postcrash fire.

The Kitfox model 2 kit was shipped from the factorywith an optional fuel system that incorporated two wingtanks and a cylindrical aluminum header tank designedto be installed between the firewall and the instrumentpanel. The standard fuel system installation incorporateda non-metal fuel tank between the firewall and theinstrument panel. According to factory personnel, thealuminum firewall with which this kit was supplied hassince been updated to a stainless steel firewall. Photo-graphs of the airplane while it was being built showedthe fuel tanks installed in the wings, and the header tankinstalled behind the firewall, with a battery box behindthe passenger’s seat.

When the airplane was inspected by NTSB investigatorsafter the accident, it was found that the firewall nolonger existed—it had melted away during the post-crash fire. Similarly, the header tank and the battery andbattery box were not found and were presumed to havemelted. Evidence suggested that the battery may havebeen relocated behind the instrument panel, in front ofthe passenger—possibly for cg reasons. The cabin areaappeared to be essentially uncompromised by the crashexcept for the fuselage structure above the left mainlanding gear leg; the lower fuselage longeron andassociated tubing were bent upward into the dooropening. The source of ignition and fuel that resulted ininitial propagation of the fire was not determined.

FAR 23.1361 requires type certificated airplanes withelectrical systems to have a master switch arrangementto allow ready disconnection of each electric powersource from power distribution systems. The point ofdisconnect must be adjacent to the sources controlled bythe switch arrangement. Typical installations include asolenoid attached to the battery box with a short lead orcable from the battery. In the instance of this aircraft,comparatively long cables to a grounding point and to abattery bus and starter solenoid were installed withoutprotection immediately near the battery. In an installa-tion such as this, a long cable could be compromisedwith a ground fault during crash damage, providing anelectrical ignition source for a fire that could not be


controlled by the pilot by shutting off a master switch.Factory wiring diagrams for the Kitfox model 2 providedto the builder of this airplane do not show a provisionfor a disconnect near the battery, and narrative descrip-tions of the electrical system do not discuss the possiblesafety hazard of the so-equipped installation.

On June 7, 1991, an Avid Flyer was substantially dam-aged when it made a precautionary landing on anabandoned road at Endeavor, Wisconsin. The privatepilot, who was uninjured, had experienced carburetoricing in the past during a low power ground run, andbelieved that carburetor icing was the reason for the lossof power. During his precautionary landing, the pilotwas unable to stop the aircraft prior to impact with atree. The aircraft was not equipped with a carburetorheat system. The fuel tank of this model Avid Flyer islocated in the cabin, between the instrument panel andthe firewall. The fuel tank ruptured during impact, butthere was no post-crash fire. NTSB determined that theprobable cause of this accident was the pilot-in-command’s inability to effectively eliminate carburetorice to avoid an emergency landing.

Crashworthiness improvements to agricultural aircraft,with their comparatively high exposure to risk of colli-sions with terrain, included placing fuselage-mountedfuel tanks inside bladders that are designed to containfuel after tanks rupture. FAR 23 regulations for typecertificated aircraft require firewalls to be fireproof andbe able to sustain open flame of 2,000 +/-150 degrees Ffor at least 15 minutes. Materials that meet approvalwithout further testing include stainless steel sheet, 0.015inches thick and terne plate, monel metal or mild sheetsteel, 0.018 inches thick. Aluminum alloy does notnormally satisfy the fireproof standard. Additionally,FAR requires that each fuel tank must be isolated frompersonnel compartments by a fume-proof and fuel-proofenclosure that is vented and drained to the exterior of theairplane.

Fuel Starvation Accidents

On April 11, 1991, a Glasair SH2F sustained substantialdamage during a forced landing near Green Bay, Wis-consin. The private pilot, who had 549 hours total timeand 350 hours in the make and model airplane, sustainedminor head injuries when the aircraft turned over. Theaircraft had departed the airport for a flight test of anewly installed engine. The airplane was in a maximumrate of climb and at approximately 3300 MSL when itwent into an “over the top” maneuver, followed by anose down configuration, then another climb maneuver.During the second climb, the engine stopped. Attemptsto restart the engine were unsuccessful. The pilot at-tempted a forced landing on a road, striking a wire onapproach and subsequently impacting a ditch andturning over. Investigation revealed that the fuel pickupline was capable of unporting with low fuel level during

abrupt or negative g maneuvers. Total fuel capacity was34.5 gallons. Approximately 5 gallons of fuel werepresent in each wing, and 6 gallons were in the fuselagetank when the engine power loss occurred.

NTSB determined that the probable cause of this accidentwas the pilot-in-command’s failure to refuel the aircraftprior to departure. A factor contributing to the accidentwas the unporting of the fuel pickup line in the fuel tankduring abrupt or negative g maneuvers. Effective Sep-tember 21, 1984, the kit manufacturer provided informa-tion concerning fuel starvation during slips with low fuellevels; however, guidance specific to unporting undernegative g maneuvers was not provided. The Glasair isconsidered to be an aerobatic aircraft, so it is subject tonegative g operations.

On October 1, 1991, an Avid Flyer, equipped with aRotax 582 engine, sustained substantial damage during aforced landing near Sonora, California after a loss ofpower. The airline transport pilot was uninjured.

The pilot reported that he had refueled the aircraft priorto departure with about 19 gallons of aviation fuel. Hedeparted, climbed to about 10,000 feet, and flew aroundsight-seeing, before turning to return to his takeoff point.He reported that the engine began to surge and the rearcylinder EGT indicated a potential fuel starvationproblem. The engine was not producing sufficient powerto maintain flight and the pilot set up for a forcedlanding in a mountain meadow. He stated that theengine continued to surge even after the throttle wasreduced to idle; this caused him to overshoot his in-tended landing spot. The aircraft collided with trees atthe far end of the meadow. The pilot noted that theaircraft was not equipped with an electric boost pump.He speculated that the engine power problem was due toeither vapor lock or a malfunction of the engine drivenpump.

NTSB determined that the probable cause of this accidentwas fuel starvation due to vapor lock.

Flight Test and Design

Unlike type certificated aircraft, where the prototype isthoroughly flight tested before certification is granted, inthe United States the amateur-built aircraft is not neces-sarily tested to any recognized standard.

During the investigation of a June 18, 1992, takeoffaccident at Woodland, Washington involving a Glasairamateur-built airplane modified with wing tip exten-sions, it was determined that the aircraft had an in-creased wingspan of 4 feet. This increase in wing spanand area reduced takeoff and stall speed so that it waspossible for the aircraft to become airborne and climb atan airspeed (less than Vmca) insufficient for adequaterudder authority. The aircraft crashed after liftoff during


initial climb when the pilot was unable to maintaindirectional control. The pilot was fatally injured and theaircraft was destroyed by ground impact and post-crashfire.

The kit manufacturer reported that approximately 100kits were sold with the original small rudders and wingdesign before the rudder was redesigned and enlarged.Later, a retrofit wing tip extension kit became available,and it was possible to put the wing extension kit on oneof the original 100 kits with the small rudder; for a time,it was also possible to acquire a Glasair kit with theextended wing and with the small rudder. The kitinvolved in this accident was delivered April 5, 1984; thelarge rudder became standard with all kits shipped afterSeptember 1985. Before the accident, the kit manufac-turer had recommended that the larger rudder beinstalled on all Glasairs retrofitted with extended wingtips. The rudder had not been changed on the airplaneinvolved in the accident. The builder (and part owner)stated that he had discussed the larger rudder with thepilot, but both decided that the airplane was operatingwith no problems with the small rudder.

In the United States, modifications such as wing exten-sions or configuration changes on type certificatedaircraft require additional flight testing and recertifica-tion through either a supplemental type certificate (STC)or the original type certification (TC) process. Similarly,homebuilt aircraft require reinspection and flight testingafter modifications, although many modifications aremade to homebuilts where FAA oversight is not solicitedor received. In the case of type certificated aircraft, FAR23.143 requires an airplane to be safely controllable andmaneuverable during takeoff, climb, level flight, descent,and landing, clearly mandating that adequate rudderauthority be available during takeoff and climb.

It is the author’s opinion that it is inappropriate for a kitmanufacturer to market a kit airplane that does not havesufficient control authority available in all regimes of theflight envelope. That said, it is possible to successfullyoperate amateur-built aircraft that have some character-istics that are outside of the parameters defined by FAR23.

Pilot Flight Training and Flight Test Preparation

Clearly there are examples of aircraft whose charac-teristics fall outside the normal performance envelope oftype certificated aircraft. One example is the Glasair justdiscussed that crashed during takeoff. Another exampleis the Swearingen SX-300, an all-metal two-seat 300horsepower airplane with a 239 knot cruising speed, 70.7square foot wing area (31.1 pounds per square foot), and72 knot stalling speed.

On March 11, 1992, a Swearingen SX-300 crashed into acanal following a loss of control while maneuvering

three miles southeast of the Okeechobee, Florida airport.Visual meteorological conditions prevailed. The airplanewas destroyed and the private pilot and his pilot-ratedpassenger were fatally injured. Witnesses reported thatthe airplane flew over the airport at 1500 feet thendeparted the traffic pattern to the Southeast. The SX-300was observed performing a 360 degree turn to the right.One of the witnesses observed that the bank angle wasapproximately 90 degrees. The aircraft was then seen tospin counterclockwise, descending nose and left winglow, until impact with the water. The engine was re-ported to be running with no failure or malfunctionheard. Examination of the cockpit indicated that the flapand landing gear controls were in the up position.Performance charts provided by the kit manufacturernoted that with the flaps and gear up, stall speed at abank angle of 60 degrees is approximately 163 knots (andapproximately 92 knots at 30 degrees bank, and 82 knotsat zero degrees bank). Stall speed with gear and flapsdown and 60 degrees bank is about 145 knots, 80 knots at30 degrees bank, and 72 knots at zero degrees bank.

NTSB determined that the probable cause of this accidentwas failure of the pilot-in-command to maintain air-speed, resulting in inadvertent stall/mush.

On October 14, 1992, another Swearingen SX-300 collidedwith terrain while on landing approach to runway 25(3600 feet runway length) at Rosamond Skypark, Califor-nia. Visual meteorological conditions prevailed at thetime of the accident. The two pilots aboard, father andson, were fatally injured and the aircraft was destroyed.Witnesses stated that when the airplane was about a mileout on final approach it suddenly and violently rolled tothe right and descended into the ground. The airportmanager, who was also the fixed base operator, notedthat winds that day were out of the South at about 25mph, with gusts to 30 mph, and he had grounded hisrental fleet. He also stated that the SX-300 had made twoor three close-in left hand approaches to the airport. Hesaid that each time the airplane would overfly therunway centerline and then correct with steep bankedturns back to the runway. The airport manager said thatthe event appeared to be an accelerated stall. The rightseat occupant was the owner and builder of the airplane.He held a commercial certificate and had 3100 hourstotal time recorded on his most recent airman’s medicalcertificate application. The left seat occupant had re-ceived his private pilot’s certificate on March 13, 1992.According to his log book, he had accrued about 128hours total time, of which 46 were in the SX-300, with 34as pilot-in-command. It was not determined which pilothad been at the controls at the time of the accident.

NTSB determined that the probable cause of this accidentwas the pilot’s inadequate compensation for the windconditions and his failure to maintain an adequateairspeed while maneuvering, which led to an inadvertentstall/spin.


Initial Flight Tests—Training Issues

EAA has recently developed its Flight Advisor programto assist builders and pilots of amateur-built aircraft.FAA and EAA have recognized that the accident rate inamateur-built aircraft is higher during the flight testperiod. In many instances, because type specific trainingis not available, the builders make the initial test flightswithout benefit of adequate preparation. An example ofan accident involving a comparatively low-time pilotwith no experience in the type of aircraft he had builtfollows.

On May 28, 1994, an amateur-built KR-2 airplane crashedafter takeoff at Puyallup, Washington. The private pilot,who was the builder and sole occupant of the two-seataircraft, was uninjured. The aircraft sustained substantialdamage. The pilot’s intention was to conduct slow andhigh-speed taxi-testing of the aircraft in visual meteoro-logical conditions in anticipation of an initial flight test.He had planned to add power and take off if he encoun-tered difficulties during the test runs. During a highspeed run, the pilot encountered oscillations about thepitch axis, followed by the aircraft veering toward theleft edge of the runway. The pilot added power and tookoff. After takeoff, he realized that he was just above stallspeed at 75 percent power. He reported pushing in fullthrottle; the noncertificated engine (a derivative of aVolkswagen automotive engine) then sputtered to a lowidle. He stated that the aircraft mushed, slowed, thenbroke in a stall quickly, rolling to the left before crashing.Later testing by the builder determined that the electricboost pump was not functioning properly; while itprovided adequate fuel flow at reduced power settings,insufficient fuel flow was available at high powersettings.

NTSB determined that the direct cause of this accidentwas an improperly functioning fuel boost pump. NTSBnoted, however, that the pilot had no previous flight testexperience, nor any flight time in a KR-2.

The KR-2 is a small aircraft, 14.6 feet long, with a 20.8foot wing span and 80 square feet wing area, 900 poundsgross weight, and an advertised cruising speed of 180mph using a 2100 cc Volkswagen-derived engine. Forcomparison purposes, one of the smaller common two-seat type certificated airplanes is the Cessna 150, which is23.75 feet long, with 32.75 foot wing span, 160 square feetwing area, 1600 pounds gross weight, and an advertisedcruise speed of 117 mph. Small airplanes such as the KR-2 may have pitch and control sensitivity that are appre-ciably different from that of the type certificated aircraftwith which this pilot was familiar and in which he hadtrained. The pilot encountered pitch oscillations duringhis high-speed taxi tests of sufficient amplitude to makehim elect to attempt a takeoff; the availability of typespecific flight training, with emphasis on pitch and

control sensitivity, may have prepared him to conducthis taxi tests without encountering control difficulties.

Because of the comparative lack of availability of typespecific training in amateur-built aircraft designs, pilotstransitioning into the aircraft, and even flight instructorswho are expected to train others to fly an aircraft, oftenrely on “check-outs” from owner/builders, many ofwhom are not trained and qualified as flight instructors.

One factor limiting type specific flight training in theUnited States is the regulatory constraint that limits useof amateur-built aircraft for compensation or hire.Because FARs prohibit commercial use of amateur-builtexperimental aircraft, an owner can pay an instructor toreceive instruction in the owner’s aircraft but cannot payfor the use of another experimental aircraft to receiveinstruction. As a result, a Certified Flight Instructor (CFI)who owns an experimental aircraft cannot give instruc-tion in that aircraft and charge for the use of that aircraft.Under current regulations, the instructor can charge forhis/her time but not for the use of the aircraft. Therefore,builders of kit aircraft are limited by regulations infinding instruction in a similar aircraft before the criticalfirst flight.

Contractual Builders

One current concern in the United States is the frequencywith which amateur-built aircraft are being built forowners, providing an apparent subterfuge to FAA rulesrequiring that such aircraft are to be built for educationaland recreational purposes only.

The BD-10, a two-seat single-engine turbojet airplane,was designed by James Bede and marketed by Bede JetCorporation as a transonic (Mach .9 cruise) amateur-builtkit airplane. In 1988, Peregrine Flight International,known then as Fox Aircraft, acquired the rights fromBede Jet Corporation to assist Bede Jet customers inassembling BD-10 kits as a builder’s assistance center.Peregrine later acquired marketing rights for the certifi-cated version of the aircraft, and opted to pursue FAAtype certification. On December 30, 1994, the prototypePJ-1, assembled from one of 11 BD-10 kits being as-sembled at Peregrine’s facilities, crashed while undergo-ing flight testing near Minden, Nevada. At the time ofthe accident, Peregrine reportedly had 61 employees andwas assisting customers in the assembly of five of thekitplanes. An estimated time to build a BD-10 is 13,000 to14,000 hours—about 6 or 7 years of full-time work forone person. With build-times that extensive, and withbuilder assistance being provided by professionals,practicality alone suggests a potential for abuse of therequirement that the amateur-builder personally com-plete the major portion of the fabrication and assembly ofan amateur-built aircraft.


However, defining “major portion” has become increas-ingly subject to varied interpretation. “Fast-build” kits,in which major portions of components are prefabricatedby kit manufacturers, have become acceptable. Theapparent rationale suggests that the greater proportionof hours required to complete the project still remainswith the amateur-builder, even when preformed majorcomponents are provided. Safety is probably enhancedby having the kit manufacturer provide factory-builtmajor or critical assemblies to builders who do notpossess the expertise or equipment to complete thoseoperations but do have the skills to complete the manylabor-intensive assembly and finishing operationsinvolved in building any aircraft. There have beenoccasions when those who market kits have attempted toeliminate repeated operations by fabricating, for ex-ample, all but one of the wing ribs, on the premise thatthe educational process is served by building one rib; torequire the kit purchaser to build more than one isconsidered redundant.

As the kits being offered become more elaborate andcomprehensive, more closely approaching completedaircraft, they are subject to being increasingly perceivedand marketed as substitutes for type-certificated produc-tion airplanes. The appeal of homebuilding, as themarket base is broadened, has expanded beyond theoriginal base of highly informed and critical enthusiaststo attract those less intimately involved with the hobbyof building aircraft for educational or recreationalpurposes.

Ultimately, for some individuals, the building processmay become incidental to the acquisition of an aircraft.This manifests itself in contractual builders, who willbuild a homebuilt for willing parties for a negotiated fee(either by registering the contractual builder who then“sells” the aircraft to the purchaser, or by registering theairplane with the purchaser named as the builder),effectively negating the intent of the regulations. A safetyissue arises when the “owner-builder” of an aircraft builtby a contractual builder is issued a repairman’s certifi-cate that allows him/her to perform condition inspec-tions and other maintenance on an aircraft that he/shehas hardly seen during the construction process. Al-though contractual builders have the potential to buildbetter, safer aircraft than their customers, this process isnot within the intent of current regulations.

FAA has recently addressed concerns about contractualbuilders by publishing AC20-139, Commercial Assis-tance During Construction of Amateur Built Aircraft.This AC clarifies current regulations and defines suchterms as major portion, builder centers, compensation,commercial assistance, and acceptable assistance. It alsodefines what is not acceptable, and the current penaltiesthat exist for noncompliance.

Dissemination of Safety-Related Information

Because of the nature of experimental amateur-builtaircraft certification, regulation, and registration, theindustry is not subject to the formal mandatory system ofFAA airworthiness directives. Regulatory requirementsnotwithstanding, a number of kit manufacturers havedevised their own systems for notifying their customersof product changes and safety issues. Many, followingthe example of type-certificated aircraft manufacturers,issue service bulletins and service letters, evaluatingthem according to importance.

While the FAA must be notified of ownership changes,there is no requirement for a new owner to notify kitmanufacturers of change of ownership. Absent othermeans of tracking secondary market owners (thoseowners who purchase a previously built, partiallyfinished, or unfinished experimental aircraft), all the kitmanufacturers surveyed rely upon new owners to notifythe company of ownership changes. Discussions withseveral kit manufacturers suggest that their lists accu-rately identify about 90 percent of the current owners oftheir kit aircraft.

Even though the majority of homebuilts are not originaldesigns but are built with some degree of conformity toplans, kits, or designs, the builders frequently do notrefer to the original design’s nomenclature when choos-ing model names or numbers when registering theirhomebuilts. Additionally, due to the high turnover of kitmanufacturers in the marketplace, the potential use of akit manufacturer’s mechanism for contacting currentowners of a specific amateur-built design does not existfor entities that are no longer in business. The currentFAA registration process, with the builder being listed asthe manufacturer and with no systematic reference to theoriginal design nomenclature, provides no effectivealternative mechanism for contacting owners of anygiven series or design of amateur-built aircraft. Kitmanufacturers who were interviewed during thisinvestigation noted that their ownership records areabout 90% accurate, but they have difficulties in main-taining current records when aircraft change hands inthe secondary marketplace.

Besides kit manufacturers and those who market plans,other suppliers can have difficulty disseminating safetyinformation. For example, the make, model, and series ofpowerplants used at the time of manufacture are amatter of record for type certificated aircraft. Should anengine manufacturer wish to provide owners of typecertificated aircraft with safety information, or shouldthe FAA need to contact such owners to distributeairworthiness directives concerning an engine, it ispractical to do so using the FAA aircraft registry database. Because there is no record of the make, model orseries of powerplant installed in amateur-built aircraft,the FAA registry is not usable for this purpose.


On December 22, 1992, a Velocity HXB sustained sub-stantial damage during a forced landing after a loss ofpower. The private pilot was flying on top of cloudsunder VFR on top rules when the engine quit duringcruise flight. After descending below the clouds, the pilotobserved swampy terrain in the Savannah River. Thepilot elected to conduct a forced landing in the river.Subsequent examination of the engine revealed that theidler gear had disengaged from the crankshaft gear. Thehead of the bolt that secured the idler gear was found inthe oil sump. Its associated washer was not found.Lycoming, the manufacturer of the engine installed inthe Velocity, had issued a mandatory service bulletin,and the FAA had issued an Airworthiness Directiverelated to the crankshaft gear train of this model engine.The AD and service bulletin called out specific actionconcerning the crankshaft gear to be taken if the enginewas ever subjected to a propeller ground strike. WhileNTSB did not assert that this gear train problem oc-curred due to a previous propeller strike, it is worthnoting that the Velocity’s owner would not normallyreceive a copy of an Airworthiness Directive related tothe engine installed in his aircraft, nor would he nor-mally receive engine manufacturer’s safety informationthat would otherwise be disseminated by the enginemanufacturer’s customers.

The above accidents raise concerns about the ability ofmanufacturers to effectively disseminate safety andflight information to owners of amateur-built experi-mental aircraft. Although FAA must be notified ofownership changes, there is no requirement for a newowner to notify kit manufacturers of change of owner-ship. Consequently, despite efforts by manufacturers todevise systems for notifying their customers of productchanges and safety issues, the kit manufacturers mustrely on new owners to provide notification of ownershipchanges. The available information indicates that theownership trail for notification purposes is imperfect,even for kit manufacturers who remain active. In thoseinstances where the designer/kit manufacturer is eitherinactive or no longer in existence, dissemination ofservice information and safety-of-flight information isinconsistent at best, if not impossible.

Interest Group Role

One kit manufacturer interviewed during a site visitnoted that “the homebuilt airplane community is a close-knit community: we tend to police ourselves.” Even ifthat statement is taken at face value, the long-termviability of self-policing may be compromised as thehomebuilt market expands, bringing in participants fromoutside the initial community. It also appears true that aclose-knit community may not be sufficiently introspec-tive to note when its safety record is deficient.

An example of successful industry self-policing can befound on the periphery of aeronautical activity. The

efforts of the United States Hang Gliding Association(USHGA), which has created its own licensing system forhang-glider pilots (Beginner, Novice, Intermediate,Advanced, and Master Ratings) and which providestechnical oversight for hang-glider designs, appear tohave played an important role in decreasing the fatalityrecord from a high of 40 fatalities in 1974 to 8 in 1993.The author suspects that strong peer pressure, awardsprograms, competitive events, achievement awards, andpilot proficiency programs have contributed to the hang-gliding avocation’s much improved safety record.

Similarly, the Experimental Aircraft Association con-tinues to play a strong proactive role in providingpositive reinforcement by giving awards for craftsman-ship, publishing a multitude of periodicals and books,designating Technical Counselors—who provide tech-nical guidance at no expense to builders—and by sup-porting a strong chapter organization that offers peerguidance and pressure to members and builders.

EAA has consistently played a proactive role in aviationsafety. Having defined an appreciably higher accidentrate during the initial flight test period on homebuilts,EAA implemented a flight counseling program with over300 volunteer Flight Advisors functioning in a contextsimilar to EAA Technical Counselors. (Flight Advisorsare not intended to serve as flight instructors or testpilots, but they do provide guidance for evaluation ofpilots and builders, helping them to determine if theircurrency and proficiency are adequate to fly theirairplanes, and providing recommendations for necessarytraining.) As an incentive, Avemco Insurance Companyand EAA make flight testing insurance available to thosewho avail themselves of the Flight Advisor program.

The Popular Rotorcraft Association (PRA) was foundedin 1962 as a voluntary, non-profit organization. PRA hasover 4,000 members, with 76 chapters worldwide.According to PRA, about 90% of homebuilt rotorcraft aregyroplanes (as opposed to helicopters), and about one-third to one-half of those are operated as ultralightvehicles.

PRA believes that the majority of homebuilt rotorcraftaccidents could be prevented through increased andenhanced pilot training and that many accidents are dueto lack of understanding of rotary-wing aerodynamicsand flight principles, as well as ignorance of the properprocedures to be used in the event of a power loss.

PRA expresses concern for the high fatality rate inhomebuilt rotorcraft accidents, but believes one possibleexplanation is that many non-fatal accidents don’t getreported. Also, it believes that the majority of accidentsoccur with unregistered gyroplanes, which are fre-quently operated as illegal ultralight vehicles.


PRA was active in encouraging FAA to publish anexemption that permits dual instruction to be given forhire or compensation using non-certificated gyroplanes.Due to the virtual lack of certificated gyroplanes, it hadbeen effectively impossible for pilots to obtain dualgyroplane instruction except as a gratuity.

The Small Aircraft Manufacturers Association (SAMA) isa national trade association of over 40 suppliers of kits,engines, propellers, avionics, and equipment. In 1993,SAMA members produced over 2,000 aircraft kits.SAMA’s original objectives were to focus on revitalizinggeneral aviation by developing, certificating, and pro-ducing a new generation of affordable, safe, and efficientgeneral aviation aircraft. SAMA was instrumental in thesuccessful effort to create a separate certification stan-dard for primary aircraft. SAMA is a strong advocate ofenhancing safety by education and increasing public andcustomer awareness rather than regulation, and cites theavailability of newsletters, magazines, forums, andmanufacturers’ house journals as effective means ofdissemination.

Conclusions

“We see no ‘reckless’ building of homebuilt air-planes.”— President, prominent kit manufacturer.

Most knowledgeable sources feel that the preponderanceof amateur-built experimental aircraft are of acceptableor better craftsmanship, and that most builders whosuccessfully complete an aircraft and prepare it for testflight do so in a conscientious manner. Like the kitmanufacturer who stated that “we see no ‘reckless’building of homebuilts,” the author sees little evidence of“reckless” building. If anything, most homebuildersseem hungry for knowledge and are very conscientiousabout their avocation.

The author does see, however, a safety record potentiallycompromised, both by the nature of the activity and by

controllable factors. Observations suggest that safety canbe enhanced without significantly affecting the freedomsenjoyed by homebuilders. Ultimately, an improvedsafety record would only enhance the movement’s abilityto flourish.

Homebuilt aircraft clearly differ from type certificatedaircraft. Many are smaller, with small cockpits or cabins,heightened responsiveness, potentially less stability,higher wing loadings, fewer doors or more difficultmeans of egress, and potentially greater performance.Others are of low power, low wing loadings, lightstructures, and low gross weight. Many of these charac-teristics define pilots’ attraction to amateur-built aircraftdesigns.

The pilots having accidents in homebuilt aircraft appearto be a bit more experienced than their counterpartsflying TC’ed aircraft of similar weight, power, andseating capacity. Those involved in homebuilt aircraftaccidents are older than the pilots involved in accidentsin type-certificated aircraft, enough so that if allhomebuilt pilots over the age of 60 stopped havingaccidents, the homebuilt airplane accident rate mightmore closely match that of similar category type-certifi-cated aircraft.

The author feels that with the growth of the kit market,the homebuilt aircraft market now encompasses a pilotpopulation that is not confined to first-time builders andEAA/PRA members. The experimental aircraft nowbeing marketed as kits are often viewed and sold asalternatives to TC’ed aircraft, particularly in the second-ary marketplace.

The challenge of interest groups and regulatory agenciesis to educate and inform the users and marketplace as tothe very real differences between homebuilts and typecertificated aircraft, and to improve a safety record thatis a matter of concern—all without constraining theenthusiasm and freedom that are implicit in homebuiltactivity.


For a number of years the aviation accident investigationcommunity has been wrestling with the concept ofevaluating what role airline management plays in thecause of an accident. The International Civil AviationOrganization (ICAO) in its 1970 (1) manual on accidentinvestigation did not address the issue of managementdirectly, but referred to such issues as how pilot manualswere written as items of interest to the investigator. In a1984 ICAO Accident Prevention Manual (2), a smallchapter was devoted to Management and anotherchapter on Risk Management. More recently, ICAO, in aHuman Factors Digest (3), specifically addressed uppermanagement, who set goals and manage resources as akey element affecting safety and profits. Line managerswere also identified as key elements, as they implementthe decisions of upper management. James Reason’sbook on Human Error (4) formed the theoretical basis ofICAO’s development of the role management played insetting up defenses to guard against hazards. In a laterHuman Factors Digest (5), ICAO has provided a moredetailed look at the influence of management in safe andunsafe organizations and what management’s contribu-tion to safety should be.

The National Transportation Safety Board (NTSB) uses aCorporate Culture Checklist (6) to evaluate the effect ofsafety management in accidents. A number of questionsare suggested that include such items as recent corporatechanges, involvement of senior management in safety,attitudes of line personnel, and possible recriminationfollowing report of a hazardous condition.

C.O. Miller, a former Director of the Bureau of AviationSafety at NTSB (1968-1974), has written a number ofarticles (7-9) on the subject of safety management ofaviation systems. He has also advocated multiple causefinding to replace the confining single cause findingwhich bound NTSB for many years. Mr. Miller shares hisviews from the point of view of a manager and investiga-tor and suggests a special “management group” withinthe investigation process to aid in ferreting out themanagement factors.

In an excellent text on safety management, Wood (10)relates how to set up a safety program and monitor itsprogress. It will serve all aircraft investigators well tostudy this text from a safety professional.

James Reason (11), a psychologist who has assisted in anumber of serious accidents such as the Herald of FreeEnterprise ferry boat capsize in Zubrugge, Belgium, hasbegun to have an impact on investigation with histheories of the latency of factors that cause accidents.Management plays a unique and important role inpreventing accidents according to Dr. Reason.

In a paper for ISASI (12), Alan Diehl, a psychologist,former accident investigator for NTSB in the UnitedStates, and consultant to the US Air Force Safety Agency,described the impact ergonomics can play in utilizationof equipment, i.e., common controls. Selection, mainte-nance, and employee relations were also subjects Dr.Diehl suggested were critical in managing an aviationoperation.

In one of the most recent papers on management’s role insafety, Jan Meyer (13) combined human factors andorganizational analysis into a common model. A series ofquestions are suggested to evaluate the organizationalculture of the company. This paper is important becauseit shows the differences in national cultures as well asorganizational cultures that an investigator can encoun-ter.

The Challenge of Evaluating Management’s Effect onSafety in Flight Operations

Investigators who have been associated with the airlineindustry have seen a number of examples of poormanagement’s effect on the safety of that airline. Mostoften it is hard to prove because of the distance thatexists between the cockpit, where operations are directlyconducted, and the Board room, where decisions affectthe method of operations, or the climate of operations.Selection, training, equipment, culture, and financialhealth all interact to sculpt the environment for allemployees. In James Champy’s book on management(14), he states the greatest tool of management is lead-ership. Some examples can be found in the airlineindustry, but it is more common to see cutting em-ployees and services during periods of poor financialperformance. “Downsizing” was common in the airlineindustry before that term became popular. Sometimes itwas called seasonal hiring and firing, and sometimes itwas called just plain furloughing. In all cases it had a

Management of Aviation Operations — Was the Board Room Active inPreventing the Accident?

Richard B. Stone WO0837Scott T. Young, Ph.D.,

Salt Lake City, Utah, USA.


dire effect on companies such as Pam Am and TWA. Theemployees were disheartened, mistrusted the motives ofmanagers, and reacted in malevolent ways.

How a company handles adversity may clearly indicatethe imbedded style of management or culture of thecompany. Meyer (13) describes how an effective com-pany handles conflict as opposed to how that samematter is handled by a mediocre company. The mediocrecompany rushes to fix blame on someone else, while theeffective company finds ways to solve the problem.Meyer goes on to state that the investigator can learn alot about the company by watching its performanceduring the investigation process.

The Business Process Method of EvaluatingManagement

W. Edwards Deming (15), the eminent quality manage-ment guru, attributed the vast majority of manufacturingdefects to problems caused by management systems.Certainly, some defects are caused by human error, butin most cases workers are doomed to make mistakesbecause of elements within their work environment.

Deming’s writings helped transform world business intoa quality mind set. Airline companies and airplanemanufacturers embraced Total Quality Managementtechniques and the accompanying assortment of statisti-cal tools, including Pareto charts, statistical processcontrol charts, fishbone diagrams, etc. These techniquesinvolved the application of flow charts and detailedprocess analysis to detect causes of manufacturingdefects.

Can the majority of airplane accidents, like manu-facturing defects, be traced to management systems?When a report concludes “human error,” could this errorbe attributable to management systems?

In the investigation of a crash in Alberta, Canada, theCanadian Aviation Safety Board wrote (5), “Althoughthe final decision in a aircraft cockpit rests with thecaptain, that decision is often influenced by factors overwhich he had no direct control.”

The issue of individual accountability arose in a dissent-ing opinion of an investigation of a runway collisionbetween a Boeing 727 and a Beech King Air A100 (5):

I also disagree with the notion that agenciescause accidents. Failure of people and failure ofequipment cause accidents. Shifting the causefrom people to agencies blurs and diffuses theindividual accountability that I believe is criti-cally important in the operation and mainte-nance of the transportation system.

In these two investigations the issue of blame—manage-ment system or human error—is no different from themanufacturing analogies found in Deming’s arguments.

Is it possible to establish a management system in whichan airplane accident cannot happen? Probably not. But itis possible to create a culture of concern for safety thatcan save lives. Meyer (13) encouraged a managementculture audit as a safeguard to prevent accidents. Reason(4) pointed out that a delicate balancing act exists be-tween safety and production goals. An organization witha culture heavily stressing production goals couldnegatively impact safety goals. The previously men-tioned accident in Alberta, Canada, fell into this cat-egory. The Canadian Aviation Safety Board concludedthat the pressure for the pilot to meet his schedulecaused him to compromise safety, and resulted in acrash.

Managers who want to create a system within a safety-conscious culture that will reduce the possibility of anaccident must begin with the very basic tenants ofmanagement: the establishment and embracement of amission, strong leadership, detailed and explicit policiesand procedures, and a motivated work force. Deming(15) listed sixty-six questions to “help managers.” Someexamples of Deming’s questions are:

• What proportion of your workers has a chance tounderstand the requirements of the next operation?

• Are you changing supervision to leadership, at leastin some of your organization?

• How do you distinguish between the quality asyour customer perceives it and quality as your plantmanager and workforce perceive it ?

• What proportion of the troubles that you have withquality and productivity are the fault of (a) produc-tion workers? (b) the system (management responsi-bility)?

Perhaps Deming’s (15) most important principle was thatthere must be an organizational transformation led bysenior management to instill a quality culture within thecompany. All the tools employed in total quality man-agement and process analysis are futile employedwithout this transformation of culture and attitude.Meyer’s (13) culture analysis may be one approach to getto the core of management problems. Another approachwould be to systematically train all levels of manage-ment on tradeoffs between production and safety. Whensenior management makes choices that favor reducingcosts over safety, pilots will often reflect that way ofthinking with their decision-making in the cockpit.


Case Studies of accidents that involve managementfactors

Case Study #1Eastern Airlines L-1011, May 5, 1983, Miami, FloridaThis near tragic accident, with 162 passengers on board,was caused by mechanics who failed to replace O-ringseals on the metal chip detectors following routineinspections of all three engines of the L-1011. When thefirst engine ran low on oil, the crew shut the enginedown. When the second and third engines failed due tothe loss of oil, the first engine was restarted and pro-vided the power to return to Miami. NTSB focused itsinvestigation on the maintenance procedures andsupervision, mechanic training, and FAA oversight.

The mechanics who reinstalled the chip detectors werenot aware of the need for O-ring seals or the need tomotor the engine long enough to check for oil leaks.While the procedure had been revised due to a numberof recent and similar incidents, oversight by managersand retraining of mechanics had not occurred. FAA wasinvolved in the redesign of the procedure, but had notchecked to see if proper guidance reached line mechan-ics. The safety department at EAL had recently lost muchof its staff due to cost cutting measures.

In its findings, NTSB pointed to the failure of linemechanics to follow proper procedures in installing chipdetectors and checking for oil leaks. The Board alsofound the Maintenance Department of EAL first linesupervision of mechanics inadequate. Middle and uppermanagement were cited for not identifying the problemas a significant one nor taking proper corrective actions.FAA was cited for failing to see the problem as a majorconcern.

In the authors’ view, the accident investigation wasthorough but did not reach high enough within thecompany to identify real shortcomings. To some extent itis easier in retrospect to see the changes that wereoccurring within the airline industry now that some timehas passed. In 1983 Continental Airlines was on strike,Braniff Airlines was attempting to return from bank-ruptcy, and Eastern Airlines was struggling with acorporate culture of difficult relations between themanagement and workers. A few years later Eastern wastaken over by Texas Air Corporation and the finaldemise of Eastern began.

It was apparent that previous incidents signaled theweakness of the system. The safety department wasunable to act in a timely manner because of lack ofpersonnel who tracked the data, and because of poorcooperation between maintenance and safety. Uppermanagement must have known of the poor relationshipbetween safety and maintenance, did nothing to breachthe gap, and left itself blind to problems maintenancewould rather forget about. Blaming the individual

mechanic was convenient and easy. The corporateculture of Eastern was ripe for this type of shifting ofblame. In the end it did nothing for safety and shouldhave been addressed by NTSB. A similar review of FAAmanagement would uncover a culture of focusing on thefailure of the individual, whether it be an airline, a pilot,a mechanic, or a controller.

Case Study #2USAir B-737 and Skywest Metroliner runway collisionat Los Angeles International Airport, February 1, 1991A Skywest Metroliner was cleared into takeoff position,just after sundown in Los Angeles, at a point furtherdown the runway than the normal takeoff point. AUSAir B-737 landed on the same runway and a groundcollision resulted, killing 32 occupants of the two aircraft.

During the investigation it was found that the air trafficcontroller who was handling the two aircraft was dis-tracted and forgot about the Skywest airplane. An aid tomemory for air traffic controllers, in the form of a datacard, was not used for aircraft that departed frompositions other than the normal takeoff position at theend of the runway. The pilots of the B-737 were unable tosee the Skywest in the takeoff position on the runwaybecause of night conditions. The Surface DetectionRadar, which would have shown the position of aircrafton the ground, was unreliable and not used by thecontroller. The standardization of Air Traffic Control(ATC) procedures was to some degree left in the handsof the individual facilities’ managers. The controllerresponsible for the runway had been evaluated a fewweeks before the accident, and was found to be weak incertain important skills and procedures.

The NTSB findings cited the careless lack of re-dundancies that should have supported the air trafficcontroller, especially in terms of memory facilitatorssuch as the data cards and backup personnel. These wereclearly in the purview of the local manager. Acquisitionof new equipment, a task assigned to WashingtonHeadquarters, was seriously behind schedule and mayhave prevented the accident. In addition, the lack ofconspicuous aircraft lighting and poor voice communica-tions procedures were cited.

NTSB recommended, as in previous accidents, that safetyoversight of ATC facilities be transferred to an organiza-tion independent of FAA. FAA had taken steps toaccomplish this but had maintained control within FAAand compromised this important function.

NTSB once again focused on the Los Angeles towersituation, but provided little insight into the manage-ment of FAA except as it pertained to new equipmentand local styled procedures. One of the most telling factsin this accident was the note by one of the investigatorsthat the manager of the facility, shortly after the accident,called Headquarters to discuss whether this occurrence


might be classified as an operational error by the LosAngeles tower. An operational error is considered aserious infraction within the FAA/ATC system. Head-quarters assured Los Angeles ATC personnel that thiswould not be considered an operational error. Thisincident is typical of the mentality of military styleorganizations, where errors are cause for disciplinerather than an indication of a problem in need of correc-tion. The NTSB investigators should have followed thisincident to its highest source, for it depicts an environ-ment of hiding problems rather than a willingness tosolve them.

Case Study #3Northwest Airlink BA 3100, December 1, 1993,Hibbing, MNDuring a night, non-precision approach, in marginalweather, NWAirlink 5719 crashed short of the runway.During the last portion of the flight, a high rate ofdescent was initiated. All 16 passengers and 2 crewmembers were killed.

The captain had a long history of failed examinations.Most recently he had failed a proficiency check in May of1993. Though there was ample evidence of his difficultiesin passing check rides and other examinations, as well ashis inability to conform to standard behavior of a pilot incommand, the company management, in the person ofthe Director of Flight Operations for Express II (theparent company), stated he was unaware of the captain’shistory. The first officer, while younger than the captain(42 vs. 25), also had a history of failed FAA examinationsfor private pilot and instructor pilot.

NTSB’s Probable Cause was stated as: “The NationalTransportation Safety Board determines that the prob-able causes of this accident were the captain’s action thatled to a breakdown in crew coordination and loss ofaltitude awareness by the flight crew during anunstabilized approach in night instrument meteorologi-cal conditions. Contributing to the accident were: Thefailure of the company management to adequatelyaddress the previously identified deficiencies in airman-ship and crew resource management of the captain; thefailure of the company to identify and correct a wide-spread, unapproved practice during instrument ap-proach procedures; and the Federal AviationAdministration’s inadequate surveillance and oversightof the air carrier.”

Of the 18 Findings reported by NTSB, three were di-rected at the management factors of this accident. Theywere: “16. The airline’s flight operation managementfailed to implement provisions to adequately oversee thetraining of their flightcrews and the operation of theiraircraft. 17. FAA oversight of the airline was inadequate.18. FAA guidance provided FAA inspectors concerningthe implementation of the Air Carrier Operations Bulle-

tins is inadequate and has failed to transmit valuableinformation as intended to airlines.”

The technical/operational aspects of the investigation ofthis accident were carried out in an exemplary fashion.However, discovering the relationship of management’srole in this accident was sadly lacking. Both FAA andExpress II were derelict in their duties to prevent anoccurrence such as this.

In the case of FAA, some evidence of a dispute wasapparent in that the Principal Operations Inspector forFAA in Memphis refused to oversee and certify theExpress II operation in Minneapolis. The Flight Stan-dards District Office in Des Moines, Iowa, was given theresponsibility to monitor Express II, although the aircarrier did not operate into that city.

Evaluating Management Factors in the Post AccidentAnalysis

To get at the facts of whether air operations are con-ducted in a safe fashion, the trained accident investigatorknows that the elements of properly maintained equip-ment, properly trained flight crew members, all operatedin accordance with company policies and manufacturers’instructions, are key elements that exist in a safe organi-zation. The investigator also looks for other factors, suchas organization and communications for safety, as anadditional layer of dedication toward the safety mission.

However, accident investigators have not been providedthe tools to evaluate the role that managers play in theevents preceding an accident. Business schools in ouruniversities use multiple tools to develop a picture ofbusiness methods in place in a corporation. Studentswho are destined for management positions seek MBA(Masters of Business Management) degrees and aretaught how successful businesses conduct their affairs.

The lessons coming from our business schools teach usthat management techniques are not very mysterious.The best and safest managers are good leaders, unafraidof the truth, dedicated to quality, able to relate andcommunicate with workers, and who take seriously theirresponsibility to the public.

The accident investigator should look for these qualities.Audits may take the form of interviews in and outsidethe company, questionnaires, financial statement re-views, etc. Items such as the following should be in-cluded in a review of company practices:

• culture• financial stability• decision process by managers• written and unwritten policies• communications vehicles• commitment by senior managers


For instance, the investigator must understand theculture of a company, especially as it relates to safetymatters such as trust and confidence between managersand line workers. Safety problems are most often pre-ceded by near accidents, and the worker must feel his/her report of the near mishap or hazardous conditionwill not result in disciplinary action. An environmentwhich results in disciplinary action taken against theworker who reports a hazard or near-accident is counter-productive. Managers who literally hide their heads inthe sand are not accepting their ethical responsibilities toworkers or the public.

One method that is in current use in evaluating themanagement system is the use of realistic scenarios inwhich the respondent is forced to make a decisionamong conflicting alternatives. For example, one sce-nario might be designed around the problem of theairline that has the reputation for good on-time perfor-mance. In order to preserve that reputation, an arrivingflight has the option of making up time by conducting asomewhat hazardous approach to save time. Scenariosshould be administered to all levels of management andthe flight operations. Ideally, the scenario respondentsare in agreement as to the desired course of action. Highdegrees of conflicting responses indicate the need forsome management training, and the possibility thatcommunications between senior management, middlemanagement, and line workers is not satisfactory.

A typical example of a scenario might look like this:

A charter flight is enroute to an island destination.The weather was marginal before departure, butdispatch indicated that improvement was expectedwhen the flight was due to arrive. Upon arrival,heavy rain and gusty winds are present. Both theceiling and winds are close to allowable limits.

As the Captain what is your decision?

a. Try a landing because your chief pilot willexpect you to?

b. Make a landing because company profitswill be negatively impacted if you proceedto another airport?

c. Hold in the area for improvement and if noimprovement proceed to another airport?

d. While enroute, change your destination toone that will provide alternate transporta-tion to the passengers, or where you canwait for an improvement in the weathersuch that you can land at the originaldestination?

As a middle level manager what do you expect theCaptain to do?

a. Attempt a landing but not endanger thepassengers and crew?

b. Make a landing as long as it is legal soprofits are not negatively impacted?

c. Hold in the area for improvement and if noimprovement proceed to another airport?

d. While enroute, change the destination toone that will provide alternate transporta-tion to the passengers, or where they canwait for an improvement in the weather sothat a landing can be made at the originaldestination?

Conclusion

There appears to be no lack of interest in the aircraftaccident investigation community in seeking the rela-tionship that exists between management practices andsafety of flight operations. Yet only general guidelinesexist to assist the accident investigator in the conduct ofan investigation.

Accidents such as the ValuJet crash in the Florida Ever-glades on May 11, 1996, have brought a very tight focusto issues addressed in this paper. ValuJet had farmed outmany of its maintenance, training, and service responsi-bilities to outside organizations. Following the accident,FAA, which was the certifying authority, stated thatValuJet was in complete compliance with FAA regula-tions. However, within a month FAA was back-pedaling,and removed a high level safety manager. Some weekslater the Administrator of FAA announced that he wouldresign at the end of the year. It ought to remind all of usthat safety guardians’ management system, such as thatof FAA, may need to be reviewed as much as anoperator’s management system.

The authors have undertaken the task of describing thepresent system, of evaluating management’s role inaircraft accident causation, briefly describing the methodof management evaluation carried out in businessschools, and reviewing past accident investigations thatcontained elements of questionable management prac-tices.

In a future paper, the authors intend to be more specificabout how the investigator can conduct a thoroughreview of management system practices that influencethe safety of flight operations.


Bibliography

1. Manual of Aircraft Accident Investigation (Doc 6920-AN/855/4). International Civil Aviation Organiza-tion. Montreal, Canada, 1970.

2. Accident Prevention Manual (Doc 9422-AN/923).International Civil Aviation Organization.Montreal, Canada, 1984.

3. Human Factors Digest No. 7. Investigation ofHuman Factors in Accidents and Incidents (Circular240-AN/144), International Civil Aviation Organi-zation. Montreal, Canada, 1993.

4. Reason, James T. Human Error. Cambridge, Cam-bridge University Press, 1990.

5. Human Factors Digest No. 10. Human Factors,Management and Organization. Circular (247-AN/148), International Civil Aviation Organization.Montreal, Canada, 1993.

6. Personal communiqué. May 7, 1996.

7. Miller, C. O. “Management Factor InvestigationFollowing Civil Aviation Mishaps”. forum 21, 1988.ISASI, Sterling, VA.

8. Miller, C. O. “Investigating the ManagementFactors in an Airline Accident.” Brazilian Congressof Flight Safety, Rio de Janeiro, Brazil. November26, 1990.

9. Miller, C. O. “Accident Prevention Principles/Policies for Senior Aviation Managers.” System

Safety, Sedona, Arizona. September 23, 1995.

10. Wood, Richard H. Aviation Safety Programs - AManagement Handbook. IAP, Inc., Casper WY,1991.

11. Reason, James T. “Identifying the Latent Causes ofAircraft Accidents Before and After the Event.”forum 24, 1991. ISASI, Sterling, VA.

12. Diehl, Alan E. “The Aviation Safety Impact ofOrganizational Ergonomics and Stressors.” forum26:4, December 1993. ISASI, Sterling, VA.

13. Meyer, Jan. “Beyond Reason: How to Conduct anInvestigation of Organizational and NationalCultures.” forum 27:5, December 1994. ISASI,Sterling VA.

14. Champy, James. Reengineering Management. NewYork, Harper Business, 1995.

15. Deming, W. Edwards. Out of the Crisis. Cambridge,MA. MIT, 1982.

Richard B. Stone (WO0837), is the current InternationalPresident of ISASI. He served two terms as the US Councillorfrom 1984 to 1988, and was Chairman of the ISASI Interna-tional Seminar in Atlanta, October, 1987. Captain Stoneretired from Delta Airlines in 1992, after thirty five years offlying, and has investigated many major aircraft accidents.

Scott T. Young is the chairman of the Management Depart-ment at the David Eccles School of Business at the Universityof Utah in Salt Lake City. He is the author of ManagingGlobal Operations (Quorum Books, 1996).


Introduction

Considerable literature exists concerning the detailedengineering analysis of aircraft accidents. However, onlylimited information is readily available specificallyrelating to on-site engineering investigation require-ments for typical general aviation accidents, i.e., acci-dents involving single and twin engined aircraft under5,700 kg (MTOW). This paper outlines an engineeringguide for investigators conducting general aviationaccident investigation in the field.

The guide aims to provide a framework for the fieldinvestigator to follow, via a series of checklists groupedtogether under related engineering topics. It is intendedfor use during the limited period of time that the investi-gator is present at the accident site. By completing theappropriate checklist items, the chances of overlookingperishable evidence available only at the accident scenewill be minimised. In this manner, the investigationprocess is streamlined and the overall efficiency of theinvestigation is enhanced.

Background

A review of engineering texts found no useful on-siteengineering guide existed that could be readily appliedin the general aviation environment. Although theauthor’s review determined that most engineering textsdo contain considerable amounts of information that isrelevant to on-site data collection, this information israrely presented in a step-by-step format that is condu-cive to work in the field. It is also mainly directedtowards the investigation of larger aircraft types. Inaddition, bulky and volumous engineering texts, al-though essential as part of the later detailed engineeringanalysis, are not necessarily appropriate at accident sites,particularly if they are in a remote location.

Using the Checklist

The guide is made up of a series of checklist forms. Theseforms are composed of individual tick-boxes andprompts that are grouped together by engineeringsystem. It is intended that the investigator should workthrough the checklist forms in a thorough and logicalmanner until all items have been addressed and the

relevant information appropriately recorded. Not everychecklist item will, of course, be applicable to eachaccident scenario. It is left up to the discretion of theindividual investigator to determine whether or not aparticular checklist item is appropriate or relevant.However, by following the guide, aspects of the investi-gation will be addressed that might otherwise have beenoverlooked or possibly not considered.

The broad content of this paper can be broken into fourseparate sections:

• Notification/Identification• Arrival• Initial Investigative Actions• Collection of Factual Information

The first three sections represent the initial actions thatshould be undertaken prior to any physical investigativework. The individual checklist steps for these threesections appear on the first two checklist forms entitledIdentification and Initial Investigative Actions, and areexpanded upon in the following text. It is recommendedthat these three sections be addressed as thoroughly aspossible, as their content is applicable to most accidentscenarios. The remaining Collection of Factual Informationsection constitutes the majority of the engineeringchecklist forms. These forms may be completed in anyorder, depending upon the type of accident.

The completion of the Collection of Factual Informationchecklist is self-explanatory. It does, however, assume abasic knowledge and understanding of engineeringinvestigation in order to be completed effectively. Thechecklist has been kept brief and concise to ensure its“user-friendliness” is not compromised. Its strengths relymore on ease of use and basic simplicity than on detail.

Notification/Identification

The on-site investigator should consider the need toaccess maintenance manuals and parts cataloguesrelevant to the accident aircraft. Efforts to secure thesedocuments should be made prior to departure to theaccident site. These documents form an essential part ofany effective engineering investigation and, if practi-cable, represent an invaluable asset at the accident

Light Aircraft Accident InvestigationAn Engineering Guide

Pieter van Dijk A03838Bureau of Air Safety Investigation


location. The identification of partial, individual, ordamaged components is made all the easier with theavailability of such documents

In addition, steps should be undertaken to quarantinethe accident aircraft’s maintenance documentation (e.g.,log books, worksheets, recent servicing schedules, a copyof the maintenance release, etc.). This may require theinvolvement of other investigators or governmentofficials located closer to the aircraft’s maintenancefacility. It is also desirable that an initial review of thedocumentation be conducted as soon as possible, focus-ing on any possible abnormalities, unincorporatedAirworthiness Directives, overdue servicings or inspec-tions, etc., that may have accident implications. Theseissues may require specific attention during the on-siteinvestigation.

The opportunity may also exist for the appointment oftechnical advisers to assist with the engineering investi-gation. This would involve securing the dedicatedservices of one or more experienced specialists on theaircraft type concerned, usually for the duration of theon-site phase of the investigation. This approach cangreatly assist the overall efficiency of the investigation.The ability of a specialist technical adviser to recogniseand identify components and/or possible abnormalitieswithin the wreckage is likely to be superior to that of thegeneralist investigator. It is preferable that an adviser beselected who has not been associated with the mainte-nance of the aircraft, to minimise potential conflicts ofinterest.

If the accident site is remote or located away fromsupport facilities, consideration should be given to theneed to take the necessary tools to conduct at least abasic engineering investigation. A list of recommendeditems that could be considered appropriate for anengineering fly-away kit are also included as part of theNotification checklist.

Arrival Checklist

Local police, fire, ambulance, and emergency authoritieswill normally be among the first on the accident scene.The investigator should establish from these personnelthe extent to which the wreckage may have been movedor disturbed. Similarly, the existence of any police orother photographs, as well as any television or amateurvideo footage, should be determined during these initialconsultations. It is also advisable that the investigatorphotograph before hand any aspect of the wreckage thatis to be disturbed as part of the (safety) Arrival checklistactions.

It is outside the scope of this paper to cover all the sitesafety precautions and procedures from an occupationalhealth and safety perspective. However, the investigatormust ensure that potential hazards of an engineering

nature that exist within the wreckage have been ad-equately addressed. The Arrival checklist should becompleted prior to the commencement of any investiga-tion work.

Initial Investigative Actions

Once the Arrival checklist has been completed, an initialwalk-through of the accident site should be conducted.This initial wreckage survey is an integral part of theoverall engineering investigation. It is intended toestablish an overview of the investigation task, as well asto provide a first impression of the type of accident. Thispreliminary determination is likely to lead to one ormore hypotheses that may influence the future directionof the investigation.

The different types of accidents can be divided into thefollowing six categories:

• Inflight collision• Airframe failure inflight• Stall, spin or spiral• Controlled flight into terrain (CFIT)• Fire/explosion inflight• Forced landing or ditching.

The initial wreckage walk-through also provides anexcellent opportunity to photograph the undisturbedwreckage. Photography of the accident site and in-dividual wreckage items should be prolific. Specialattention should be given to all obstacle and groundmarks as well as cockpit instrumentation settings andcontrol positions. A comprehensive photographic recordmay prove to be crucial reference material during lateranalysis when access to the wreckage may no longer bepossible. The wreckage survey should also try to estab-lish whether or not all aircraft components are containedwithin the main wreckage area. This can be achieved byattempting to locate and identify the extremities of theaircraft (i.e., nose, tail, and wing tips). If these items canbe located, the remainder of the wreckage will, in mostcases, be contained within the boundaries of these items.

Sketching of the first details on the Wreckage DistributionDiagram should now be undertaken. It is recommendedthat a Wreckage Distribution Diagram be drawn for allaccidents, especially where the aircraft does not remainwhole. It should display as much detail of the wreckageand surrounding environment as reasonably practicable.The aim of the diagram is to produce a scaled plot on agrid, showing the proximity of components relative toeach other and to the surrounding landscape features.The diagram should also show the position and dimen-sions of any ground marks, trees, and/or structures thatmay have been involved in the accident sequence.

An accurate Wreckage Distribution Diagram is especiallyuseful during the analysis phase in the event that the


aircraft has broken-up inflight, with consequentialwreckage scattered on the ground. Analysis of thediagram will provide valuable clues as to the airspeed,height, and break-up sequence of the aircraft. Thecompleted diagram will show facts about the accidentthat might otherwise be more difficult to ascertain whenviewing individual wreckage components in isolation. Itwill also provide a practical means of visualising theentire wreckage path. Similarly, aerial photography ofthe accident site and wreckage trail should be consideredif resources permit.

Where possible, fuel samples should be taken at thecompletion of the Initial Investigative Actions phase. Ananalysis of the fuel will determine whether or not itmeets the required specification, or contains contami-nates. This is an essential component of the overallinvestigation process and should not be overlooked. Iffuel contamination cannot be discounted as a possiblecontributing factor to the accident, immediate arrange-ments should be made to obtain and analyse fuelsamples from the last fuel source used by the aircraft.Consideration should also be given to quarantining thefuel source until the results of the fuel sample analysisare known.

Collection of Factual Information

On completion of the Initial Investigative Actions, it isrecommended that the remainder of the engineering on-site investigation should be conducted in accordancewith the relevant sections of the On-Site EngineeringChecklist forms. By addressing each engineering systeminturn, the investigator will be prompted to consideraspects of the wreckage that may otherwise be over-looked. Not all checklist items will be appropriate inevery case, as the circumstances surrounding eachaccident will vary. The investigator should apply thechecklists as appropriate. Each system checklist also hasspace at the bottom of the form allotted for additionalnotes and/or diagrams so that particular items ofinterest can be recorded or expanded upon.

If the checklist is to be used for an accident involvingmulti-engined aircraft, additional extra copies of theappropriate engine and propeller checklists should bemade prior to dispatch. This will ensure that all checklistdetails are able to be recorded on separate forms relatingto each individual powerplant. Once all boxes have beencompleted on a particular checklist, the appropriateengineering system box on the master engineering checklist, located on the Initial Investigative Action form, shouldbe completed. The investigator will thus be able tomonitor the overall progress of the investigation.

Summary

The process of air safety investigation can be greatlyassisted by the availability of specialist investigationguides. Such guides should be based upon the collectiveknowledge and experience of all appropriate members ofaccident investigation organisations and the air safetycommunity. The guides should be subject to criticalreview and continual update, together with the input ofany innovative ideas, in order to establish and maintaintheir relevance and usefulness as an investigative tool.

In developing this paper, the author has had the oppor-tunity to review leading texts available on the subject,and has been able to draw upon the expertise andexperience of several other engineering investigatorsfrom within the Bureau of Air Safety Investigation.Despite this valuable input, the On-Site EngineeringChecklist as it now stands is only a first step towards amature document.

References

1. Bruggink, Gerard M. Managing a Small Scale Investi-gation. Transportation Safety Institute. OklahomaCity, Oklahoma, June 1985.

2. US Air Force Guide to Mishap Investigation. AFP 127-1Volume 1, 29 May 1987

3. Manual of Aircraft Accident Investigation. InternationalCivil Aviation Organisation, Fourth Edition 1970.

4. Dole, Charles E. Safety Technology Workbook. SouthernCalifornia Safety Institute, 1994.

5. Wood, Richard A. and Robert W. Swegginis. AircraftAccident Investigation. Endeavor Books, 1995.

6. Manual for Major Accident Investigation. Bureau of AirSafety Investigation, 1994.

Pieter van Dijk joined the Royal Australian Air Force inJanuary 1984 as an Engineering Cadet at RAAF Frognall. Hegraduated from the Royal Melbourne Institute of Technologyin December 1987 with a Bachelor of Engineering in Manufac-turing Systems. He left RAAF in February 1993, and com-pleted a Masters of Business Administration in TechnologyManagement from Deakin University. He later took up theposition as a Policy Analyst (Engineering) with the Institu-tion of Engineers, Australia, before joining the Bureau of AirSafety Investigation in May 1995 as a Air Safety Investigator(Engineer).


LIGHT AIRCRAFT ACCIDENT INVESTIGATION

ON-SITE ENGINEERING CHECKLIST

IDENTIFICATION

DATE OF OCCURRENCE: .........................................................TIME OF OCCURRENCE (LOCAL): ..............................................DATE OF INVESTIGATION: ......................................................LOCATION: ........................................................................NAME OF INVESTIGATOR: ......................................................ORGANISATION: ..................................................................

AIRCRAFT DETAILS:

REGISTRATION: ..................................... SERIAL NUMBER: ......................................MAKE: ................................................ YEAR OF MANUFACTURE: ...........................MODEL: .............................................. TOTAL AIRFRAME HOURS: ..........................

ACCIDENT DETAILS:

ACCESS AIRCRAFT MAINTENANCE MANUAL

ACCESS AIRCRAFT PARTS CATALOGUE

ARRANGE QUARANTINE OF AIRCRAFT MAINTENANCE DOCUMENTATION

ARRANGE SPECIALIST TECHNICAL ADVISER

CHECK CONTENTS OF FLY-AWAY KIT

NOTIFICATION CHECKLIST :

LIAISE WITH POLICE TO ESTABLISH EXISTENCE OF PHOTOGRAPHS/VIDEO FOOTAGE

DETERMINE EXTENT TO WHICH WRECKAGE HAS BEEN INTERFERED WITH

ASSESS VOLATILITY OF CARGO AND FUEL

ENSURE APPROPRIATE PERSONAL ATTIRE FOR ACCIDENT SITE CONDITIONS (OH&S)

DISCONNECT AND REMOVE BATTERY

RELEASE TYRE PRESSURES

RELEASE HYDRAULIC/PNEUMATIC PRESSURE

RELEASE HYDRAULIC PRESSURE IN LANDING GEAR STRUTS

ORGANISE REMOVAL OF OXYGEN BOTTLES

REMOVE ANY FLARES AND SQUIBS

ARRIVAL CHECKLIST:NOTE: TAKE RELEVANT PHOTOGRAPHS PRIOR TO DISTURBING WRECKAGE

ISSUE 1 (SEP 96) REFERENCE NO:



ELECTRICAL

NOTES/DIAGRAMS:

CHECK STATUS OF CIRCUIT BREAKERS

CHECK ASSOCIATED WIRING FOR LOOSE CONNECTIONS

PHOTOGRAPH DAMAGED ELECTRICAL COMPONENTS AND WIRING

CHECK ASSOCIATED WIRING FOR CHAFFING OF INSULATION - ESPECIALLY AROUND CLAMPS HOLDING

WIRE BUNDLES

CHECK FOR EVIDENCE OF BURNING IN WIRE BUNDLES

- EXTERNAL BURN: INSULATION BURNT, BUT WIRE BRIGHT AND SHINY UNDERNEATH

- ELECTRICAL OVERLOAD: DISCOLOURATION THROUGHOUT ENTIRE WIRE STRAND CROSS

SECTION

INSPECT SELECTED SEVERED ENDS OF ELECTRICAL WIRE

- POWER ON AT SEPARATION: ROUNDED MELTED GLOBUALS ON WIRE STRAND ENDS

- POWER OFF AT SEPARATION: CLEAN NECKING DOWN AT WIRE FRACTURE ENDS

EXAMINE LIGHT BULBS UNDER MAGNIFYING GLASS (i.e. WARNING LIGHTS)

- POWER ON AT IMPACT: STRETCHED FILAMENT

- POWER OFF AT IMPACT: INTACT FILAMENT - OTHERWISE A CLEAN BREAK WITH NO SIGN OF

FILAMENT STRETCHING

CHECK FOR EVIDENCE OF LIGHTNING STRIKE

CHECK STATE OF BATTERY


SHOULD THE PRE-IMPACT INTEGRITY OF ANY ITEM BE IN DOUBT,REMOVE FROM WRECKAGE AND TAG FOR FURTHER ANALYSIS.



FLIGHT CONTROLS

REFERENCE NO:

NOTES/DIAGRAMS:

ACCOUNT FOR ALL FLIGHT CONTROL SURFACES

PHOTOGRAPH POSITIONS OF ALL CONTROL SURFACES AND ASSOCIATED DAMAGE

PHOTOGRAPH AND MEASURE EXTENSION OF ACTUATORS

CLOSELY INSPECT STRUCTURE ADJACENT TO ALL FLIGHT CONTROL SURFACES FOR NICKS, MARKS,

EXCESSIVE WEAR AND/OR LOCALISED DAMAGE etc. THAT MAY INDICATE POSITION AT TIME OF

IMPACT.

TORQUE TUBES CONTROL RODS

ACTUATORS HINGES

CHECK CONTINUITY AND INTEGRITY OF ALL CONTROL CABLES AND RODS etc. FROM COCKPIT TO

CONTROL SURFACE FOR SIGNS OF PRE-IMPACT DAMAGE

CHECK CABLE ATTACHMENT POINTS

CHECK FOR PRESENCE OF FOREIGN OBJECTS IN WRECKAGE

REMOVE HYDRAULIC FILTERS AND SEAL TO RETAIN CONTENTS

ISSUE 1 (SEP 96)

DETERMINE THE POSITION OF THE FLIGHT CONTROLS AT IMPACT. ATTEMPT TO ESTABLISH WHETHER

OR NOT PHYSICAL CONTROL SETTINGS MATCH CORRESPONDING COCKPIT SETTINGS.

INSPECT FOR EVIDENCE OF JAMMING, INTERFERENCE, FLUTTER, IMPROPER ASSEMBLY AND/OR LACK OF

LUBRICATION etc:



FLIGHT C ONTROLS (cont d)

REFERENCE NO:

NOTES/DIAGRAMS:

CHECK FOR EVIDENCE OF GROUND MARKS (GRINDING DAMAGE) ON AILERONS

AILERON UP ON IMPACT: GRINDING ON BASE OF AILERON

AILERON DOWN ON IMPACT: GRINDING ON TIP OF AILERON

ISSUE 1 (SEP 96)

FLIGHT CONTROL

LEFT AILERON

RIGHT AILERON

RUDDER

ELEVATOR

PHYSICAL POSITION COCKPIT CONTROL POSITION

FLIGHT CONTROL

FLAPS

AILERON TRIM

RUDDER TRIM

ELEVATOR TRIM

PHYSICAL POSITION SELECTOR POSITION INDICATOR POSITION



LANDING GEAR

REFERENCE NO:

NOTES/DIAGRAMS:

PHOTOGRAPH ALL LANDING GEAR COMPONENTS

TRICYCLE FLOATS SKIDS TAILWHEEL AMPHIBIOUS

FIXED RETRACTABLE

CHECK BRAKE FLUID RESERVOIR

ISSUE 1 (SEP 96)

CONFIGURATION OF LANDING GEAR:

TYPE OF LANDING GEAR:

PARK BRAKES: ON OFF

LANDING GEAR SELECTION: UP DOWN

LANDING GEAR DEPLOYED: YES NO

CHECK EFFECTIVENESS OF UPLOCKS

CHECK EFFECTIVENESS OF DOWNLOCKS

CHECK CONDITION OF HYDRAULIC LINES

NOTE DIRECTION OF STRUT FAILURE

CHECK FOR EVIDENCE OF BRAKING

CHECK CONDITION OF TYRE(S)

SHOULD THE PRE-IMPACT INTEGRITY OF ANY ITEM BE IN DOUBT,REMOVE FROM WRECKAGE AND TAG FOR FURTHER ANALYSIS.


PRESENCE OF SLASH MARKS ON THE GROUND:

YES NO

IF SO;

x

x = DISTANCE BETWEEN SLASH MARKS (metres)

x = ............ (m)


PROPELLER NO.

NOTES/DIAGRAMS:

TYPE: CONST ANT SPEED FULL FEATHERING FIXED PITCH OTHER: .................

COMPOSITION: METAL WOOD COMPOSITE OTHER: ........................

NUMBER OF BLADES: .................

PROPELLER STILL CONNECTED TO CRANKSHAFT YES NO

PROPELLER FEATHERED: YES NO PROPELLER PITCH SETTING: ..............................

INDICATE BLADE DAMAGE:

MANUFACTURER: .................................................................

MODEL: .............................................................................

SERIAL NUMBER: .................................................................


NOTE: PROPELLER DAMAGE IS GENERALL Y MORE SEVERE WITH HIGHER ENGINE RPM ON IMPACT


PHOTOGRAPH ENGINE AND ALL ENGINE COMPONENTS IN-SITU

ENGINE COMPONENTS ACCOUNTED FOR YES NO

THROTTLE CABLES AND CONNECTIONS INTACT YES NO

MIXTURE CABLES AND CONNECTIONS INTACT YES NO

FUEL CONNECTIONS INTACT YES NO

FUEL LINES CLEAR YES NO

REMOVE FUEL FILTERS AND SEAL TO RETAIN CONTENTS

TAKE FUEL SAMPLES FROM LOWEST POINT IN EACH TANK

DETERMINE FUEL SELECTOR VALVE SETTINGS/ INTEGRITY

TAKE OIL SAMPLE

REMOVE OIL FILTER AND SEAL TO RETAIN CONTENTS

INSPECT FOR EVIDENCE OF PRE-IMPACT OIL LEAKAGE OR STAINS i.e. AIRFLOW PATTERNS


ENGINE (RECIPROCATING) NO.

ATTEMPT TO DETERMINE THE PRE-IMPACT INTEGRITY OF THE PROPULSION SYSTEM

i.e. WAS THE ENGINE CAPABLE OF DEVELOPING SUFFICIENT POWER AT IMPACT?

MANUFACTURER: ..................................................................

MODEL: ..............................................................................

TYPE: ................................................................................

SERIAL NUMBER: ..................................................................

MEASURED

ESTIMATED


FUEL TANK CONTENTS : TANK

LEFT

RIGHT

FUEL LINES

TOTAL

QUANTITY



ENGINE (RECIPROCATING) NO. (cont d)

NOTES/DIAGRAMS:

CHECK FUEL INJECTORS

CHECK CARBURETOR

CHECK FUEL PUMPS

NOTE POSITION OF ENGINE TIMING MARKS

CHECK DISTRIBUTOR

CHECK MANIFOLD LEADS

CHECK EXHAUST PIPES FOR EVIDENCE OF HEAT TINTING

ENGINE BLOCK INTACT YES NO

ENGINE CAPABLE OF POSSIBLE TEST RUNNING YES NO

SHOULD THE PRE-IMPACT INTEGRITY OF ANY COMPONENT BE IN DOUBT,

REMOVE FROM WRECKAGE AND TAG FOR FURTHER ANALYSIS.


CHECK PROPELLER GOVERNOR

CHECK SPARK PLUGS

CHECK MAGNETO(S)



TECHNICAL ADVISOR DETAILS

ADVISOR 1:

NAME: ............................................................................................................................TITLE: .............................................................................................................................

COMPANY/DEPARTMENT: ....................................................................................................ADDRESS: .......................................................................................................................

PHONE: ............................... FAX: .................................. MOBILE: ...........................

ADVISOR 2:NAME: ............................................................................................................................

TITLE: .............................................................................................................................COMPANY/DEPARTMENT: ....................................................................................................

ADDRESS: .......................................................................................................................PHONE: ............................... FAX: .................................. MOBILE: ...........................

ADVISOR 3:

NAME: ............................................................................................................................TITLE: .............................................................................................................................

COMPANY/DEPARTMENT: ....................................................................................................ADDRESS: .......................................................................................................................

PHONE: ............................... FAX: .................................. MOBILE: ...........................

NOTES: ................................................................................................................................................................................................................................................................

.......................................................................................................................................

......................................................................................................................................

......................................................................................................................................

........................................................................................................................................

.......................................................................................................................................

......................................................................................................................................

......................................................................................................................................

........................................................................................................................................

.......................................................................................................................................

......................................................................................................................................

......................................................................................................................................



IDENTIFY AND ACCOUNT FOR ALL THE MAJOR ENGINE COMPONENTS (PHOTOGRAPH)

CHECK THE ENGINE CASING FOR EXTERNAL EVIDENCE OF MECHANICAL FAILURE i.e. HOLES

CHECK THE ENGINE CASING FOR EVIDENCE OF BURN-THROUGH i.e. DISLODGED COMBUSTION BURNER

ISOLATE FUEL CONTROL UNIT AND REMOVE FROM WRECKAGE FOR FURTHER ANALYSIS

ISOLATE FUEL FLOW TRANSMITTER AND REMOVE FROM WRECKAGE FOR FURTHER ANALYSIS

CHECK BLEED AIR VALVE SETTINGS

REMOVE AND RETAIN MAGNETIC CHIP DETECTORS

TAKE OIL SAMPLE FROM BEARING LUBRICATION SYSTEM

CHECK PUMPS, GENERATORS etc. FOR EVIDENCE OF ROTATION DAMAGE i.e. FUNCTIONING ON IMPACT

CHECK STATUS OF THRUST REVERSERS

CHECK EXTENT OF GROUND DEBRIS THROUGHOUT ENGINE i.e. THE HIGHER THE RPM, THE FURTHER

THE INGESTION

CHECK FOR EVIDENCE OF ROTATIONAL INTERFERENCE OF COMPRESSOR BLADES WITH CASE OF

STATOR VANES i.e. ROTOR SHIFT - POSSIBLY DUE TO BEARING FAILURE

CHECK FOR EVIDENCE OF FOREIGN OBJECT INGESTION i.e. DAMAGE TO REAR OF COMPRESSOR

SECTION (CHECK FOR IMPRINT MARKS ON EARLY STAGE BLADES)

CHECK BACK SIDES OF COMPRESSOR BLADES FOR EVIDENCE OF SOOT - COMPRESSOR STALL

BLADE DAMAGE:

BENT IN OPPOSITE DIRECTION TO ROTATION - COMPRESSOR ROTATING ON IMPACT

NOT BENT OR BENT IN DIFFERENT DIRECTIONS - COMPRESSOR NOT ROTATING ON IMPACT

INTERNAL CASING DAMAGE:

KNIFE MARKS DUE TO BLADES - COMPRESSOR NOT ROTATING ON IMPACT

ROTATIONAL SCORING OR GOUGING - COMPRESSOR ROTATING ON IMPACT


ENGINE (TURBINE) NO.

ATTEMPT TO DETERMINE THE PRE-IMPACT INTEGRITY OF THE PROPULSION SYSTEM

i.e. WAS THE ENGINE CAPABLE OF DEVELOPING SUFFICIENT POWER AT IMPACT?


MANUFACTURER: ..................................................................

MODEL: ..............................................................................

TYPE: ................................................................................

SERIAL NUMBER: ..................................................................

COMPRESSOR:



ENGINE (TURBINE) NO. (cont d)

NOTES/DIAGRAMS:

CHECK CONDITION OF TURBINE BLADES

LONGITUDINAL CRACKING, JAGGED OR DEFORMED - TURBINE OVER TEMPERATURE

BLADES COATED WITH METALLIC GRIT ON FRONT FACE - OVERHEATING UPSTREAM

EVIDENCE OF BLADE CREEP - PERMANENT ELONGATION DUE TO STRESS AND HIGH

TEMPERATURE eg:1. ROTATIONAL BLADE SCORING ON INTERNAL TURBINE CASING

2. NECKING OF CENTRE OF BLADE

3. PERMANENT ELONGATION OF BLADE

SHOULD THE PRE-IMPACT INTEGRITY OF ANY COMPONENT BE IN DOUBT,

REMOVE FROM WRECKAGE AND TAG FOR FURTHER ANALYSIS.


TURBINE:

MEASURED

ESTIMATED

FUEL TANK CONTENTS: TANK

LEFT

RIGHT

FUEL LINES

TOTAL

QUANTITY



INITIAL INVESTIGATIVE ACTIONS

CONDUCT INITIAL WRECKAGE WALKTHROUGH (TAKE PHOTOGRAPHS)

ACCOUNT FOR ALL AIRCRAFT EXTREMITIES

COMMENCE PLOTTING WRECKAGE DISTRIBUTION DIAGRAM

CHECK FOR IMPACT MARKS WITH GROUND, STRUCTURE, TREE etc. UPSTREAM FROM WRECKAGE

ALONG AIRCRAFT FLIGHT PATH. RECORD DETAILS ON WRECKAGE DISTRIBUTION DIAGRAM

TAKE FUEL SAMPLES

CHECK FOR EVIDENCE OF BIRD STRIKE

APPROXIMATE SLOPE OF TERRAIN: ..............(Odeg) SLIDING DISTANCE: ...................(m)

CIRCLE AIRCRAFT IN 3 VIEWS TO INDICATE APPROXIMATE ATTITUDE AT TIME OF PRINCIPAL IMPACT

INITIAL INVESTIGATIVE ACTIONS

WRECKAGE DIAGRAM

FLIGHT CONTROLS

COCKPIT/INSTRUMENTS

CONDUCT ENGINEERING INVESTIGATION IN ACCORDANCE WITH RELEVANT SECTIONS OF ON-SITE

ENGINEERING CHECKLISTS (TICK APPROPRIATE BOX ON COMPLETION OF EACH SECTION)

MASTER ENGINEERING CHECKLIST:

RECIPROCATING ENGINE

TURBINE ENGINE

PROPELLER

STRUCTURAL

LANDING GEAR

ELECTRICAL

FIRE

NOTES/DIAGRAMS:


150o 135o 120o 90o 60o 45o 0o30o 150o135o120o90o60o45o30o 180o

0o 150o135o120o90o60o45o30o 180o10o-10o

0o

45o

15o

30o

45o

15o

30o30° 30°

45°45°



COCKPIT/INSTRUMENTS

ENGINE AND PROPELLOR CONTROL POSITIONS:CONTROL NUMBER 1 NUMBER 2THROTTLE/POWER LEVER

PITCH/PROPELLER

FUEL MIXTURE

CARB. HEAT/ALT. AIR

IGNITION SWITCH

TURBOCHARGER

WATERMETHANOL

L.P. COCK

H.P. COCK

FUEL CONTROL

FUEL SELECTOR

COWLS

FUEL BOOST PUMP

GEN./ALT. SWITCH

OTHER (DESCRIBE)

ENGINE AND PROPELLER INSTRUMENT INDICATIONS:INSTRUMENT NUMBER 1 NUMBER 2MANIFOLD PRESSURE

TACHOMETER

VDO/TACHO HOURS

FUEL FLOW

FUEL PRESSURE

FUEL QUANTITY

TGT/TOT/TIT/CHT

TORQUEMETER

OIL TEMPERATURE

OIL PRESSURE

OIL QUANTITY

E.P.RE.G.T.CARBURETOR TEMPERATURE

OTHER (DESCRIBE)


RECORD DETAILS AS APPROPRIATE



WRECKAGE DISTRIBUTION DIAGRAM

SCALE: ELEVATION: ARROW TO INDICATE NORTH:

SKETCH POSITION (AND RELATIVE PROXIMITY) OF WRECKAGE, MAJOR COMPONENTS

AND GROUND MARKS TO PERMANENT GROUND FEATURES. INCLUDE BURN AREAS.




FLY-AWAY KIT

SET OF OPEN-ENDED SPANNERS

ADJUSTABLE SPANNER

SOCKET SET

MALLET (SMALL)

CROW BAR (SMALL)

SCREW DRIVERS

WIRE CUTTERS

HACK SAW

STANLEY KNIFE

SWIVEL HEAD MIRROR

INCLONOMETER

APPROVED SAMPLE CONTAINERS (1 LITRE)

TAPE MEASURE

RULER

MAGNIFYING GLASS (10X)

TORCH AND SPARE BATTERIES

CAMERA AND SPARE FILM

CALCULATOR

COMPASS

TAGS

STRING

MARKING PENS

PLASTIC BAGS (DURABLE)

SEALING TAPE

OTHER.............................

APPROPRIATE CARRY CASE




COCKPIT/INSTRUMENTS (cont d)

FLIGHT AND OTHER INSTRUMENTS: INSTRUMENT 1 INSTRUMENT 2

ALTIMETER/SUBSCALE

AIRSPEED

TURN AND BALANCE

ARTIFICIAL HORIZON

V.S.I.

DIRECTIONAL GYRO

MAGNETIC COMPASS

CLOCK

O.A.T.

OTHER (DESCRIBE)

VACUUM PUMPS VACUUM PLUMBING VACUUM MANIFOLD

RETRIEVE FDR/CVR

CHECK PITOT STATIC SYSTEM INCLUDING EXPOSED PORTS

CHECK AUTOPILOT SELECTIONS

CHECK LIGHTING SWITCHES

CHECK CABIN HEATER i.e. CARBON MONOXIDE

CHECK INSTRUMENT PLACARDS

INSTRUMENT READINGS AT IMPACT MAY BE OBTAINED VIA WITNESS MARKS ON INSTRUMENT FACE.CONSIDER REMOVAL OF INSTRUMENT FOR FURTHER ANALYSIS.

CHECK FLIGHT REFERENCE SYSTEM:

NOTES/DIAGRAMS:




STRUCTURAL

NOTES/DIAGRAMS:


ACCOUNT FOR AS MUCH OF THE AIRCRAFT AS POSSIBLE

INVENTORY THE WRECKAGE

EXAMINE WRECKAGE (ESPECIALLY PRIMARY STRUCTURE) FOR EVIDENCE OF:

CORROSION DAMAGE

FATIGUE CRACKING

STRUCTURAL OVERLOADING

CHECK WRECKAGE FOR PRESENCE OF STRUCTURAL MODIFICATIONS

CHECK ANY COMPOSITE PANELS FOR EVIDENCE OF DELAMINATION

RECONSTRUCT PORTION OF AIRCRAFT WHERE FAILURE OCCURRED

CHECK INTEGRITY OF SEATS AND HARNESS

HARNESS USED ON IMPACT YES NO

RECORD AMOUNT OF STRUCTURAL DEFORMATION

ALL STRUCTURAL FAILURES ARE AS A RESULT OF OVERSTRESS. OVERSTRESS WILL OCCUR AS ARESULT OF DESIGN, MANUFACTURING OR MAINTENANCE INADEQUACIES; CORROSION OR FATIGUE

DAMAGE; AS WELL AS PILOT, WEATHER OR WAKE INDUCED MANOEUVRING

NOTE: DO NOT MATE CORRESPONDING FRACTURE SURFACES AS MICRO EVIDENCE OF

FAILURE MODE WILL BE DESTROYED.



FIRE

NOTES/DIAGRAMS:


HOT ENGINE SECTION

ENGINE EXHAUST

BRAKES

HEATER

SMOKING

ATTEMPT TO ESTABLISH WHETHER OR NOT ANY FIRE DAMAGE TO THE WRECKAGE

WAS PRESENT PRIOR TO IMPACT

POSSIBLE IGNITION SOURCES:

ELECTRICAL

LIGHTNING STRIKE

STATIC ELECTRICITY

OVERHEATED EQUIPMENT

AIRCRAFT FUEL

ENGINE OIL

HYDRAULIC FLUID

CARGO

POSSIBLE FUEL SOURCES:

ANTI-ICE FLUID

BATTERY GASES

WASTE MATERIAL

FIRE CHARACTERISTICS

TEMPERATURE

FLOW PATTERN

SOOT PATTERN

SCRATCHES

METAL DROPS

FRACTURE EDGES

INFLIGHT FIRE

5400OC

AIRSTREAM

AIRSTREAM

ON SOOT

AIRSTREAM

CLEAN

GROUND FIRE

2500-3800OC

RANDOM

RANDOM

UNDER SOOT

GRAVITY

BURNED/SOOTED


Figure 1. Civil Aviation Authority - Polygon ofcertainty

The New Zealand CAA Solution• Much academic work has been carried out recently by

ICAO and others into new methods for looking ataccidents and incidents. These method focus on thesystematic organisational approach, rather thanconcentrating on the active failures, which has been afeature of recent past investigations. It is importantthat a more balanced approach, using both active andlatent failure information, is adopted in the future.

• Since 1987, the New Zealand Civil Aviation Authorityand its predecessors have been developing a com-puter based system for information capture andanalysis. This paper will look at the driving forcebehind this development and at the cultural andenvironmental changes that have occurred within theframework of New Zealand’s legislated policy ofsafety at reasonable cost. It will discuss the AviationSafety Monitoring System (ASMS) in general, andshow how the new methodologies have been incorpo-rated. Data capture methods will be demonstratedand the general failure type profiles for the NewZealand aviation environment will be presented.

This system represents one method of bridging the gapfrom academic knowledge to a practical working man-agement information tool. Many aviation authoritiesaround the world are working to perfect such databasesystems, largely in isolation. The final part of this paperwill explore the reasons for this segregation and invitepossible solutions for future database developments.

At the recent FSF, IFA IATA conference held in Seattlelate last year, concern was expressed in relation to theexpected increase in commercial air traffic, and thepossibility of the number of accidents doubling. It iswidely accepted that 70 to 80 percent of these accidentswill involve Human Factors.

It is perhaps fortuitous that these problems are nowstarting to be better understood and ICAO has recentlypublished a series of circulars that give guidance on howto address Human Factors in a practical way. Circular253-AN/151 is the latest.

Many of the methodologies presented in the most recentpublications are based around work carried out by

Professor James Reason and Professor Willem Waganaar.Fortunately, Jim has covered both the academic prin-ciples and offered practical application methods.

One element of this is the concept of the safety spacewhere an organisation may be either more or less likelyat risk of having an accident depending on a number ofspecified parameters being identified.

From a regulators point of view, we have a safety spacewhose boundaries are defined by aviation rules andadvisory circulars. These requirements must be satisfiedbefore an organisation is allowed into the aviationsystem. As a regulatory authority, we then monitorcompliance with these rules as one method of establish-ing continued safe operation. We also measure thenumber of accidents and incidents that are occurring todetermine how safe we are in relation to defined safetytargets. The ultimate goal of 100% compliance with therules and no safety occurrence will probably never beachieved. It is important, however, to set targets thatrepresent a level of safety performance the communityfinds acceptable, and to measure in an accurate way ourprogress towards them.

The diagram in Figure 1 is one means of depicting thesafety space and where people often operate in relationto the defined boundaries. This diagram depicts theconcept of a known safe and predictable environmentsurrounded by an area of certain failure. It represents a“snap shot,” a moment in time in the operation of, in our

Latent Failure and Human Factors

Richard WhiteCivil Aviation Authority

New Zealand


Control steps1. Decide what is to be controlled

2. Select units to measure it with

3. Choose the desired target standard

4. Devise a way to carry out this measure-ment

5. Carry out the measurement

6. Compare the measured results to targetstandard, and

7. Take steps to adjust actual measuredperformance to target standard

Figure 2.

case, an aircraft. However, the concept applies equallywell to other risk environments. The number of sidescomprising the polygon is dependent on the current taskand environment. They represent the “rules” that applyat a given moment in time. If these “rules” are adequateand complied with, the operation will continue in safety.

It is not so much the shape of the polygon that is impor-tant, or its area, but the fact that beyond its sides lays thearea of increasing likelihood of failure. We believe thatoperators who are able to recognise the shape of the rulesthat surround their operation, and comply with them,are the operators who have the best chance of goodsafety performance.

But life is not black and white—or red and green in thiscase—and we believe that there is an ill-defined areabetween the red and the green where many aviationactivities are conducted. This is of particular concernbecause people learn that it is possible to operate outsidethe rule boundaries without accident or incident, andtherefore believe the rule boundaries to be inappropriate.The problem with this approach is that, because theboundaries of the orange are not defined, activity outsidea green boundary may be successful one day and fail thenext. The unsuspecting person is none the wiser as towhy the failure occurred.

Alongside this concept it is important to accuratelymeasure the level of rule compliance being achieved.

This is only a valid measure if the rules are seen asmeaningful by all the players in the aviation system. In1989, the New Zealand aviation system was reviewed bya consultancy team based around “Swedavia,” a divisionof the Swedish Civil Aviation Authority. Their report the“Swedavia McGregor Report,” concluded, amongst otherthings, that New Zealand’s Civil Aviation Regulationswere hard to understand and were disjointed. Thisreport had a major impact on Civil Aviation in NewZealand, and advocated that a new partnership basedapproach was needed.

One outcome was that we launched a project to com-pletely re-write the rules. This was to be carried out inconsultation with industry and to be harmonised withboth FAA and JAA whenever possible. The importantpoint to consider here is that a partnership was estab-lished between the regulator and all the other industryparticipants. This partnership approach has been veryimportant in establishing a good working environmentthat has flowed through to other initiatives developedfrom the Swedavia review.

In essence a change in culture was under way whereimproved working relationships were developed at alllevels of industry.

Another significant change now adopted into the CivilAviation Act was a move towards placing emphasis onmanagement systems as a means of underpinning safetyrelated behaviour. We have therefore included properlydocumented and functional management systems as arequirement in the new rules. In our view, these systemsrequire the features of good quality management prac-tices and the NZ CAA’s new rules therefore reflect ISO9000 Quality Management practices. We also decided toimplement these in house. In order to ensure that ourown field staff understood the methodology, an inten-sive training programme was put in place. For example,all our Flight, Ops, Maintenance, and other inspectorswere trained to the requirements published by theregistration board for assessors with the Institute ofQuality Assurance in London. Most of our staff passedthe entry criteria and were awarded either assessor orlead assessor status. This was a recognised internationalqualification and gave us confidence that we had acalibrated measuring tool, “The Auditor.” We have sincedeveloped an in house course and all field staff areknown as Safety Auditors. The concept of safety auditwas borne.

Having embraced the quality concept, we then started tochange our own internal process to ensure that this new-found knowledge was put to good use. The Audit Groupdecided to implement safety audits as one method ofmonitoring industry. They chose to use the seven stepsof control as recognised by the principles of qualityassurance. These steps are shown in Figure 2.

We decided that compliance with the new rules wouldbe controlled as one of our principle measures. The other‘target’ would be measured in terms of accidents andincidents. For this to be a realistic measurement weconsulted with various industry groups and gainedagreement to effect a downward trend in the accidentrate. The target trends, shown in Figure 3, are within thebounds of the rates of similar accidents in the UK,Canada, Australia, and the United States.


Actual SafetyPerformance

Required Trend

Targeted Levelfor year 2000

Next Target

Continues toapproach zero

20001996

Accidentsper

100,000FlightHours

Identification ofthis ‘gap’ assistsin prioritising andfocusing safetyprogrammes

Figure 3. Safety Outcome Targets Set for Each Industry Sector

FAULTFOUND

CAUSESREMOVED

CORRECTIVEACTION

PLANNING

CAUSAL FACTORSESTABLISHED

SAFETY CONTROL LOOP

Figure 4. Safety Control Loop

No targets were set for incidents other than to effect acontinuing downward trend in airspace and defectoccurrences. As the new rules are introduced, it isanticipated that better reporting may cause an initialupward trend. Causal Factors are also recorded to allowcomparisons to be met. By capturing this data, togetherwith accurate rate information, the system can yieldstatistical compliance information on any combination ofrecorded parameters. In short, this system allows us toset targets and measure our progress towards thosetargets and know when the targets have been met.

The methods of how we went about achieving this isbeyond the scope of this paper. However, suffice to saythe Aviation Safety Monitoring System (ASMS) wasborn.

The Safety Control System

The design of the Civil Aviation Authority’s system forthe measurement, evaluation, and correction of detecteddeficiencies is based on the concept of a Quality Control(QC) loop. The QC loop is a well known and acceptedquality management tool that is used by organisations tomeasure, control, and improve their processes. TheAuthority is responsible for the measurement, control,and improvement of aviation safety across the entireNew Zealand aviation industry. The QC loop principlecan be used in the Authority’s management of aviationsafety. From the Authority’s perspective, the QC loop(Figure 4) can be termed a safety control loop.

The Authority continuously monitors identified safetyparameters within the Civil Aviation environment.From the proactive perspective, we plan and carry out asafety audit programme to assess the compliance ofcertificated organisations with their own declaredsystems and procedures. Prior to obtaining certification,

the organisation must demonstrate that such systemsand procedures will ensure continuing compliance withall relevant Civil Aviation Rules (and thus remain in the“green” safety space).

From the reactive perspective, we monitor occurrencessuch as accidents, incidents, and defects that are notifiedfrom external sources. When a deficiency is detected bysafety audit, or an occurrence is notified to the Author-ity, the sequence of events initiated are those describedby the safety control loop. In these cases, the causalfactors, or reasons, for the fault need to be determined,corrective action planning undertaken, and the causes ofthe fault removed by implementation of the correctiveaction.

Deficiencies may also be detected by analysis of collec-tive information that has been received and recordedover a period of time. This trend information canindicate causal factors that may not be apparent bysimply looking at individual events. The correction ofthese types of faults follows exactly the same sequenceon the safety control loop.


NON-COMPLIANCE

13

CAUSAL FACTOR

CORRECTIVEACTION REQUEST

OCCURRENCES

TAIC OREXTERNAL

INVESTIGATIONREPORTS

CORRECTIVEACTION

RESPONSES

A/DS EROPS

CO STAT

CAUSAL FACTORSASSIGNED

CATEGORISE RECORD

RESPONSEADDRESS

C.A.R.

CLOSE ACTION

FOLLOW UP

OPEN/CLOSE

NO FURTHERACTION

CLOSEOCCURRENCE

ASSIGN TASK NON-COMPLIANCE

REVIEW

CAUSAL FACTOR

FURTHERCORRECTIVE

ACTION

CORRECTIVEACTION

REQUEST

IMPLEMENTCORRECTIVE

ACTION INITIATECORRECTIVE

ACTION

AUDITORS

ASMS RULES SCOPE OBJECTIVECALENDAR

SCOPE OBJECTIVE

A/DRULESENFORCEMENTENTRY CONDITIONSEDUCATION

02

04

08

07

13

0503

14

13

06

DETERMINEASMS OCC REF.

CLOSE

OPEN

YES

NO

YES

NO

CARRY OUT TASK

SAFETY CONTROL PROCESS:INDIVIDUAL OCCURRENCES

YES

NO

Figure 5.

System Flow Charts

The Safety Control System may be described as twoseparate processes. The first relates to the actions carriedout with respect to individual occurrences, the seconddeals with the collective analysis and actioning of allstored information. These processes are described ontwo flow charts.

Individual Occurrences

As stated, information is being fed into the system fromtwo main sources, safety audit findings, and notifiedoccurrences (Figure 5).

Safety Audit Findings

In the proactive audit process, we carry out a program ofvisits to organisations holding aviation approvals inorder to examine their management systems, equipment,and facilities. On return to the office, the auditor entersthe finding information onto the Aviation Safety Moni-toring System (ASMS) computer database.

Notified Occurrences

Occurrences that are notified to the Authority includeaircraft accidents, airspace incidents, bird incidents,aircraft defects, dangerous goods incidents, security

occurrences, erroneous information occurrences andcomplaints.

Categorise and record (Box 2)

When an occurrence is notified, the first priority is tocategorise the event into the appropriate category and tolog the occurrence on the ASMS database. Once this hasbeen carried out, responsibility for managing the occur-rence is assigned automatically to a Safety Analystaccording to a preset assignment scheme.

Open/Close (Box 3)

The decision to open the occurrence for investigation orto close it for no further action is now made by the SafetyAnalyst.

Assign task (Box 5)

Assuming the occurrence is to be progressed further, theinvestigation task needs to be assigned to appropriatepersonnel and the scope and objective of the task de-fined.

Carry out task (Box 6)

The next step is carrying out the investigation. Thisfunctions on very similar lines to the audit process. The


SAFETY CONTROL PROCESS:COLLECTIVE OCCURRENCES

HOT LIST

NON-COMPLIANCE

CAUSALFACTORS

OCCURRENCES

CORRECTIVEACTION

REQUEST

CONCERNSFROM NON-DATABASESOURCES

DATABASE

AUTOMATICALERT SYSTEM

CORRECTIVEACTION

REQUESTTRACKING &FOLLOW UP

SYSTEM

REPORTS &GRAPHICSSYSTEMS

WEEKLYREVIEW

MONTHLYMANAGERS

ASMS REVIEW

11

10

13

02

14

0911

12

AUDIT PROGRAMME

INVESTIGATION

ENFORCEMENT

RULES

ENTRY CONDITIONS

EDUCATION

AIRWORTHINESS

Figure 6.

cause of each deficiency is identified, and appropriatecorrective actions are initiated. Detailed descriptivefactors, identified deficiencies, causal factors, andcorrective actions are recorded as the investigationprogresses.

Review (Box 7)

At this stage, further corrective actions that may need tobe carried out by the Authority are identified andinitiated as appropriate. If there is to be no furthercorrective action, or corrective action has been initiated,we loop (Box 3) back to the beginning of the loop todecide whether to close the occurrence or carry outfurther investigation.

Corrective Action Responses (Box 14)

As a result of corrective action requests, responses willbe received. These will either be from external clientswho have been subject to audit or investigation, or frominternal sources who have been assigned correctiveactions. These responses must be reviewed to determinewhether the original request has been addressed and, ifso, the action closed off or if not satisfactory some formof further follow-up action taken.

Collective Occurrences

As time goes on, the database amasses a wealth ofinformation. This information can be categorised intofour main areas (Figure 6).

• Non-compliances

• Causal Factors

• Rate of Occurrences

• Corrective Actions

Non-compliances: Automatic Alert System (Box 10 notyet completed)

The information concerning non-compliances is used bya computer-based automatic alert system to generate a“flag” should problems be detected in certain areas. Thesystem runs in a similar way to an aircraft maintenancereliability programme. The system continuously moni-tors individual Rules or predefined groups of Rules forrate of compliance. The information necessary to carryout this monitoring is fed into the system as a result ofaudit or investigation findings that are referencedagainst Rules. Each time an audit or investigation iscarried out, compliance with a set of Rules is either


OrganisationalFactors

For example:

Communications

Management

Structure

Goals

Local Error orViolation Factors

For example:

Morale

Fatigue

Equipment

Procedures

Active Failures

Eg Errors;

Information

Diagnostic

Goal

Strategy...

AND/OR

ComponentsComponentsComponents

ORGANISATIONORGANISATIONORGANISATION TASK/ENVIRONMENTTASK/ENVIRONMENTTASK/ENVIRONMENT INDIVIDUALSINDIVIDUALSINDIVIDUALS DEFENCESDEFENCESDEFENCES

Latent Failures

For exampleStructural/Mechanical/Other

Figure 7.

directly or indirectly tested. The Rules to be tested arespecified on the checklist applicable to the task.

Corrective Action Tracking & Follow-up (Box 14)

Individual corrective action requests entered into thesystem must be tracked to ensure they are controlled andproperly closed-off. When a finding is entered, a correc-tive action date will be specified. When a response isreceived, the action will either be closed or a new datespecified.

This part of the system automatically provides informa-tion on which corrective actions have exceeded thespecified date. This system applies to the tracking of bothexternal and internal corrective actions.

Reports and Graphics (Box 9)

This system provides for the generation of standard andtailored reports and graphics from the informationcontained within the database.

Trend statistics highlight faults that may not be apparentfrom individual occurrence data. They enable the safetyperformance of industry groups, specific clients, andRules to be gauged on demand.

The system also enables the production of reports andgraphics that indicate occurrence and corrective actionclose-out performance.

Monthly Review (Box 11)

The next stage of the system comprises a monthly reviewof the data produced by the systems previously de-scribed.

Corrective Actions (Box 12)

The monthly review group is empowered to plan correc-tive actions appropriate to the overall trend informationit is receiving and to initiate and monitor the implemen-tation of those actions.

Summary

The Safety Control Loop Process is a control mechanismthat not only involves the identification and measure-ment of faults, but also involves the planning andimplementation of corrective actions. When applied tothe Civil Aviation Authority, the loop provides thenecessary dynamic feedback to provide managementwith confidence that safety objectives are being met. Theloop also provides management with a mechanism toensure the continual improvement of safety.

During the development of this system, it becameapparent that identifying causal factors in a structuredway would be an important feature. Initially the ICAOADREP cause codes were used. Unfortunately, thesegave unsatisfactory results and did not really identify thetrue root cause. About this time we became aware ofwork going on by Jim Reason and Willem Wagannar onthe Shell Delta Tripod project. We were also in contactwith David O’Hare, from Otago University, who was


working on cognitive failure analysis for aircraft accidentinvestigation. David and his team had developed a sixstep sequence of information processing. We reviewedall this information, and decided to re-configure thedatabase in the causal factor area to follow what hadnow been termed “The Reason Model.” We also modi-fied the model in the active failure area, and substitutedthe error conditions from David’s work.

Reason Model (modified)

The Reason model, in graphical form, is as shown inFigure 7.

General Failure Types ( GFT )

Typical organisation failure items can be produced forselected organisations or industry groups on demand(Figure 8). These can be generated from the pro-activeaudit information, and from re-active information gainedduring occurrence investigation. We can compare these,and carry out Pareto analysis to determine which indus-try sectors are having particular problems. We can thendevise suitable corrective action programs. Resources aretargeted to the areas of maximum potential safetybenefit, and the results are monitored for an improve-ment in safety performance.

We can also monitor trends in critical occurrences in thesame way. This is particularly important for the largeorganisation. Accidents are fortunately few, and soincidents are a rich source of information for under-

standing problem areas within an organisation. Theseincidents are the free lessons that did not fully penetratethe systems safety defences. In terms of the ‘Polygon ofCertainty,’ they are return trips into the orange space. Bytaking this pro-active approach to identifying the latentfailures that exist in all organisations, these companieswill improve their safety performance. Many learnedcommentators have described how improved safetyperformance aligns with improved financial perfor-mance. As regulators, we see this pro-active abilitywithin the industry environment as essential if we are tohave a major impact on the accident rate.

So what was the New Zealand Civil Aviation Authoritysolution.

To summarise, we created a management informationsystem hosted on an up-to-date database medium. Thissystem is capable of addressing all our processing needsto allow applicants into the aviation system. We issue allaviation documents through this system. We schedule allour safety audits and safety investigations and record theresults so they are “live” on the system. We also analysethe outputs to provide up-to-date trend information toCAA management and industry as appropriate. We aretaking a leading role in helping industry to understandthe benefits of this systemic approach to incident investi-gation. We have held a seminar for the Chief ExecutiveOfficers of our larger organisations where it was agreedthat the new safety environment can only succeed if themessage comes from the top. The more progressive ofour industry organisations are already adopting these

0

50

100

150

200

250

300

350

400

450

500

Figure 8. GFT Profile

Pro

ce

du

res

No

tFo

llow

ed

Ina

de

qu

ate

Co

ntr

ol

& M

on

itorin

g

De

sig

n S

yste

mD

efic

ien

cie

s

Ris

k M

isp

erc

ep

tion

Po

or

Pla

nn

ing

Ina

ed

qu

ate

Ch

ec

kkin

g

Po

or

Pro

ce

du

res

Inte

rpre

tatio

nD

iffic

ulti

es

Po

or

Tra

inin

g


new policies and are able to identify root causes forthemselves. This will allow them to carry out their ownPareto analysis, and put in place the necessary correctiveactions.

We have written the new rules to encompass the use ofmanagement systems, including the requirements to putin place procedures for identifying causal factors stem-ming from internal audits and safety investigations intocompany incidents. We have not mandated whichmethod of causal factor identification methodologiesshould be used (there are a number available). However,we do recommend that the methods promulgated byICAO based on “Reason” are one acceptable method ofcompliance.

Present and Future Development

Generally the samples that a regulator can take provideonly a snapshot of the state of the aviation system at anyone time. Safety is more likely to be achieved throughmaking individuals and organisations operating theCivil Aviation system more accountable for their safetyperformance. Traditionally, aviation authorities havecarried out the inspection function, and recorded theresults of those inspections in paper files. The individualinspectors were given responsibility for sectors of theindustry, and functioned as a form of quality control.The time has now come to gather and make use of theinformation in a much more systematic way. The majorauthorities around the world have already recognisedthis, and are currently enhancing their computer systemsto take advantage of recent software developments.Database management systems such as Microsoft “Ac-cess” are readily available.

The United States, through NASA, has ASRS. FAA isdeveloping SPAS. UK CAA is upgrading the old SDAUdatabase and CHIRPS. Canada has ASIS. The AustralianBASI has OASIS. Boeing has MEDA. The Nordic coun-tries have AIRCRAMS.

The airlines are also putting together very effectivesystems based on current academic thinking and flightdata parameter analysis. In the United States, the flightoperations quality assurance program (FOQA) is beingdeveloped. American Airlines has developed ASAP,which may become the prototype for other airlines in theUnited States. In Europe, British Airways has developedBASIS. to which many international agencies subscribe.In Australia, Qantas has also developed similar monitor-ing systems.

The above are just some examples of the work going onat the moment, largely in isolation. If we don’t dosomething soon, we will run out of acronyms. It is ourview that the regulatory authorities around the worldshould start to cooperate in putting together one com-mon system. It is hard to pin down why there is not

more cooperation between organisations developingthese systems. Certainly industry has recognised theneed to share data on a global scale, and at the Seattleconference it was mooted that the Flight Safety Founda-tion might be an appropriate organisation to coordinatethis.

It may be that the lack of cooperation in building thesesystems is simply that different organisations havedifferent starting points in the cycle and are just develop-ing existing processes. Recent work by FAA and JAA onharmonising the existing rules and regulations hasshown that there is a willingness to co-operate betweendifferent regulatory agencies. Perhaps the time is right toconsider building a common database that could be usedby all. The various bilateral agreements that are inexistence indicate that the guidelines found in the ICAOannexes have produced essentially similar regulatorysystems.

It is our view that the regulatory authorities around theworld should start to cooperate (perhaps through ICAO)in building a common database system.

For our part, we would be happy to contribute to thiseffort by making available information on the NewZealand CAA system. We would also be happy tosupport any effort to bring together the people respon-sible for database development and safety analysisaround the world to discuss just what is possible. Weknow from our own experience that these systems thatcan gather both pro-active and re-active information takea long time to develop. However, without the benefit of arelational database such as ours, it would be verydifficult to measure the improvements that need to takeplace.

If we are to make an impact on the expected rise in thenumber of accidents, then the academic principles andpractical solutions put forward here today must be oneof the most powerful tools available to reduce the latentfailures present within our aviation system.

Aviation Safety is EVERYONE’S Responsibility

References

ICAO (1993) Human Factors Digest No. 10 Circular 247-AN/148 - Montreal Canada.

ICAO (1995) Human Factors Digest No. 12 Circular 253-AN/151 - Montreal Canada.

Reason, James. (1991). “Identifying the Latent Causes ofAircraft Accidents Before and After the Event.” ISASI22nd Annual Seminar Papers, Canberra Australia.


Wagenaar, Willem A. (1995). Chairman and ChiefExecutives Safety Seminar Papers Civil Aviation Author-ity of New Zealand, Wellington, New Zealand.

Wright, Colin. (1995). British Airways Safety InformationSystem (BASIS) Chairman and Chief Executives SafetySeminar Papers Civil Aviation Authority of NewZealand - Wellington New Zealand.

O’Hare, David, Mark Wiggins, Richard Batt, and DianneMorrison. (1991). “Cognitive Failure Analysis for AircraftAccident Investigation.” University of Otago, NewZealand.

White, Richard and Ian Patterson. (1995). “Joint PapersInternational Symposium on Aviation Human Factors.”The University of Auckland, New Zealand.

Butler, Garry and Richard White. (1992). “The AviationSafety Monitoring System (ASMS).” Internal Papers.Civil Aviation Authority of New Zealand, Wellington,New Zealand.

Nalder, Peter, (1993), “The Polygon of Certainty.”Internal Papers. Civil Aviation Authority of NewZealand, Wellington, New Zealand.

ISASI (1995) Forum: Proceedings of the 26th InternationalSeminar of the International Society of Air Safety Investiga-tors. Seattle, Washington, United States.

FSF, IFA and IATA (1995) Forum: Proceedings of the JointMeeting Managing Safety. Seattle, Washington, UnitedStates.


Introduction

The Boeing Commercial Airplane Group’s (BCAG) goalto enhance aviation safety is the “industry workingtogether to eliminate accidents, through continuousimprovements in air traffic systems, airline operationsand airplane designs.” The Airplane Safety AwarenessProcess (ASAP) is just one of the many processes andprojects Boeing has implemented to continuouslyimprove the already excellent aviation safety record.

ASAP…In the Beginning

In 1992, a team of Boeing senior managers was formed todesign a proactive safety process that could identifypotential safety concerns before they became seriousevents. The team was tasked with developing a reliableprocess for gathering relevant data, identifying potentialhazards, assessing the risk, and prioritizing the need foraction of potential airplane safety issues. The processwas also to include steps to maintain the necessarymanagement visibility and responsibility. By early 1993,the straw-horse process they designed became theAirplane Safety Awareness Process (ASAP).

The ASAP Process Today

The original straw-horse process designed by the SeniorManagement team has undergone several changes andimprovements since its implementation in 1993. Theprocess described here details how ASAP works today.

There are four main parts to ASAP. These include howitems are identified, assessing the safety impact of theidentified items, corrective action, and closure.

Identifying Potential Issues & Entering ASAP

There are several methods currently used to identifypotential safety for entry into ASAP:

The first method of entry into ASAP is the PotentialAirplane Safety Concerns (PASC) reporting process. Tocontinuously improve our products, we need feedbackfrom the people that design, build, inspect, and supportour airplanes. Any BCAG employee may submit aconcern through the “Potential Airplane Safety Concern”

(PASC) reporting system. PASC forms are found ondisplay boards throughout all BCAG facilities. PASCforms are filled out and submitted to the Airplane SafetyEngineering organization, and are initially screened toremove occupational safety items and obvious non-safety items. The removed items are closed out withdocumented reasoning for non-inclusion into ASAP.PASC forms that have not been screened out are thendistributed to a focal within the Safety Engineeringorganization who performs an initial engineering analy-sis to determine if the item should be entered into ASAPfor further safety review. All inputs are treated withconfidentiality, and, if desired, can be submitted anony-mously.

Another method used for entering potential safety itemsinto ASAP is the Service Related Problem (SRP) process.The SRP Process is an existing BCAG process adminis-tered by the Service Engineering organization, anddesigned to resolve in-service problems. Service Engi-neering is responsible for reviewing the many telexesreceived from Boeing Field Representatives throughoutthe world. If the event described in a telex meets theSRP’s documented safety criteria, then an SRP Accep-tance Board is held. Should the Acceptance Board acceptthe SRP as a potential safety item, the Acceptance Boardthen directs the potential safety issue to the ASAPprocess for confirmation of the SRP safety determination.A manager and technical focals are assigned to each SRPitem to coordinate and manage the item toward resolu-tion. The SRP Acceptance Board can also provide direc-tion for any interim action, and coordination withregulatory agencies and Air Transport Association(ATA) Lead Airline involvement, if necessary. Thepotential safety issue continues to be worked, towardcorrective action, within the SRP process. If a potentialsafety issue enters the ASAP process through somemethod other than the SRP Acceptance Board process,then Safety Engineering can request, through the ChiefProject Engineer, an SRPC Acceptance Board. Thisinitiates interim corrective action and coordination withcertification while awaiting a Safety Review Boardreview.

Yet another entry method is through Boeing organiza-tional safety teams. There are several organizationswhich have existing Safety Steering Teams implemented

The Airplane Safety Awareness Process

Sarah C. WilliamsAirplane Safety Engineering

Boeing Commercial Airplane GroupSeattle, Washington


to review safety items within their own organizations.Each of these teams acts as a screening process forpotential safety issues. Items are reviewed by the teamsand passed on to ASAP directly. The teams provide atechnical focal for each of their issues.

In addition to the above methods, any member of aSafety Review Board or any BCAG senior manager cansend a potential safety item directly to ASAP without aSafety Engineering review. There are some other lesser-used methods by which a potential safety item can enterASAP, but the ones described here are the major meth-ods currently used.

After an item has made it through one of the methodslisted above, it is entered into the ASAP database,assigned focals from the responsible technical organiza-tion and Safety Engineering, and the issue is reviewed toidentify the Boeing models affected. The assigned focalsare responsible for coordinating cross-program issuesamong all affected models and systems. Finally, eachitem is prepared for review by the appropriate SafetyReview Board.

Assessing the Safety Impact

The Safety Review Board (SRB) is the central part ofASAP and is responsible for assessing the safety impactof each potential safety item entered into the process.There are three different types of Safety Review Boards;the Program Safety Review Board, the Cross-ProgramSRB, and the BCAG SRB.

The 747/767, 737/757, and 777 Program Safety ReviewBoards are individual boards that review only ASAPitems affecting the models associated within eachprogram. The members of a Program Safety ReviewBoard include the following:

• Program Director of Engineering• Program Model Chief Project Engineers• Program Director of Service Engineering• Program Model Chief Pilots• Air Safety Investigation• Customer Training & Flight Operations Support• Program Safety Engineering• Chief Engineer - Safety & Airplane Perf. (facilita-

tor)• Chief Design Engineer for Specific ASAP Issues

A Cross-Program Safety Review Board is the same as aProgram Safety Review Board with the exception that itis held for ASAP items where models from more thanone Program are affected. The goal of a Cross-ProgramSafety Review Board is to coordinate actions and resultsbetween the programs. The board members for eachindividual Program Board are members of the Cross-Program Safety Review Board.

The purpose of the Program and Cross-Program SafetyReview Boards is to review each ASAP item, decidewhether or not the item is a safety issue, provide di-rection for corrective action to resolve the issue, andreview and approve the final disposition of an ASAPissue.

The BCAG Safety Review Board is a high level boardcomposed of senior management from BCAG. TheBCAG Board is the process owner of ASAP, and isresponsible for the process’ development and implemen-tation. The BCAG Safety Review Board’s task is todiscuss and determine BCAG safety philosophies, aswell as to work toward Boeing’s goal of enhancingaviation safety.

Each Program and Cross-Program Safety Review Boardoperates under a few basic ground rules. Safety ReviewBoard representation is mandatory at all Safety ReviewBoards. If a Safety Review Board member is unable toattend a board, he/she is responsible for designating analternate with authority to make safety determinations.Every member must be represented in order for a SafetyReview Board to be held. Another ground rule is thatdetermination that an issue is not a safety problem willbe reached by a consensus of the board members. Thismeans that it only takes one Safety Review Board mem-ber to indicate that an item is a safety issue for it to bedesignated as a safety issue. There must also be consis-tent results between Boeing models. If results are notconsistent, technical justification for the difference mustbe documented. Any item where there are not consistentresults, or where there is no technical justificationbetween the models, must be reviewed by the BCAGSafety Review Board. The BCAG Safety Review Boardwill determine necessary actions to resolve the differ-ences. One other ground rule is that any Safety ReviewBoard member may call a special board meeting at anytime.

Each ASAP item is presented to the Safety Review Boardby the technical focal. Each presentation is focusedtoward helping the board make a safety determinationfor the particular potential concern. Presentationsinclude expectations of the Safety Review Board for thespecific issue being addressed, and a brief statement ofthe issue or the concern. The board also needs to beaware of the airplane level hazard or concern associatedwith the ASAP item. The presentation also addresses theapplicable airplane models affected by the issue, andwhether the airplanes affected are in commercial serviceor in manufacturing production. Presentations alsoinclude a thorough historical study, detailing the numberof times the event has happened, and other pertinenthistorical information. The presentation is required tohave a risk or safety analysis addressing the concern. Thesafety analysis documents any assumptions or otherinformation used in the analysis. Finally, the presenta-


tion addresses any actions that have been taken and anyfuture planned actions.

After the presentation, the Safety Review Board performsseveral tasks. The board must first determine if a safetydetermination can be made with the information pro-vided in the presentation. If a safety determinationcannot be made, the board can assign action items to theresponsible focals and reschedule the item for re-reviewat a later date. If enough information is available, thenthe board makes its safety determination. Safety determi-nations are made by the board using a process that firstlooks at whether an item meets a set of safety guidelines.These guidelines are used only as a tool to help the boardmake a determination. Other considerations in makingthe safety determination are whether the hazard hasbeen mitigated, and whether the risk outlined in thesafety analysis is acceptable. Together, all these factorscontribute to the board’s safety determination of anASAP item.

Other duties the Safety Review Board must accomplishafter the presentation include making a determination ofthe Service Bulletin type (ALERT, Significant, or Stan-dard Bulletin) and capturing any lessons learned foundin reviewing and resolving the ASAP item. All thesetasks may not be accomplished in one Safety ReviewBoard meeting, requiring the item to be brought back toa future Safety Review Board in order to accomplish allthe board’s tasks. All decisions and determinations madeby a Safety Review Board are documented in the ASAPdatabase, and copies of presentations and other pertinentmaterial are retained in ASAP files.

Corrective Action

Once the Safety Review Board has made its safetydetermination, issues enter the corrective action phase ofthe ASAP process. ASAP, however, does not directlyhandle the corrective action of each issue. Correctiveaction for ASAP issues is the responsibility of the ServiceRelated Problem (SRP) process owned by the CustomerServices organization. During ASAP’s design, CustomerService personnel were involved in the coordinationefforts to assure implementation of effective and reliablemethods for passing ASAP items through the SRPprocess.

After a safety determination has been made, the boardcan provide direction for any corrective action deemednecessary. If an item presented already has an associatedsafety SRP and is determined to be a safety issue, theboard provides direction through the existing SRPprocess. The SRP team assigned to each issue is thenresponsible for determining the root cause and managingthe resolution of the issue within the SRP process. TheSRP team must, however, report back to the SafetyReview Board for confirmation that the resolution of theissue has adequately addressed the safety concern.

Should an item presented NOT have an associated SRPand is determined to be safety issue, the Safety ReviewBoard has authority to accept a safety SRP. This author-ity reduces the duplication of holding a SRP AcceptanceBoard. When accepting an SRP, the Safety Review Boardperforms the same tasks as the SRP Acceptance Board,including selecting an SRP manager, directing interimaction, and determining regulatory agency notificationrequirements. Again, the SRP team must report theirresolution back to the Safety Review Board for review.

Finally, if an ASAP item presented has an associatedsafety SRP and is determined NOT to be a safety issue,the board has authority to downgrade the SRP categoryor close the SRP. The board can still recommend correc-tive action through the SRP despite a non-safety vote.

Closure of ASAP Items

Once all the Safety Review Board tasks and correctiveactions have been accomplished, the ASAP item is readyfor closure. Every ASAP item is required to be closed outwith a written and signed closure document.

Before closure may begin on an ASAP item, the follow-ing items must be accomplished. First, any Safety ReviewBoard action items or corrective actions must be accom-plished and reviewed by the Safety Review Board ifnecessary. Next, any Service Related Problems (SRP)associated with an ASAP item must be closed out, and,finally, any engineering design changes or ServiceBulletins must be released. Once an ASAP item meets theapplicable criteria, it is ready to start closure. The respon-sibility for completing closure documentation rests withthe safety engineer and the technical focal.

As stated before, every ASAP item has a closure docu-ment that captures all the pertinent information as-sociated with the issue. This document is designed torecord the history of the issue’s progress through theASAP process and all the determinations made and thereasons for each decision. The closure also captures allother associated documentation and information re-garding the issue. Each closure form contains the fol-lowing information: First, a description of the issue orconcern, all appropriate Safety Review Board reviewdates, and the safety determinations made for the issue.For non-safety issues, the reasons or technical justifica-tion for a non-safety determination are documented.Closure documentation also contains any correctiveaction or appropriate action that has already beenaccomplished. This should include document identifica-tion, what the action accomplished, why it is believedthis action addresses the concern, and the model effectiv-ity of the action (i.e., models, in-service or productionairplanes, etc.). The closure also documents any design,manufacturing, training, or operational lessons learnedthrough the process.


After all this information is captured in the closuredocument, the document is reviewed by the SafetyReview Board and signed by board members uponacceptance. Each closure is then filed, along with all theother information associated with the ASAP item. Thedatabase and files provide a complete record of safetyissues and the results of their identification and reviewfor future use.

What Have We Learned

Throughout ASAP’s development and implementation,there have been many opportunities to learn from boththe successes and not-so-successful events associatedwith ASAP. A few of the lessons learned are describedbelow.

One of the most important things learned in developingand implementing this process is the need for seniormanagement understanding and approval of the process.Senior management must take ownership of the processand participate in the Safety Review Boards. Middlemanagement, although not directly participating on theSafety Review Boards, must be educated in the processand its importance. Engineers and technical personnelmust also be educated in the process.

The development and implementation of a new processis bound to affect any existing processes. Coordinationwith all the affected processes within all programs isimportant to developing a complete process. Thiscoordination must also be accomplished early in theimplementation to avoid confusion and deliberate work-arounds of the process.

Finally, the Safety Review Boards must focus on thesafety determination of the issue presented, and not onresolving the issue. A strong description of the re-quirements and expectations of the presentations to the

Safety Review Board, as well as self-disciplined boards,is needed to maintain focus on the safety determination.

Conclusion

The three years of development, implementation, andoperation of the Airplane Safety Awareness Process havebeen challenging, and the process has gone throughmany changes and struggles. As in any changing envi-ronment, there will always be future opportunities toimprove the process, making it better and more efficient.Despite the numerous challenges and revisions, theprocess works well and has been very successful. As ofJuly 1, 1996, over 190 potential safety items have beenidentified and worked through the process. BCAGbelieves that ASAP has positively contributed to its goalof improving airplane safety by reducing the number ofaccidents, and is in keeping with our unwaveringcommitment to the safety of our products and the airtransport system.

Sarah C. Williams is an Airplane Safety Engineer with theBoeing Company’s Airplane Safety Engineering organization.Sarah was the Process Manager for the Airplane SafetyAwareness Process (ASAP), and was involved in ASAP’sdevelopment and implementation for the last three years.Sarah is currently working on the 747-500X/-600X program.She was the Technical Committee Co-Chairman for the 1995ISASI Seminar in Seattle, WA, and is the current NewsletterEditor for PNRC of ISASI. She graduated from The GeorgiaInstitute of Technology in 1991 with a Bachelors of Science inMechanical Engineering, and a Minor in Composite MaterialsEngineering. She obtained a Master’s of Aeronautical Sciencein 1994 from Embry-Riddle Aeronautical University atMcChord AFB’s branch campus. She has worked for theBoeing Company for five years. EMAIL:[email protected]

beru.univ-brest.frberu.univ-brest.fr/~singhoff/doc/papier_a_trier/safety_proceed1.pdf1 1996 seminar...

Documents