1JHSAT - June 2006

JHSAT Status Briefing to IHSTJune 29, 2006

Mark Liptak FAA ANE-110Jack Drake HAI

2JHSAT - June 2006

JHSAT Status Briefing

Objective:

Brief the IHST on the status of the JHSAT

Obtain IHST concurrence on key program elements, avoid surprises or questions downstream:

dataset selectionadequacy of team membershipscoring criteriageneral program timingoutreach approachuse of existing safety reports

3JHSAT - June 2006

JHSAT Status BriefingSlide Topics:

SynopsisCharterWeb Site Team CompositionDataset SelectionProcess Flow - Timing/ScheduleJHSAT Scoring CriteriaEstablishing Standard Problem StatementsScoring MethodInternational PartnersLeveraging Existing Safety ReportsIssues for IHST Awareness/Concurrence

4JHSAT - June 2006

Synopsis

The JHSAT team is fairly well solidified, 14 members, making progress on adapting the CAST/JSAT process for analysis of helicopter events. We’re off to a good start.

Team members are enthusiastic about the basic CAST process, believing it will produce high value results. Have engaged in many detailed discussions on dataset selection, event scoring, data quality, mission diversity, etc.

Recent meetings spent evolving and detailing the JHSAT process and analyzing helicopter accident event data. We have made significant progress, but its been more time consuming than expected.Looking for ways to boost efficiency.

Industry is stretched thin, IHST should consider reiterating it’sappreciation for the commitment needed to make this happen.

The following slides contain information on key JHSAT topics.

5JHSAT - June 2006

Charter

Final charter developed and agreed to by team, posted on JHSAT web site:

Goal: Provide a prioritized assessment of the most safety critical hazards to commercial, private and military rotorcraft in worldwide operations as derived from selected rotorcraft data sources.

Provide intervention strategies to the IHST and Joint Helicopter Safety Implementation Team (JHSIT) that maximize the likelihood of reducing worldwide helicopter accident rates by 80 percent by 2016.

Provide a JHSAT report documenting the team’s findings to the IHST by 1Q 2007.

6JHSAT - June 2006

Web Site

The JHSAT has established a web site at http://www.ihst.org/jhsat/jhsat_top.htmId=jhsat, password=helicoptersafe

This is an access controlled site, please limit to the IHST only.

The JHSAT is using it to post NTSB data, team analysis results, meeting minutes, agendas, action lists, etc.

7JHSAT - June 2006

JHSAT Team Composition

Mark Liptak (FAA ANE) JHSAT co-chair Jack Drake (HAI) JHSAT co-chair Barry Rohm (Rolls Royce) Ray Wall (Bristow)Roy Fox (Bell) Tony Alfalla (Sikorsky) Ed Stockhausen (Airmethods) Joe Syslo (Eurocopter)Laura Iseler (IHST) Sandy Hart (NASA)Clark Davenport (FAA ASW) Matt Rigsby (FAA ASW)Ann Azevedo (FAA Risk consultant) Steve Gleason (Schweizer)

JHSAT team membership is light on small helicopter experts. We have discussed participation with Robinson, they do not appear too enthusiastic. Schweizer has committed only as a part time participant. Industry is stretched thin and long term commitment is difficult. We could use additional support from the pilot/operational side and perhaps a training expert.

8JHSAT - June 2006

JHSAT Dataset Selection

The initial dataset selected for JHSAT analysis is year 2000 NTSBaccidents. This will serve as the basis for the 1Q 07 report out.

The JHSAT will conduct a detailed analysis of these accidents. There are approximately 190 accidents to be analyzed.

This NTSB dataset is nearly fully populated with final narrative causal/factual information. Also, less likely to encounter litigation constraints with this dataset.

Consideration of accident causal factor trends for the last 24 years gives the team high confidence that detailed analysisof year 2000 events will yield high value recommendations that will mitigate long standing trends (see next slide)

Future JHSAT activity will target year 2001, 2002, etc.

9JHSAT - June 2006

24 Years - Little Change

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Aircraft Powerplant Auxilary Equipment Human Performance Environment / Facilities Unknown

24 year look back – Relative stability in helicopter accident causal factors

Final causaldata not yet fullydeveloped

US dataset represents 50% of worldwide fleet

10JHSAT - June 2006

JHSAT Basic Process Flowchart

CharterDevelopment

EstablishTeam

Select Data Set

ReviewNTSB

Docket Data

DevelopEvent

Sequence

IdentifyProblems

(what/why)

Assign StdProblem

Statements

ScoreProblem

Validity &Importance

Identify InterventionStrategies

ScoreInterventionAbility/Usage

Prioritize byOverall

Effectiveness

TechnicalReview

&Expert

Validation

Conflicts? ReportResults

No

Yes

IHST

JHSIT

11JHSAT - June 2006

JHSAT Process

JHSAT is structured to be similar to the CAST/JSAT processwhich has been highly successful in the part 121 community

Basic tenets:

Engaging stakeholders/experts from the helicopter community

Findings based on real world helicopter accident data

Recommendations ranked by a structured scoring method

JHSAT is sticking to these basic tenets but making significant modifications elsewhere in the process. Our last two meetings have centered around completing foundational process work and processing the event data.

12JHSAT - June 2006

JHSAT Scoring CriteriaValidity (V) - How valid do we think the Problem Statement is in this accident based on the available accident information?

Range:Guess = 0 Absolutely Sure = 4

Importance (I)- How important is this particular Problem Statement in the causal chain in this accident?

Range:No importance = 0Cause of accident = 4

Ability (A) - How effective is this particular intervention in mitigating the problem statement of this accident?

Range:No effect on Problem Statement = 0 Would Prevent Problem Statement = 4

Usage (U) - How sure are we that this intervention will be utilized as expected in the real world.

Range:No Utilization = 0100% Utilization = 4

Applicability (App) – likelihood that the problem statement will be present in future aircraft/operations

Range:No Utilization = 0100% Utilization = 7

Importance (I) - How important is this particular Problem Statement in the causal chain in this accident?

Range:No importance = 0Cause of accident = 7

Ability (A) - How effective is this particular intervention in mitigating the cause or contributing factors of this accident?

Range:No effect on Problem Statement = 0 Would Prevent Problem Statement = 7

Confidence (C) - How strongly do we believe the intervention will perform as expected in the real world?

Range:No Confidence = 0 Complete Confidence = 7

JSAT Scoring Criteria

13JHSAT - June 2006

JHSAT Scoring Criteria

The scoring criteria used by the JHSAT are similar to those used in CAST/JSAT with the following exceptions:

JSAT’s Applicability was a forward look at fleet characteristics and assessing intervention effectiveness. JHSAT has eliminated this criterion due to the belief that future fleet mix and mission will not change enough to make this a meaningful issue to consider.

JHSAT has added a Validity criterion. This is a scoring factor based on the spectrum of documentation of helicopter accidents. Some are very well documented, others very poorly.

The Confidence and Usage criteria are similar, however some of the detailed definition language is still being worked.

14JHSAT - June 2006

Determination of Standard Problem Statements (SPS)

A key element to the JSAT process is establishing standard problem statements (SPS) when identifying specific causal factors in accidents. Correctly executed, this allows for identifying high value common threads between accidents, and thus identifies high value mitigation areas.

JHSAT is developing a standard problem statement list from three sources:

To jumpstart our process we drew from the SPS developed by the HAAT team(1). We also pulled SPS statements determined to be applicable from the CAST SPS list (~500 topics) These two sources gave us a starting point. As we analyze events we are continuing to populate our SPS list with unique helicopter problem areas as we encounter them.

(1) Helicopter Accident Analysis Team, NASA, circa 1997, early version of the Boeing APS/CAST JSAT process.

15JHSAT - June 2006

Scoring Method

The JSAT process developed and used an arithmetic relationship to rank recommendations by an Overall Effectiveness (OE) rating. Using the scoring criteria from the previous slide, the relationship is as follows:

OE = ((I x A) x 2/(I +A)) x C/6 x App/6

This relationship places weighting factors on the Importance and Applicability scoring criteria.

The JHSAT has not yet made a determination on whether a weighted arithmetic relationship is appropriate. We have intentionally constructed our scoring criteria such that we will have flexibility later in the process to use a variety of arithmetic relationships to help identify the most effective recommendations.

16JHSAT - June 2006

USDataset Selection

JSAT ProcessUnderstanding

Refining

JHSAT Analysis

IdentifyFCAA &MilitaryPartners

Indoctrinate in JHSATProcess

ConductJHSATbased

analyses

Expert Review/

Validation

MitigationRecs - Global

MitigationRecs

FCAA & Mil

Coordinating JHSAT US and International Efforts

First SetUS Mitigation

Recs

Mar/Apr 06May/June 06

May-Dec 06Jan 07

1Q 07

4Q 07

TBD 07TBD 07

TBD 06/07Sept 06

TechReview

Jan 07

17JHSAT - June 2006

US CivilDataset

Selection (1/2 of

helicoptersWorldwide)

JSAT ProcessUnderstanding

Refining

JHSAT Analysis

IdentifyFCAA &MilitaryPartners

Train FCAA &

Military Partners in JHSATProcess

FCAA & Military Partners conductJHSAT

analyses on their

accident data

Expert Review/

Validation

MitigationRecs - Global

MitigationRecs

FCAA & Mil

Coordinating JHSAT US and International Efforts

First SetUS Mitigation

Recs

Mar/Apr 06May/June 06

May-Dec 06Jan 07

1Q 07

4Q 07

TBD 07

TBD 07TBD 06/07

Sept 06

TechReview

Jan 07

IHST

18JHSAT - June 2006

Coordinating JHSAT US, International and Military Sources

Matt Rigsby from FAA-ASW will be acting as the contact point for identifying overseas and military partners.

To date we have interest from:

Canada – may use Canada as a test case, good dataset, strong interest, close proximity

EASA, UK, Australia, Chile, US Navy

Looking into:Brazil, Japan

Any other entities that possess large helicopter accident datasets that could be processed by the JHSAT method will be considered.

19JHSAT - June 2006

Leveraging Existing Safety Reports

JHSAT is reviewing the recommendations of the following reports:

NASA - U.S. Civil Rotorcraft Accidents, 1963 Through 1997NASA - Analysis of US Civil Rotorcraft Accidents from 1990 to 1996 and Implications for a Safety Program NASA - ASRS Rotorcraft Incident Study - Draft Data Summary Aviation Safety Reporting SystemNASA - Helicopter Accident Analysis TeamAMPA - A Safety Review and Risk Assessment in Air Medical TransportCRS - Report for Congress - Military Aviation SafetyOGP - Safety Performance of Helicopter Operations in the Oil and Gas Industry - 2000 DataTSB Canada - Lessons Learned from TSB Investigation of Helicopter Accidents (1994 - 2003)Bell Textron - History of Helicopter SafetyOther NTSB and international reports may be considered.

Approach: Identify recommendations that call for systemic change via oversight, policy, SOP development, investment areas, etc. Examples are changes to safety culture across the industry, adopting SMS approaches, foundational work for universal adoption of a new technology. Recommendations calling for specific design changes, specific training topics, specific changes to specific platforms or operational missions, etc, will not be considered at this point. We will allow the JHSAT process and NTSB data to guide us on the latter type of recommendation.

20JHSAT - June 2006

Issues for IHST Awareness/Concurrence

Dataset selection will be NTSB year 2000 accidents.

Team membership is light on small helicopter pilots/ops, may need help from IHST to push for renewed commitment from some industry members.

Scoring criteria adapted from that used in the CAST/JSAT.

General JHSAT program timing (slide 15)

Early work underway to identify international partners, will teach them the JHSAT process and have them analyze their own datasets.

Reviewing existing safety reports for systemic change recommendations, if any identified they will be passed to the IHST/JHSIT this summer/fall.