Design, Development and Delivery of Safety Management System Training to

Transport Canada: Six Years and Counting

David Wilson, Stantec (formerly Jacques Whitford)

Roger Barker, ISR

2

Outline

• The Mandate

• The Plan

• Course Design and Development– Transport Canada’s Start Point– Inclusion of International and Audit Content

• Course Delivery– Institutional Challenges– Regional Challenges

• Internal vs. External Reaction

3

The Mandate

“The broad objectives of the SMS course are:

• For Delegated Officers (DOs) and Management to develop a detailed understanding of SMS, thereby instilling confidence and obtaining buy-in to TC’s SMS program

• For DOs to develop the skills necessary to execute their mandates in the assessment of SMS”

4

Aviation mission and vision

To develop and administer policies and regulations for the safest civil aviation system for

Canada and Canadians using a systems approach to managing risks

– Continuous improvement – High level of public confidence

Flight 2005: A Civil Aviation Framework for Canada

5

0.0

0.5

1.0

1.5

2.0

2.5

3.0

Safe Environment

Safety Awareness Safe Person in Safe Environment

Supervision Driven

Team Driven

Behavior Driven

INC

IDE

NT

RA

TE

Command & Control Programs Management Systems

Evolution of Safety Management

6

Towards Zero...

…is the condition to which risks are managed to acceptable levels.

WorkplaceSafetyStandards

7

TC Requirements

• “Aviation organizations should establish and maintain a safety management system that conforms to all of the requirements of applicable Canadian Aviation Regulations (CARs) and Standards”

8

TC Requirements

There are 9 parts to the CARs:1. General Provisions2. Aircraft Identification and Registration and Operation of a

Leased Aircraft by a NON-registered owner3. Aerodromes and Airports4. Personnel Licensing and Training5. Airworthiness

– Aircraft Certification (AC) – Maintenance & Manufacturing (M&M)

6. General Operating and Flight Rules7. Commercial Air Services8. Air Navigation Services9. Repeals and Coming into Force

9

The Plan

• Phase 1: Analysis– Deliverable 1: Training Needs Analysis, Learning

Strategy, Final Work Plan and Background Update• Phase 2: Evaluation and Design for CBA and AMM

– Deliverable 2: Course Syllabus, Course Materials• Phase 3: Pilot Course

– Deliverable 3: Pilot Course, Course Evaluation• Phase 4: Course Delivery to CBA and AMM

– Deliverable 4: CBA Course, AMM Course

10

The Plan cont’d

• Phase 5: Validation of Course Delivery to CBA and AMM– Deliverable 5: Confirmed Assessment Protocol, Course

Validation• Optional Phase 6: Adaptation for Other Functional Branches

– Deliverable 6: Adapted Course(s)• Optional Phase 7: Course Delivery to Other Functional

Branches– Deliverable 7: Other Functional Branch Course(s)

• Optional Phase 8: Validation of Course Delivery to Other Functional Branches– Deliverable 8: Course Validation(s)

11

Course Design

• Definition of the target population

• Identification of the SMS knowledge level of target audience

• Review of CARs and notices of proposed amendments (NPAs) to CARs relating to SMS

• Review all applicable national and international (e.g., UK, Australia and GAIN) aviation SMS policies and guidelines

• Assessment of the existing safety culture within the target audience

Safety Metrics Assessment

• Survey-based tool from sub-consultant

• Distributed internally to approx. 1000 DOs: 150 replies

• Intent was to identify:– Internal issues/barriers within TC– Issues/barriers external to TC (customer,

industry, etc.)

12

Training Needs Assessment

• Front line staff were fully aware of their needs and challenges– Different measures are required to measure safety and

SMS;– The interdependencies of attitude, behaviour, and

motivation cannot be ignored– Inspectors need practical demonstration of a functioning

SMS from the ground up,– If support, guidance, and structural leadership from

management are not forthcoming (as happened with Risk Management training), then SMS training will fail.

13

Global Review of TC SMS • Global references:

– OHSAS18001

– GAIN

– CASA Australia - Guide to Aviation SMS

– SMS for Commercial Air Transport Operations - Guide to Implementation (UK)

– OGP Checklist for an audit of safety management

• Resulting additions to the TC model:– Continuous Improvement

– Feedback

– Annual Management Review of the SMS

– Contractors must meet safety standards which do not impinge on the SMS of the organisation

– Inter and Intra-company Safety Performance Comparison

14

15

The TC SMS Model

1. Safety Management Plan• Safety Policy

• Non-punitive Safety Reporting Policy• Roles, Responsibilities & Employee Involvement

• Communication• Safety Planning, Objectives & Goals

• Performance Measurement• Management Review

2. Document Management• Identification & Maintenance of Applicable Regulations

• SMS Documentation• Records Management

6. Emergency Preparedness

5. Quality Assurance

4. Training

3. Safety Oversight• Reactive Processes • Proactive Processes

• Investigation and Analysis• Risk Management

Course Development

• Addressing Gaps:– Audit Approach– Approach to Assessing SMS: Assessors vs.

Inspectors– The TC SMS Model vs. Plan-Do-Check-Act

16

Course Delivery

• Initial 4-day course (‘03-’04)– 19 deliveries

• V2: 4-day course (’04-’05)– 18 deliveries

• V3: 3-day course (’05-’09)– 15 deliveries (eLearning course starts in fall ‘09)

• 2-day awareness course

• 2-day multi-model course

• ½ day refresher course

17

3-day Course Content• History of SMS• Introduction to TC SMS• SMS Elements

– Safety Management Plan– Documentation– Safety Oversight– QA, Training, EPR

• Safety Performance Measurement • Safety Culture • Implementation of SMS • SMS Evaluation Guide• Process to Assess Industry SMS• Completing the Assessment

18

19

The Assessment should get to the “Real” Issues !!

DD

Addition of Broader Audit Approach

• DOs used to inspections and checklist-driven audits

• ‘Process’ vs. ‘procedure’ view not well differentiated: had to get up out of the trees

• No methodology for producing defensible, quantified assessments of SMS: development of the SMS Assessment Process

20

21

SMS Assessment Process

• Pre-Assessment

• On-Site Observations

• Post Assessment and Follow Up

22

Assessment Criteria

Plugging the Holes with PDCA

• Gaps in management system elements identified in course design were not addressed until V2/V3 of the course:– Management review– Proactive process (risk management)

• Some backlash from attendees as a result, however intent was to arm DOs as best possible to allow them to engage industry

23

Institutional Challenges

• SMS needs a Paradigm shift within aviation from :– DOs are responsible for aviation safety

• responsible for operator’s actions and take accountability for them

– Operators must get all actions approved by TC

– Audits of operators are only done by DOs and the operators cannot be trusted to do carry out internal ones themselves

– Conflict of interest in the "old way of doing things"

24

Regional Challenges

• Different degrees of involvement and engagement by Regional Directors

• Regional cultural differences: adversarial vs. partner approaches from POIs/PMIs

25

Internal vs. External Reaction

• Within TC:– Sounds good in theory– Are we prepared to back this up?– How does this system scale from large to

small?– Will TC really enforce the SMS CARS?

26

Internal vs. External Reaction cont’d

• Within industry:– Larger operators/AMOs/airports: we do this

now (but may lack some rigour…)– Hazard analysis and identifying risk significant

activities often a weakness– Smaller operators: what is it exactly that you

want me to do (and when exactly do I get to make a living…)?

27

In Conclusion

• Training of TC Civil Aviation a major undertaking (time, personnel, cost)

• Paradigm shift just now starting to show broadly in TC

• Communicate, communicate, communicate: can it ever be enough?

• We see senior managers taking more of a lead, and see that aviation experts are needing to become more proficient in management

28

In Conclusion cont’d

• SMS is about risk control - the loop of hazard-risk-objectives-measures-audit-management review drives a big shift in the way management perceive their roles and forces communication into the organizations

• Tough part awaits: small operators

• Grateful to have been able to play a role

29

Questions