Root Cause Analysis: An Evaluation of the RCA Training

Program

by Douglas Bushong

Prepared for the management team at North Utility

Boise State University

EDTECH 505

Abstract

Overview An evaluation was conducted to assess the effectiveness and impact of the RCA

training program. The evaluation began shortly after the initial RCA course was developed, and focused on three questions:

1. Are the front-line supervisors conducting a full RCA for each incident? 2. Are the details of the RCAs being communicated on a weekly basis? 3. Have the month-to-month preventable accidents decreased?

During the evaluation, three key pieces of data were analyzed: (1) the RCA reports completed by the front-line supervisors before and after the training, (2) the weekly conference calls that occurred in the weeks following the training, and (3) the month-to-month incident rates.

Objective 1 - Full RCAs for Each Incident The evaluation showed an improvement in the overall quality of the RCA reports

upon completion of the initial seminar. While the quality of the reports reflect a general consciousness raising, additional training on initial investigation process is necessary before this objective can be considered complete.

Objective 2 - Regular Communication of RCA results. The calls occurred on a weekly basis, and are expected to continue. Most of the

supervisors and managers attend the call, the RCAs have been discussed, and corrective actions have been spread throughout the organization. Even the call that was missed was postponed specifically to ensure that the RCAs of that week would reach more people.

Objective 3 - Reduced Incident Rates The increased awareness of the investigation process did not translate into reduced

incidents in the first month (October). The dramatic reduction the following month may be the start of a trend, but it is too early to tell. Continued monitoring is required.

Recommendations Based on the data, the evaluation submits the following recommendations:

1. Develop and deliver a follow-up training on the initial investigation to the supervisors based on the 5 Ps document.

2. Add the extended initial investigation training to the RCA seminar. 3. Continue to provide one-on-one training to supervisors as incidents occur. 4. Prepare for group follow-up training when the storm season is over 5. Continue to monitor the incident rates.

Introduction

North Utility (hereafter referred to as "the company"), is an electric and gas utility that provides generation, transmission, and distribution services to the Great Lakes area. Normal activities include installation and removal of services, repair/replacement of damaged poles, and repair of natural gas leaks. The company has over 2000 employees.

In recent years, the company has seen an increased number of preventable accidents and OSHA recordable injuries. The investigations of these incidents typically placed the blame of the incident on the individuals directly involved and/or treated the symptoms of the problems rather than the core problems.

The company's Vice President of Operations and the management team recognized a deficiency in the incident investigation/root cause analysis process, and asked the training department to develop a root cause analysis (RCA) training program for the front-line supervisors. The purpose of the training was to teach supervisors how to properly investigate safety incidents and identify long-lasting solutions that address the root causes of the problems.

The RCA training program was rolled out in two steps. The first part was an initial one-hour seminar that provided a brief overview of the RCA process. The second part was one-on-one or small group extended training with the supervisors on how to conduct the RCAs through case-studies and actual incidents. This is a much more extensive process that allows the them to see the process in action.

An evaluation was conducted to assess the effectiveness and impact of the RCA training program. The evaluation began shortly after the initial RCA course was developed, and focused on three questions:

1. Are the front-line supervisors conducting a full RCA for each incident? 2. Are the details of the RCAs being communicated on a weekly basis? 3. Have the month-to-month preventable accidents decreased?

This report contains the findings of this evaluation, and provides recommendations for further action.

Program Description

The intended users of this program are the front-line supervisors and their managers. These are the people who will be conducting the initial investigations and RCAs, and who will usually report the results of the RCA during the weekly safety call.

Program Objectives

The program objectives were based on the three questions identified in the introduction. There were three objectives.

1) A full RCA will be completed for each incident. 2) A conference call will be conducted each week to review and share the RCA

results with all of the supervisors in the department. 3) Month-to-month preventable accidents will decrease.

Program Components

The program consisted of the initial 9-step presentation, a "5 P" document provided during follow-up training to help supervisors conduct investigation interviews, and the reports reviewed during the weekly conference call.

9-Step Presentation

The supervisors were taught a 9-step process during the initial seminar. This process is derived from the TapRoot RCA method, and includes the following steps.

1) Initial Investigation - based on the 5P model 2) Timeline development 3) Identification of causal factors 4) Identification of the root causes - asking "why" 5 times from each causal factor. 5) Analysis of the root causes 6) Development of corrective actions. 7) Layer of Protection Analysis, or "Barrier Analysis" 8) Implementation of corrective actions 9) Monitoring of corrective actions.

At the end of the initial one-hour seminar, the supervisors were given a CD-Rom with a copy of the presentation, the instructor's guide for the presentation, a template containing the 9-step process for completing their own RCAs, and a 45 minute video of the presentation to allow them to review as needed. The management team has received a copy of this CD-ROM, and additional copies are available upon request.

5 Ps Interview Tool

Based on immediate discussions following the seminar and follow-up sessions, an additional tool was provided to elaborate on the initial investigation interviews. This was a three-page document based on the 5P's - Parts, People, Position, Paper, and Perspective - and was used as a guideline for interview questions. This document was developed after the initial seminar, and was provided to supervisors during one-on-one training sessions. A copy of this document is included as Appendix A.

Weekly Reports

Each weekly report consisted of a safety topic of the week for the supervisors to provide to their employees, followed by one or more RCA reports describing the incidents reviewed that week. Examples of RCA reports completed before (Appendix B) and after (Appendix C) the RCA training are provided in the appendices.

Evaluation Method

This formative evaluation used a combination of a goal-based and transactional model. Clear goals were defined at the beginning of the process, and those goals were used to benchmark the success of the overall program. At the same time, the evaluator was not completely independent of the process, and provided feedback and recommendations along the way. As a result, some changes to the initial plan were made throughout the process.

Participants

Subjects

Like the intended users, the subjects of the training program were the front-line supervisors and their managers. The relatively even mix of new and experienced supervisors shared a common trait of no formal training on accident investigation.

Evaluator and Supporters

In addition to the subjects, the following individuals assisted in the development and delivery of the RCA training program.

Douglas Bushong (Evaluator) Douglas developed the initial presentation, and led the initial seminar for the

supervisors. Douglas has experience in incident investigation and root cause analysis from his time in the US Navy's nuclear power program, and currently works as an instructional designer for the company. After the initial training, Mr. Bushong led the evaluation.

Joe Wallace Joe is a safety training consultant brought in by members of the management team.

He has developed RCA training in the past for other companies. Joe provided the initial structure for the RCA training, and used his expertise to lead the one-on-one training sessions that followed.

Jodi Bauer Jodi recently transferred to the training center to support the RCA training program.

Prior to coming to the training center, Jodi worked as a meter reading supervisor for over 10 years. Jodi 's perspective was especially useful, given that the meter reading department was previously a major source of the preventable incidents. While the training program was being developed and prior to her arrival at the training center, Jodi had the opportunity to work with Joe Wallace and to apply the RCA principles in her own investigations. The feedback she provided after applying the process was useful when revising the initial training.

John Caminski John is the head of the safety advocate program. With over 20 years of experience

in line work, John provided useful insight into the attitudes and concerns of the front-line workers. John only participated on the project for a short time due to medical reasons.

Neville Smith Neville is the data analyst for the company's safety department. He collects data for

the safety calls from the company's RiskConnect incident reporting system; his data allowed the evaluator to trend the incidents.

Union Leadership In the past, many of the root cause analyses and fact findings were "witch hunts"

that focused on finding someone to blame. In order to have candor in the RCA

interviews and investigations, the union members needed to know that the RCA process was not a mechanism for discipline, but for prevention of future incidents.

Procedures

The initial training was delivered on 28SEP2011 and 29SEP2011.

The initial intention was to provide an overview of the RCA process followed by small-group training sessions. However, the small-group training sessions were replaced by one-on-one sessions because of continuous scheduling conflicts. The supervisors were regularly on-call, and the evaluation period occurred during the beginning of the heavy storm season.

To streamline the process, the training sessions were conducted during actual incident investigations in time slots already allotted by the supervisors. While less efficient, the one-on-one sessions allowed for more personalized training.

The following activities were performed by the evaluator during the 9-week period following the initial training session.

1. Observation of one-on-one, expert-led training sessions. When incidents occurred, the supervisors contacted the safety and training department to provide support during the investigations. These investigations were used as training opportunities to show the supervisors how to apply the process.

2. Observations of weekly conference calls to review all RCAs for that week. The RCA reports were reviewed and discussed during each call, and these reports were compared to the reports dating before the initial training and the 9-step process described in the training. Special attention was paid to the RCA's completed by supervisors that had received one-on-one training.

3. Continuous measurement of preventable accidents. This data was collected throughout the evaluation period by the data analyst.

Data Sources

The data sources selected for this evaluation were based on the objectives identified at the beginning of the evaluation.

Objective 1 The problem that led to this objective was a history of inadequate RCAs prior to the

roll-out of the training program. With that in mind, this evaluation looked at the RCA reports completed before and after the training was rolled out, particularly those reports completed by supervisors that had received one-on-one training. These reports were selected instead of a survey specifically because they represent the supervisors' application of the RCA process, and not just their perception.

Objective 2 To ensure constant communication, the weekly conference calls were monitored

by the evaluator to ensure that all of the RCA's were discussed and to ensure that all questions were answered thoroughly. Occasionally, the evaluator would ask questions during the call to provide clarification.

Objective 3 The month-by-month incident figures were retrieved by the company's data

analyst, and provided to the evaluator. This data was trended and included in this report.

Results

Data for Objective 1 – RCA Reports

Pre-Training Examples of typical reports completed prior to the training are included in Appendix

B. The RCA reports prior to the training did not demonstrate that the supervisors had completed all of the steps of the RCA process. These reports tended to:

Focus on the employee

Identify only a single causal factor

Treat the causal factor as the root cause

Provide specific corrective actions for the individual incident rather than generic corrective actions

Post-Training Examples of typical reports completed after the training are included in Appendix

C. The RCA reports following the training include a greater level of detail. Further, those reports completed by supervisors who had received additional one-on-one training demonstrated completion of all or nearly all of the 9 steps of the RCA process. These reports tended to:

Focus on the process

Identify multiple causal factors

Explore each causal factor to arrive at a generic root cause

Provide general corrective actions that can be performed by all supervisors to prevent further accidents.

Data for Objective 2 – Weekly Conference Calls

In the nine-week period between the completion of the initial training to the completion date for this report, eight safety conference calls were held. The conference calling service does not have a mechanism for determining who was on each call, but during each call specific supervisors were called upon to answer technical questions about the RCA. There were no instances of someone being called upon and that person not answering.

The one week that a call was not held was during the week of the Thanksgiving holiday. A large number of employees had taken a vacation during that week, and the manager leading the call did not want so many employees to miss the information presented in those RCA reviews. For this reason, he postponed those reviews until the following call.

Data for Objective 3 - Month-by-Month Incident Data

The full data set can be seen in Appendix D. The diagram below briefly shows incident trends leading up to and following the RCA training.

The blue lines show auto accident rates, and the red lines show injuries. The black line shows the total incident rate - the sum of the auto and injuries. This rate continued to trend upward from April to August with a large drop in the month of September. The initial training was provided at the end of September. October was the worst month, but was followed by a large drop in November.

Discussion

Objective 1 - Full RCAs for Each Incident

The evaluation showed an improvement in the overall quality of the RCA reports upon completion of the initial seminar. While the quality of the reports reflect a general consciousness raising, additional training on initial investigation process is necessary before this objective can be considered complete.

Objective Status: Partially Achieved.

Objective 2 - Regular Communication of RCA results.

The calls occurred on a weekly basis, and are expected to continue. Most of the supervisors and managers attend the call, the RCAs have been discussed, and corrective actions have been spread throughout the organization. Even the call that was missed was postponed specifically to ensure that the RCAs of that week would reach more people.

Objective Status: Achieved and ongoing.

0

5

10

15

20

25

30

35

40

Apr-11 May-11 Jun-11 Jul-11 Aug-11 Sep-11 Oct-11 Nov-11

Recordable Injuries Total Injuries

Preventable Auto Total Auto

Total Incidents

Objective 3 - Reduced Incident Rates

The increased awareness of the investigation process did not translate into reduced incidents in the first month (October). The dramatic reduction the following month may be the start of a trend, but it is too early to tell. Continued monitoring is required.

Objective Status: Not Achieved.

Overall Recommendation

Based on the data, the evaluator has the following recommendations:

1. Develop and deliver a follow-up training on the initial investigation to the supervisors based on the 5 Ps document. The quality of the RCA depends largely on the data initially gleaned during the investigation. The supervisors have shown that they can apply their data to the process, but they have struggled with the data collection process itself.

2. Add the extended initial investigation training to the RCA seminar. This omission was quickly pointed out by the supervisors, and should be corrected for future training sessions.

3. Continue to provide one-on-one training to supervisors as incidents occur. The reports provided by supervisors that have received the one-on-one training have shown dramatic improvements. This process is working, albeit slowly.

4. Prepare for group follow-up training when the storm season is complete. The changes that had to be made to the procedure were largely due to the storm season. As work slows down and becomes more regular in the spring, supervisors should be more available to attend group case-study sessions.

5. Continue to monitor the incident rates. There was not enough time during this initial formative evaluation to determine whether the RCA process has translated into reduced incidents. Monitoring of the recordable and preventable incidents should continue, and should be reviewed again in 12 months.

Project Cost

Personnel

Salary Information Days Worked Rate/day Cost

Douglas Bushong 20 $500/day $10,000

Total $10,000

Travel Expenses*

GSA Standard POV Mileage: $0.51/mile

Work Location # of Occasions

Round Trip Distance Travelled (in miles) Cost

South Lake Office 3 74 $113

Valparaiso Office 5 28 $71

Plymouth Office 3 100 $153

Total Travel $338

Materials**

Item

Expense Information Cost

Miscellaneous Costs

Copies, special prints, etc. $400

Total

Materials $400

Total Budget $10,738

* The work performed was near the evaluator's hometown, and no overnight travel was required. Travel was generally to attend a one-on-one training session, and most of the work was done in the La Porte office.

**The company provided access to its materials for all printing services, copies, etc. The miscellaneous cost covered expenses incurred while away from the La Porte office.

Appendix A - Incident Investigation Guidelines

Note on Employee Injuries

When an employee has been injured, the primary concern is to get the employee the necessary first aid or medical treatment. Accompany the employee to the medical provider if practical. Contact designated administrators and supply them with preliminary information to start managing the case.

Investigation - Regarding “Why” Questions

Remember, during the investigation interviews, avoid questions that begin with “why.” “Why” questions call for interpretation, but you want just factual information. Who/what/when/where questions are great, and “how” questions can be used to describe processes (“how do you normally determine which wire to use”, etc.).

The temptation to ask a “why” question often occurs when there was a deviation from procedure. The best way to deal with this is to start with “how do you normally do it” followed by “what was done differently this time” and maybe “how did you come to the decision to do it outside of the way it is normally done?”

The Five P’s of Investigation

Parts Identify, and determine the condition of any equipment, tools, PPE, or materials in use

at the time of, and associated with, the incident. Parts include physical objects; in addition to the company materials, this can include fences, doorbells, sidewalks, etc.

Position Identify and document the physical arrangement of the employee(s), tools & equipment,

and the work site.

Location: Physical placement of tools, equipment, and people (employees, public, witnesses) at time of incident. If possible, this should include the direction that each object is facing.

Timing: Chronological timeline of events surrounding the incident.

Conditions: Weather and terrain conditions.

People Identify everyone involved or that can help with the investigation of the incident.

Direct involvement: Identify employees on the job-site at the time of the incident as well as those that may have been remotely involved or have information of the factors involved (Gas Control, Electric ROC, etc.)

Witnesses: This can include employees, contractors, or the general public.

Experts: Employees, vendors, or technical resources that may have knowledge of mechanical, operational, or other issues that may have been involved in the incident.

Medical Professionals: If the employee sought medical attention, be sure to document this.

Paper Locate and review any paperwork that might contain details about the incident or the

employees involved. When we say “paperwork,” what we are really talking about here is information. It doesn’t have to be physical paper; any kind of data can be useful.

Examples of this data can include:

Written procedures governing the work

Pre-Job briefing forms

Equipment inspection forms

Time sheets Project packages

Job-site sketches Employee training records

Maintenance Records Employee accident & injury history (previous occurrences)

Video records (Security cameras, etc) SCADA logs

Applicable procedures/policies

Event logs

Handheld meter download data

Pictures, pictures, pictures! Take as many pictures as reasonably possible, understanding that only a few select images will actually go into the final report.

Perception This is the employees’ and others’ mindset and perspective on the work activity. This

can give insight into the actual vs. the approved practices in place at the time of the incident. Many of these things will not be directly questioned, but will come out during the investigation. Common phrases might include.

“The procedure takes too long” or “That procedure wasn’t applicable in this situation.”

“We’ve always done it this way and it worked.”

“We’ll never get anything done if we do it that way.”

Organizing the Data

Once the data is collected, analyze it by asking yourself the following questions:

Task

Were safe work procedures followed?

Had conditions changed to make the normal procedure unsafe?

Were the appropriate tools and materials available? Were they used?

Were safety devices working properly?

Equipment & Materials

Was there an equipment failure?

What caused it to fail?

Was the equipment / tool / material poorly designed? Substandard?

Was PPE used? Should it have been?

Physical Environment

Did weather conditions contribute to the incident or the severity?

Was poor housekeeping a problem?

Was there adequate light?

Was noise a problem?

Employee considerations

Were employees experienced in the work being done?

Had they been adequately trained on the equipment?

How much experience has the employee had with the equipment, outside of training?

Were they physically capable of doing the work?

Were they tired?

Were there any extenuating circumstances?

Leadership

Were safety rules communicated to employees?

Were rules being enforced?

Were employees trained / qualified to do the work?

Had hazards been previously identified?

Were safety issues corrected?

Was regular maintenance of equipment carried out and documented?

Were regular Work Site Observations performed on the injured employee?

Appendix B - Pre-Training RCA Reports

NOTE: All spelling, grammar, wording, etc., was left as it was written on the report. The only changes made were omissions of personally identifiable data, marked in red.

Example 1

Driver Name OMITTED Employee Supervisor OMITTED Accident Date 6/22/2011 Short Description Other Struck Stationary Object Summary of Findings While driving down a two lane road with parked cars on both sides, an oncoming vehicle approached him. He moved over to the right to allow for the oncoming vehicle to pass between him and parked vehicles. When getting over made contact with overhanging limb. Root Cause Driver did not get the big picture, once the limb and vehicle was past his field of vision he thought he had clearance to make it past limb. Vehicle Accident Type Hit fixed object Detailed Contributing Factors Keep Your Eyes Moving (Smith Key) Corrective Actions

Going to do a ride along in similar areas with parked cars on both sides of the street and over hanging trees.

Had a discussion with line department personal, concerning the placement of a truck when pulling to the side of a two lane road with parked cars on each side and over hanging tree limbs. To know what the clearances you have with trees and other objects that might over hang road ways.

Example 2

Driver Name OMITTED Employee Supervisor OMITTED Accident Date 6/10/2011 Short Description Other Going Straight Ahead Summary of Findings While South bound on cr300 W ., Driver was forced into hugging shoulder by oncoming vehicle . Driver spotted potentially low tree branch , but didn't perceive it as a hazard & continued forward . Tree branch was obscured by leaves & vines, so size was undetermined. Contact was made with the leading top edge of the bucket , & damaged the bucket to the extent of having to be replaced . Vehicle Accident Type Hit fixed object Detailed Contributing Factors Roads Corrective Actions

Assign alert driving training

Porter County was contacted & trimmed low branches in that stretch of road

Employee was told to always proceed with caution & be aware of your surroundings

Appendix C - Post-Training RCA Report

NOTE: All spelling, grammar, wording, etc., was left as it was written on the report. The only changes made were omissions of personally identifiable data, marked in red.

Example 1

Employee's Name OMITTED Employee Supervisor OMITTED Injury or Illness Date 10/7/2011 Accident Description On 10/7, while pulling conduit on a job site, employee felt a strain to his lower back. The pain progressed. The back was very stiff, and he could hardly move. Self-care measures were attempted, but it didn’t feel any better Summary of Findings Job briefing was not adequate. Little specifics on site preparation and hazard recognition Company guidelines on meter box locations and customers installing conduit Follow cable and pulling procedures Employee pulling rope from an awkward position Chain of Events that led to the incident, in detail Employees 1,2,3 have a 30 min. drive to job site. Employee 1 conducts job briefing with employee 2 & 3. Crew has approx. 35 ft. of cable to pull into customer installed conduit. The rest is direct buried to service pole (60”). Cable pull has two 90 degree elbows and one 45 degree elbow with 23 ft. straight section, with end exposed in trench. Employee 2 & 3 run snake toward meter base where employee 1 was located at meter base. Employee 1 lays a board across open trench to stand on to gain stability pulling in cable. Employee 1 attached pull rope to snake and employees 2 & 3 pull back rope. Employees 2 & 3 attached wire weasel to cable. Attached rope to wire weasel. Employees 2 & 3 are pushing cable into conduit that is laying in trench. Employee 1 is pulling rope at meter base. They get to what they think is second 90 degree elbow and wire stops. They reset the pull push task and get the cable into the meter box. Historical Operating Issues, problems or incidents None Abnormal conditions or events Most of the lakes in the area are now on sewer systems. They are now allowed to build very close to lot lines which leaves Utilities no room to get equipment back to some of the meter locations. When this happens, we are telling customers to run conduit out past the house. From customers conduit, we can direct bury out to service hook up point. Some customers will run conduit out to service point. Some will install their own cable from meter base to service hookup.

Example 1 (Continued)

Operating Conditions prior to and during the incident The site had some mud because of new trench but was dry away from the home and by meter base. Detailed Contributing Factors Ergonomics – Body Position Planning – Ergonomic practices not adequately addressed Training – Ergonomic hazards not communicated Execution – Awkward or static positions employed Work Procedures Planning – Safe procedures not defined adequately Training – Pre-job briefing not performed Hazard Recognition Planning – Hazards not fully understood or known Training – Hazard not recognized Execution – Pre-job briefing not performed properly Training Planning – Training not adequate Work Practices Planning – Safe work practices not defined Training – Safe work practices not communicated Execution – Proper practices not utilized Comments: Job briefing was written with little specifics as far as site preparation and hazard recognition. Employee has had prior back problems. Corrective Actions

Provide job briefing training to entire crew, which will improve the quality of job briefing conducted in the field.

Improve job briefing form to aid employees as they perform the job briefing.

Review with Eng. Dept and Standards Dept. the guidelines for meter base location and customer installing conduit.

Training review for all servicemen about cable pulling in conduit.

Research for a better tool or equip for pulling cable into close, cramped quarters like meter box.

Appendix D - Incident Trend Data

Incident Trends, beginning April of 2011

Ap

r-1

1

May

-11

Jun

-11

Jul-

11

Au

g-1

1

Sep

-11

Oct

-11

No

v-1

1

Recordable Injuries 4 5 8 8 6 4 9 3

Total Injuries 8 8 14 20 15 15 20 12

Preventable Auto 3 1 7 2 6 2 7 5

Total Auto 5 6 14 12 17 5 14 8

Total Incidents 13 14 28 32 32 20 34 20

0

5

10

15

20

25

30

35

40

Apr-11 May-11 Jun-11 Jul-11 Aug-11 Sep-11 Oct-11 Nov-11

Recordable Injuries Total Injuries

Preventable Auto Total Auto

Total Incidents