vpp annual self-assessments - star consultants evaluation.pdf · vpp annual self-assessments a...
TRANSCRIPT
VPP Annual Self-Assessments A Process for
Continuous Improvement
STAR Consultants, Inc. www.starconsultants.net
Paul A. Esposito, CIH, CSP Annapolis, MD 21012
410-218-8451
Prepared for: VPPPA
Reg. 3, 2013
2
WHO IS STAR?
In business since 1997
Founded by Paul Esposito, CIH, CSP
33 years of experience
Over 300 VPP Assessments performed since
1992
Located in Annapolis MD
Safety Through Accountability and Recognition
3
Workshop Outline
Drivers and Requirements
Evaluation Process Document Review
Interviews
Visual Observations
Report Format
Goals, Objectives and Action Plans
References/Resources
4
What Are The Drivers?
http://www.osha.gov/dcsp/vpp/index.html
OSHA’s S&H Program Management Guidelines 1/26/1989
OSHA Federal Register Notice 51:33669-77 (1986, September 22).
Revisions: (2000, July 24)
CSP-03-01-003. April 18th 2008
VPP Federal Register Notice January 9, 2009
5
When do you do Evaluations?
As part of the application process
At least one year prior to the application and
Re-survey, to demonstrate ability to actually
evaluate and improve
Annually once you are in the VPP
6
• Annual Self-Evaluations
WORKSITE
HAZARD
ANALYSIS
HAZARD
PREVENTION
& CONTROL
SAFETY
& HEALTH
TRAINING
• Management Commitment
• Policy • Goals, Objectives and
Planning • Visible Top Management
Leadership • Responsibility and
Authority • Line Accountability • Resources • Contract Employees • Written S&H
Management System
• Routine Hazard Analysis
• Significant Change Hazard Analysis
• Pre-use • Baselines
• Routine Self-Inspections
• Hazard Reporting • Industrial Hygiene • Investigations • Trend Analysis
• Certified Professional Resources
• Hazard Elimination and Control Methods
• Hierarchy of Controls • Engineering • Admin • Work Practices/Rules • PPE • Discipline
• Process Safety Management
• Occupational Health Care
• Preventive Maintenance
• Hazard Tracking • Emergency
Preparedness
• Managers • Supervisors • Employees • (Contractors) • Emergencies • PPE
VPP Criteria – Evaluation Report
Management
Commitment
Employee
Involvement
• (Encourage-ment
• Participation • Committees)
2008: Appx C, Section D: CSP 03-01-003!
MANAGEMENT
LEADERSHIP
7
VPP Annual Self-Evaluation
Is: Objectives and Action Plan Review
A systematic evaluation of your company to each VPP element.
An examination and verification of management records, procedures, training and the workplace environment.
Interviews with employees and supervisors in all departments
Is not: Plant audit or inspection looking for physical hazards and violations.
Corporate Compliance Audit
8
An Assessment
Document
Review
Interviews Verify/Observe
Complete the
Protocol
Preparation, Protocols and Document Review
9
Who Can Perform a Self-
Evaluation? Site Officials
Safety Committee
Quality Auditors
Corporate Personnel
Third Party
Insurance Companies
Consultant
Qualifications
Knowledge of S&H programs
Knowledge of site operations
Independent
Or any mix of these.
10
Creating the Assessment Protocol
Use public or commercial criteria lists, create a
listing of program expectations for each of the
program elements. • Site Based Participation Evaluation Report
(note: this evaluation does NOT match up to the annual
report elements)
Or, Self-Assessment Guides
are commercially available
11
Step 1: Modify or Develop
Criteria Use OSHA Form 33 (SHARP)
Use OSHA’s Program Management Guidelines (1/26/89)
Use Checklist in OSHA’s “Managing Worker Safety and Health”, Chapter 12
Department of Energy (VPP Part IV), http://tis.eh.doe.gov/vpp/policy/policy.html or
Site-based Evaluation Report
Commercially available protocols (e.g., VPPProfile™ Screen from STAR)
12
Program Evaluation - VPP
Requirements An Element of Management Leadership
Requirements:
Annual Review – conducted by qualified personnel
Evaluate their success in meeting the goal and
objectives,
The written annual evaluation must identify the
strengths and weaknesses
Must contain specific recommendations, time lines,
and assignment of responsibility for making
improvements.
And address the status of last year’s actions, goals and
objectives
Chapter III C.1.d
13
Program Evaluation - VPP
Requirements An Element of Management Leadership
Requirements: It must also document actions taken to satisfy the recommendations.
The annual self-assessment must be in place at least one year prior to VPP application submittal
Submitted as a Narrative Report, Feb. 15th to OSHA VPP (2 copies)
Injury rates and applicable contractor rates. With explanation of potential problems
Construction: must also perform a final evaluation immediately prior to completion of construction
Chapter III C.1.d
14
Malcolm Baldrige (Form 33) Approach LEVEL/
CATEGORY: 1
A. Ineffective 2
B. Developmental 3
C. Basic 4
D. VPP 5
E. World Class
1.2.1* Goals & Objectives (Action plans)
1. Site H&S goal is established for injury and illness rates.
1. Some descriptive goals exist, and are in part based on a review of injury and illness analysis and the ESP Work Plan.
2. Some objectives to
support the goals are developed and documented.
3. Some goals are owned by
Management (not H&S).
1. Objectives have been established with input and support from all levels in the organization.
2. Supporting objectives
aim at a specific area of performance that can be measured or verified.
3. Goals and objectives are
communicated to workers.
4. Goals and objectives are
developed for most teams and committees.
5. Some goals and
objectives are owned by line management.
1. Some goals and objectives also stem from an analysis of incident root causes, inspection findings, worker concerns, maintenance safety issues, and other safety data.
2. Departments each have
established one or more objectives annually as part of the H&S program.
3. The majority of
managers, supervisors and the workforce can explain program goals and objectives and the measures to achieve them.
1. Objectives are developed as part of a grass roots process, with agreement from the majority of the workforce.
2. Personnel at all levels
can explain the desired results and measures for achieving H&S program objectives.
Step-by-Step, 5 Level Approach
16
Response Scoring / Weighting
ResponsePoints
receivedPoints
available
Evaluation Criteria
1.1.1 Meets 3 3
1.1.2 DNM 0 3
1.1.3 PM 2 3
1.1.4 N/A - -
1.1.5 UD 1 3
Raw Score* 6/12 12
Score 50%
*Points Achieved/Points Available
Red = 0 – 50
Yellow = 51 – 79
Green = 80 - 100
17
Management
Employee Involvement
Worksite Analysis
Hazard Prevention
Training
Using a scoring system,
measure and compare
program elements each year
to truly determine of your
programs have improved...
> 50%
51 to 79%
< 80%
Red
Yellow
Green
Previous Evaluation Current Evaluation
ID Description # of
Priority
1’s
Section
Score
# of
Priority
1’s
Section
Score
1.1 Safety & Health Policy 3 56 2 78
1.2 Goals & Objectives 2 44 1 59
1.3 Planning 2 76 2 78
1.4 Top Management Involvement 1 79 - 85
1.5 Responsibility & Authority - 75 - 78
1.6 Line Accountability 1 62 1 62
1.7 Resources - 92 - 100
1.8 Contract Workers - 88 - 88
1.9 Written Safety & Health Management Systems 5 47 3 72
1.10 Program Evaluation - - 2 80
1.0 Management Commitment 12 59 11 69
2.1 Encouragement 5 55 3 76
2.2 Participation 4 46 2 83
2.3 Committees 3 76 2 84
2.0 Employee Involvement 12 59 7 81
3.1 Baseline Assessments 4 33 3 50
3.2 Routine Hazard Analysis 3 76 2 81
3.3 Change Hazard Analysis 3 33 2 64
3.4 Inspections 9 25 5 59
3.5 Reporting System 2 76 1 89
3.6 Industrial Hygiene Program 7 17 5 33
3.7 Investigations 3 58 1 76
3.8 Trend/Pattern Analysis 8 46 3 79
3.0 Worksite Hazard Analysis 39 46 22 66
Comparative Scores
18
Assessment Activities
Pre-Assessment Activities
Select and Schedule
Facilities/Department to
Assess
Select Team Members and
Confirm their Availability
Gather Background Info.
• Discuss program • Advance visit (optional) • Pre-survey questionnaire
Post-Assessment Activities
Prepare Draft Report
• Send to cognizant parties
Issue Final Report
Action Planning
• Proposed action(s) to address
each finding
• Responsibility/timetable
actions
• CO Endorsement
Schedule Follow-Up
Key Activities at Activity
STEP 1- Understand Management Systems
STEP 2- Assess Strengths and Weaknesses
• Internal risk • Internal controls
STEP 3 Gather Evidence
• Assessment • Verification (Interviews) • Sampling strategies
STEP 4 Evaluate Findings
• Review operations • Perform Visual Observations • Evaluate controls
STEP 5 Report Findings
•Score the Assessment
• Exit meeting
• Review Policies and Procedures • Interview responsible parties
Preparation, Assessment, Report, and Follow-up
• Define scope • Identify priority hazards • Prepare/modify protocols • Allocate resources
Plan the Assessment
19
WORKSITE
HAZARD
ANALYSIS
HAZARD
PREVENTION
& CONTROL
SAFETY
& HEALTH
TRAINING
• Management Commitment
• Policy • Goals, Objectives and Planning • Visible Top Management Leadership • Responsibility and
Authority • Line Accountability • Resources • Contract Worker
Coverage • Written S&H
Management System • Annual Self-Evaluations
30 Elements OSHA’s Program Management
Guidelines - VPP Criteria
Management
Commitment
Employee
Involvement
1/26/89 & 3/25/03
MANAGEMENT
LEADERSHIP
20
Document Review
Documents are reviewed to determine the depth of understanding about the requirements and programs. Match the records to those required by the procedures or documentation
Interview Authors and Responsible Parties to verify your understanding about the process
Identify a hypothesis about where gaps may exist
Verify program strengths by planning additional interviews (I) and visual observations (V)
Challenge the gaps to determine of they are systematic or random
Written programs by themselves have limitations, such as:
who reads them/knows them
how are they used
are they really implemented and to what extent
Gaps at this level will almost guarantee systematic failures at the
implementation stage.
21
List of Documents Completed by Site Completed by Evaluator
Document Description Owner
(Individual)
Location
(Organization)
Completed
Yes / No Comments
1.1 Policy
1) Safety and Health Policy 1) 1)
1,2 Goals & Objectives
1) Safety and Health Program
a). Goals
b). Supporting Objectives
c). Action Plans
(previous and current year’s)
1)
a)
b)
c)
1)
a)
b)
c)
1.3 Planning
1) Management Meeting Minutes 1) 1)
1.4 Top Management Involvement
Section 1.10 Program Evaluation & 2.3 Committees
1.5 Responsibility and Authority
1) Job Descriptions: (sample three or four of each)
a) Management
b) Salaried
c) Hourly
2) See 1.9 – Written Programs
1)
a)
b)
c)
2)
1)
a)
b)
c)
2)
Where do we find these?
Who owns them?
22
Goals and Objectives - Examples
Goal: Improve our Hazard Identification & Hazard Reduction.
Objective: "Conduct weekly inspections with emphasis on good housekeeping, proper use of protective equipment, condition of critical parts of equipment, and preventive maintenance. "
Goal: Improve our Accident Investigation Process.
Objective: "Determine the cause(s) of any accident within 24 hours. "
Goal: Implement a Job Safety Analysis Program.
Objective: "Complete one job safety analysis each month in each department, with follow-up revision of safe work procedures and employee training by the following month."
Goal: Improve our Emergency Preparedness Capability.
Objective: "Hold and evaluate emergency drills for tornadoes every six months and a joint fire drill/evacuation with local emergency organizations every year.”
Still need to ID Action plans, owners, deadlines
23
Worker Interviews
What questions do you ask? To start with?
Follow-up?
To avoid?
How many employees do you interview? One-on-one
Focus Groups
Who do you interview Workers
Supervisors
Committees
Contractors
Get specific examples of both strengths and weaknesses for follow-up!!!
Interviews
24
Personnel Interviews
… Are typically performed after you review documentation to determine if the information is appropriate, communicated, understood and practiced.
… Employee interviews are typically the initial step in verifying S&H program implementation and effectiveness.
… Perform personnel interviews after your document review and responsible party examinations. It is helpful to know some of the answers or problem areas before the interviews. Use the interviews to get specific examples.
Interviews
25
Interview Qs from Appx F.
1. Can you tell me what the safety and health policy is at this worksite?
2. Can you tell me what the overall goal(s) for safety and health is at this worksite? Where are we now? Ahead or behind?
3.What is one objective in your department to achieve the goals?
4. What is one method of reporting a safety or health concern?
What was the last unsafe practice you reported and/or
corrected?
5. How do your supervisors demonstrate their involvement in
safety and health?
Interviews
26
Personnel Interviews
Employee Interview Table
Employee # 1 2 3 Max - allMin - allAvg - allStd Dev - all % 4 & 5
B. Orientation and Training 67.00%
1. Did you receive safety and health training when you began
to work here? (If so, please describe). How soon after you
begin to work did you receive training? How long did it last? 2 4 5 5 2 3.67 1.25
2. If you did not get training when you were first hired (or
transferred to a new job), have you received any basic safety
and health training since that time? If so, please describe. 3 4 5 5 3 4.00 0.82
Maximum 3 4 5
Minimum 2 4 5
Average ### ### ###
Standard Deviation 0.5 0 0
1- Weakness ……………………………………………. Strength- 5
27
Element Scoring: Perception Survey
Management strengths may be considered by the % of 4’s and 5’s for each category ( # interviewed). Gaps = % scores less than 50%, Average responses less than 3.5 or Standard Deviation greater than 1.0.
Management Commitment 62% 3.2 0.87
Gaps: Goals and objectives Top mgmt involvement Communication
Employee Involvement 45% 2.3 1.33 Gaps
VPP knowledge Committee/Team members Committee/Team Activities
Worksite Analysis 70% 3.6 0.54 Gaps:
Reporting hazardous conditions Inspection results?
Hazard Prevention and Control 75% 3.7 0.32 Gaps
Training 64% 3.6 1.22 Gaps
Specific work procedures/JSAs
1 weakness ------ ------ ------ strength 5
% Score Standard Deviation Avg Score
GAPS
28
Visual Observation
Document
Review
Interviews Verify/Observe
Complete the
Protocol
Final Verification
29
Visual Observations
Take records (both accurate and suspect) and Re-survey.
Action Plans
Mishaps
Inspections
Meeting Minutes
JSA/Hazard Analysis
Each Program Element
Informally discuss process and closure with cognizant workers and supervisors
Visit areas and people: issues from interviews
Confirm or deny strengths and weaknesses.
30
Answering the Criteria
Document
Review
Interviews Verify/Observe
Answer the
Criteria
Final Assessment Step
31
Systemic or Random?
Verify if a process ensures safe behavior
Does the Process/Procedure:
Exist
Conflict with other processes or motivations
Lack documentation or communication
Lack motivation (metric)
Lack QC
Report in statistics, if possible:
“8 out of 10 reports were missing assignment of
responsibility”
32
Final Verification
To answer the criteria an evaluator:
Reviews Documents (D)
Performs Interviews (I)
Conducts Visual Observations (V)
Perform at least two of the three in
combination to fully verify your finding or
rating.
Compare your findings and evidence to the
Criteria to verify that the assessment is
complete and in enough detail.
33
Evaluation
The most comprehensive evaluation includes: reviewing written programs,
walking through the workplace, and
interviewing employees.
During this process participants should be answering the following questions relating to each element and sub-element of their safety and health management system:
Is it comprehensive?
Is it operating effectively and meeting established goals and objectives?
What improvements can be made to make it even more effective?
What goal modifications should be made for the upcoming year?
34
Annual Report
The Report will typically examine each program element from the following viewpoints:
A description of past activities and action plans from the prior year
A statement describing the current condition/status
An evaluation discussing strength and weaknesses to each program element
Corrective actions plans to resolve issues
Specific actions, responsible parties and target completion dates
Report in statistics or scores as a better way to communicate with Management
Appendix D
Rates, Significant Changes, Narrative, Success Stories
35
Calculating Your Rates
Two Tables
Site (employees and supervised temps and
contractors)
Applicable Contractors
Hours
36
Annual Report Outline
A: Summary
Site info, contractors, union (if applicable)
B: Injury and Illness Rate info
C: Significant Changes or Events
D: Narrative, each VPP Element
Ex: Incident investigation program
Narrative (strengths and weaknesses)
Status of last year’s improvement initiatives
Current next step improvements: assigned to. Target date
E: Summary Goals (last yr’s status and this yr)
F: Success Stories
37
Additional References/Resources
OSHA’s Draft Managing Worker Safety and Health, Chapter 12 Evaluating Your
Safety and Health Program (1991-1993)
http://www.labor.mo.gov/DLS/WorkplaceSafety/managing_safety.asp
Perception Surveys
National Safety Council
www.NSC.com
STAR Consultants
www.starconsultants.net
Assessment Tools
Preparing for the Voluntary Protection Programs
Margaret Richardson, Wiley Press @ 1999
STAR Consultants
VPP Self-Evaluation Manual
Esposito, P. “Building on Your VPP Success” Autumn 2010. The Leader,
Voluntary Protection Program Participants Association
Esposito, P. “Annual Self-Evaluations” published in the VPPPA Leader, Fall
2001
The Strategic Partner to help your
safety and health programs and risk
management!!!
Paul Esposito, CIH, CSP
410-218-8451
Questions?