human performance maturity model model...organizational learning human performance is top-down...
TRANSCRIPT
Human Performance
Maturity Model
October 2, 2019
Pamela Ey - Resonance Labs
Michael Legatt – ResilientGrid
Bob Case – Black Hills Corporation
Deveny Bywaters – Knowledge Vine
Merrill Brimhall - WECC
2016
Training Maturity Model
July 2016 OTS Train-the-Trainer Workshop
Pamela Ey and Tracy North, Portland General Electric
2
Introduction to Maturity Models
3
2017
First HPMMTF meeting, August 7
4
HP Maturity Model Task Force Begins
Goals:
� What is a Human Performance Maturity Model?
� Research Paper
� Interview WECC Utilities
� Build the WECC HP Maturity Model
� White Paper
Optimizing behaviors to achieve goals, by
focusing on the intersection of the
complex system with the people doing the
work
5
Human Performance Definition
� “Maturity” – term that embraces
• Competency, Capability
• Capacity
• Level of Sophistication
� Maturity models – categorize a set of:
• Characteristics, attributes, indicators, patterns
� Provide benchmark for evaluating capability level of an organization’s practices, processes and methods
� Serve as a basis for setting improvement goals
6
Maturity Modeling Description
2018� What is a HP Maturity Model?
� Research Paper, authors Pamela Ey, Michael Legatt, Jeffrey Sill
� Interview WECC Utilities re: HP wraps up
� Create the WECC Human Performance Maturity Model from interviews
� White Paper – Writing begins
7
Scientific Based Approach
8
Maturity Model Levels
Level 1Exploring and Trying
Level 2Organizing and
Stabilizing
Level 3Defining and Structuring
Level 4Orchestrating, Integrating
and Measuring
Level 5Optimizing
� HP champion launches isolated efforts
� High risk/worst thing focus
� Symptoms vs causes are HP targets
� Counting as objective HP metric
� Flavor of the day HP skeptics
� Right direction but fragmented practices
� Tools for field personnel
9
Level 1- Exploring and Trying
� Recognizing the value of event analysis
� Emergence of Human Performance plan for training
� Recognizing the importance of organizational Human Performance culture
� Integrating the pre-conditions that will catch individuals
� Expanding beyond Human Performance champion to include additional stakeholders
10
Level 2 - Organizing and Stabilizing
� Focus on process (beyond individuals) and repeatable Human Performance practices
� Emerging corporate policy Human Performance drivers
� Communication across business units to establish Human Performance Structure
� Mitigating risks before they happen, rather than counting what has happened
� Identifying and acting on HP drivers
11
Level 3 - Defining and Structuring
� Human Performance aligned growth that is more than a targeted experiment
� Non-punitive culture to support near miss or issue reporting
� Integrating Human Performance processes across business units
� Using a control database for metrics
� Aligning, quantifying and qualifying contributing factors, risks, errors, and safety
� Predictable Human Performance outcomes
12
Level 4 – Orchestrating, Integrating and Measuring
� Fully integrated Human Performance practices enterprise wide, with no Human Performance silos
� Widespread understanding of Human Performance practices and approach
� Documented principles company-wide
� Human Performance integrated with business strategy, goals, vision, mission, values
� Organizational learning
� Human Performance is top-down approach, with business drivers, embedded in the culture, focusing on risks and outcomes
13
Level 5 - Optimizing
14
Maturity Model Levels
Level 1Exploring and Trying
Level 2Organizing and
Stabilizing
Level 3Defining and Structuring
Level 4Orchestrating, Integrating
and Measuring
Level 5Optimizing
2019� May: Started weekly pilot release
meetings
� June 25: OC approves MM White Paper
� July 25: IRB approves Pilot Survey
� July 26: Survey sent to 30 Pilot participants
� October: Survey results compiled
� Q4: Build Question Bank and Level Guidance
15
Maturity Model Pilot Development
16
HPMMTF Info Easy as 1-2-3
1
2
3
Launch WECC.org
1. Business decision making tool
2. Descriptive, outcome centered
3. Multiple uses
17
Maturity Model Value
Level 1Exploring and Trying
Level 2Organizing and
Stabilizing
Level 3Defining and Structuring
Level 4Orchestrating, Integrating
and Measuring
Level 5Optimizing
HP is a large landscape of tools, practices, disciplines, perspectives…
HP advocates are always working toward:
� Leadership buy-in
� Business case
� Decision making strategies –identifying biggest impacts/measurable small wins
18
Business Decision Making Tool
Level 1Exploring and Trying
Level 2Organizing and
Stabilizing
Level 3Defining and Structuring
Level 4Orchestrating, Integrating
and Measuring
Level 5Optimizing
Each maturity level has defined outcomes/goals.
Our model:
� Helps prioritize the attributes that will achieve the aims.
� Enables us to measure and clearly communicate progress toward goals for the outcome level.
� Uses descriptive behaviors at each level (rather than frequency/importance type scales)
19
Descriptive, Outcome Centered
Level 1Exploring and Trying
Level 2Organizing and
Stabilizing
Level 3Defining and Structuring
Level 4Orchestrating, Integrating
and Measuring
Level 5Optimizing
� Bottom up – Where are we?
� Top down – Where do we aspire to be?
� Budget/planning – Cost benefit of moving to next level
� Managing risks and tradeoffs– Risks of being at a level and identifying controls
� Setting expectations – Aligned priorities
� Communicating internally and externally – Common frame of reference
� Momentum and sustainability –Aligned attributes required to sustain at a level
20
Multiple uses
Level 1Exploring and Trying
Level 2Organizing and
Stabilizing
Level 3Defining and Structuring
Level 4Orchestrating, Integrating
and Measuring
Level 5Optimizing
� Building a survey that was easy to take (few
open text answers) and fast (< 15 minutes)
� Allows an understanding of an individual’s
view of their organization’s human
performance maturity, and overall the view
the industry has on itself
� Demographics, NPS, open text and
agree/disagree/NA questions
21
Survey Discussion
� Opportunity to grow the survey:• Repeat measures: tracking maturity over time for a
company/industry
• Consistency measures: tracking differing views of maturity within an organization
• Statistical analyses
◦ For example – relationship between components of human performance maturity
◦ For example – relationship between maturity and risks of human error
◦ Leading indicator / predictor
◦ Providing information to help target interventions
◦ Could be used in management
22
Survey Discussion
� Key to achieving goals
� We know that at the root of every switching procedure, relay setting, equipment specification and operating decision, there is a human – doing their job to keep the lights on.
� The Human Performance Work Groups, along with a myriad of other WECC Committees, recognize this.
23
Human Performance
24
WECC Committee Structure
� 2019 –• Approval of Whitepaper and Maturity Model
• Pilot Project to create Tool Set
• Question Bank for Maturity Level Assessment
� 2020 –• Refine Maturity Model and question bank
• Define transition behaviors
� 2021 –• Rollout– Make the model available to general audience
• Augment, and Validate
• Task Force Completion
25
HPMMTF Three Year Strategy
� Continue to Facilitate Committee meetings
� Encourage active and robust committee
participation
� Work with Parent Committees to approve
and disseminate Committee Results
� Use WECC website and other
communications tools as appropriate to make
results available to larger audience
26
How to Boost Success
� Active Participation (not just listening in)
� More Members to Contribute
� Broader Representation from Registered
Entities
� Participation in Pilot Surveys
� Disseminate Information from Committee or
Task Force and
� Enlist support from Industry
27
Committee Support
Survey Instructions
October 2, 2019
Pamela Ey – Resonance Labs
Michael Legatt – ResilientGrid
� Opportunity to participate in measuring
Human Performance Maturity across WECC
HP participants!
� Survey will take ~11 minutes, and can be
taken on a smart phone or laptop
� Results will be shared tomorrow!
29
Online Survey
� https://www.surveymonkey.com/r/wecchp
� Or bit.ly/wecchp
30
Online Survey
Contact:Pamela Ey - Resonance Labs – [email protected]
Michael Legatt – ResilientGrid – [email protected]
Bob Case – Black Hills Corporation –[email protected]
Deveny Bywaters – Knowledge Vine –[email protected]
Merrill Brimhall – WECC – [email protected]
31
Human Performance
Maturity Model
Follow Up
October 3, 2019
Pamela Ey - Resonance Labs
Michael Legatt – ResilientGrid
Thank you for your participation
yesterday!
Survey results were analyzed
A few concepts before the results:
2
Survey Overview
Humans face cognitive biases when
analyzing their capabilities and status.
Some examples of “positive illusion”
cognitive biases:
Dunning-Kruger Effect
Illusion of Explanatory Depth
Work as done vs. imagined3
Cognitive Biases
Humans face cognitive biases when
analyzing their capabilities and status.
Some examples of “positive illusion”
cognitive biases:
Dunning-Kruger Effect
Illusion of Explanatory Effect
Work as done vs. imagined4
Cognitive Biases
5
It Affects All of Us…
THREAT!!!!
Scanning
6
It Affects All of Us…
“The first step is to measure whatever can be easily
measured.
This is OK as far as it goes.
The second step is to disregard that which can’t be easily
measured or to give it an arbitrary quantitative value.
This is artificial and misleading.
7
Setting the Stage
The third step is to presume that what cannot be
measured easily isn’t really important.
This is blindness.
The fourth step is to say that what can’t be easily
measured doesn’t really exist.
This is suicide.”
Daniel Yankelovich, Corporate Priorities (1972)
8
Setting the Stage
Promoters: 9-10
Detractors: 0-6
NPS: 100 * (Promoters - Detractors)/Total
9
Net Promoter Score Overview
10
Net Promoter Score Analysis
-100 +1000
11
Net Promoter Score Analysis
-100 +1000
-45OVERALL
12
Net Promoter Score Analysis
-100 +1000
Management-31
-38 Training
Field Operations
-63
-67 Real-TimeOperations
13
Net Promoter Score Analysis
-100 +1000
Management-31
-38 Training
Field Operations
-63
Field Operations
3y+-100
-83Real-Time Operations
3y+
Management3y+-26
0Training
3y+
-67 Real-TimeOperations
Detractors:
25% at HPMM Levels 4 or 5
Promoters:
33% at HPMM Levels of 1 or 2
What does this mean for us?
14
Net Promoter Score
Average Maturity Levels by themes:
Event Analysis/Risk Analysis 3.53
HP Drivers 3.25
Metrics and Measures 2.68
Level of Integration 1.76
15
Net Promoter Score
“Our human practices are fully integrated
across the organization with no silos”
3 agreed, 66 disagreed
88% responded
16
Key Takeaways
A few questions to be reworded/removed
as they had limited value.
“We have an organization-wide value of
organizational learning”
19 agree, 2 disagree
88% responded
(original responses)
17
Key Takeaways
...or do we?
“We have an organization-wide value of
organizational learning”
47 agree, 24 disagree
88% responded
(yesterday’s responses)
18
Key Takeaways
A few questions to be reworded/removed
as they had limited value.
“We have an organization-wide value of
continuous improvement”
19 agree, 2 disagree
88% responded
(original responses)
19
Key Takeaways
...or do we?
“We have an organization-wide value of
continuous improvement”
60 agree, 16 disagree
92% responded
(yesterday’s responses)
20
Key Takeaways
Contact:resilientgrid.com/hpmm
21