aice mar07 deming_workshop_master

© 2007 Breakthrough Systems 1

Deming’s Red Bead Experiment

AICE Quality ConferenceTuesday, 06MAR07

Jim ClausonBreakthrough Systems

http://jclauson.com/aice


Why Are We Here?

To consider:

• what the Red Bead Experiment is,

• what it has to do with quality and

• how it can have an impact on your everyday quality activities


Session Agenda

The Red Bead Experiment Numeracy System of Profound Knowledge Finding Red beads Impact of psychology Charting red beads “What will you do on Monday?”


Introductions and Expectations

• If you are sitting with someone you know, please move

• You will interview, then introduce the person next to you

• Who they are, something unique or interesting, what industry they represent, and what their expectations are today


Run Red Bead Experiment

• Click [here] for the red bead slides.


Review: Why Are We Here?

To consider:

• what the Red Bead Experiment is,

• what it has to do with quality and

• how it can have an impact on your everyday quality activities


Beyond Red Beads

• Red Beads Everywhere, oh my!!

• Finding those red beads– Red, white and non-red?

• Quantifying those read beads– Hit by a car –vs- bump a file cabinet

• Eliminating those red beads– 99.99999999999999999999999999%


Numeracy

• an ability to handle numbers and other mathematical concepts

• in the US, it is somewhat better known as Quantitative Literacy

• innumeracy is a lack of numeracy


3 Kinds of Numbers for Management.

• Facts of life. If we don't make this profit figure, we will go out of business.

• Planning, prediction and budget. Can be used to compare alternative plans.

• Arbitrary numerical targets. Generally used to judge workers.

Avoid the use of the 3rd kind of number

Henry Neave The Deming Dimension


Data Sanity

We can either react to numbers, with explanations of every percent change, with the inherent frustrations, fear, and failure

OrWe can understand our data, put it to good use,

and apply valid management principles

The choice is ours.


Through the Lens of SoPK

System of Profound Knowledge

• Appreciation for a system• Knowledge about variation• Theory of Knowledge• Psychology

From The New Economics, Deming


1 of 4: Appreciation for a System

• Pay attention to interactions more so than components

• Knowledge of statistical variation more so than discrete numbers

• Long term focus more so than short term

• Cooperation more so than fear, blame and internal competition


1 of 4: Appreciation for a System - II

• “94% of the outcome of any organization comes from the processes used, not the people”.

• “A fault in the interpretation of observations, seen everywhere, is to suppose that every event is attributable to someone (usually the one closest at hand), or is related to some special event. The fact is that most troubles with service and production lie in the system and not the people”.


2 of 4: Knowledge of Variation

• You have experienced the Red Bead Experiment

• The Theory of Variation is at the core of cost savings, Kaizen, 6 sigma…

• Dr. Deming’s early works focused on statistical variation. He added the rest of the SOPK in the last 10 years of his life.

• Stable System versus Unstable System


Variation: Deterministic –vs- Probabilistic

• Deterministic - linear, cause and effect sequences. If you do this, that will happen.

• Probabilistic - exact time, location, and effect is random. e.g. Number of Red Beads.

• Treating a probabilistic result as if it was deterministic will cause problems

• Past results will not guarantee future results


Variation: Can we predict?

• Engineers often predict accidents. Their predictions are uncanny for correctness in detail. They fail in only one way – they can not predict exactly when the accident will happen.

- Dr. Deming, Out of the Crisis page 479• Calculations after the fact, using only data available

prior to the disaster, showed there was greater than a 10% chance of the Challenger explosion occurring, given the pre-launch temperatures and prior history of O-ring burn through.


3 of 4: Theory of Knowledge - I

• Knowledge is based upon prediction

• Knowledge is built on theory– Chanticleer the barnyard rooster

– Actions taken without theory lead to losses

• Use of data requires prediction

• There is no true value of a measurement, it depends on methods, context, and use

• Operational definitions are necessary


Operational Definitions - I

• “Clean the table…


Operational Definitions - II

• Most arguments about conflicting data come down to the definition of how to count the data

• Try to be precise in your definitions, but likely something unforeseen will arise

• The beads were:– red & white or red, white & non-red?


Deming said…

• “It’s absolutely vital for business that you settle this method of counting, measuring, definition of faults, mistake, defect, before you do business. It’s too late afterwards”

-Dr. W. Edwards Deming


4 of 4: Psychology

• Extrinsic versus intrinsic motivation

• People will use the charts you make - up and down the organization

• If you do not understand the people & understand psychology, the charts will be ignored

• Competition, fear, perceptions, loss of control change the data and the chart’s message


Through the Lens of SoPK

System of Profound Knowledge

• Appreciation for a system• Knowledge about variation• Theory of Knowledge• Psychology• Thinking about all 4, we’ll concentrate on

identifying and quantifying variation


Red Beads: Find, Quantify, Reduce


Take a Step Back: Systems Thinking

• Process viewed as a system

• SIPOC

• The “new” 4M’s

• Psychology: Suboptimization

• Remember the “94/6 Rule”

• Using SPC


Production Viewed as a System


SIPOC

• Supplier

• Input

• Process

• Output

• Customer


The “new” 4M’s

• Old: man, machine, material, method• Measurement added• Person or people replaces man• Equipment replaces machine• Supplies is used for material• Process is used for method• Environment added

– Physical and mental


Group Activity - IV

• As individuals, draw a SIPOC diagram for your job

• See if you can identify all of the new 4M’s as inputs to your process

• Compare and discuss as teams

• Choose the 3 most interesting to report

• Save this work, it will be used later


Psychology: Suboptimization

• We assume that optimizing a system considers all the sub-parts

• One unit may be selfish and take an action that makes them look good, but hurts others

• One process may shift problems down the line to let others have to worry about it


No Gold Stars Here

• Awards, bonuses, gold stars can actually have a detrimental impact

• For a person driven extrinsically, each subsequent reward must be larger in order to have the same impact

• Creation of winners and losers

http://www.alfiekohn.org/index.html


Group Activity - V

Game: Win As Much As You Can

A Decision Making Exercise, Illustrating the Effects of Human Behaviors and Psychology on Performance Measures

[link] to game


The “94/6%” Rule

• It is critical to separate system causes from individual causes of variation

• Deming started at 85% systems and 15% worker and had moved up to 96% and 4% by his death

• What are the implications of the 96/4% Rule?


Breather – We OK?

• What have we addressed?

• Numeracy

• System of Profound Knowledge

• Looking at your work as a system

• Suboptimization exercise

• 96/4% Rule

• We OK?


Statistical Process Control

• Control Charting provides knowledge of variation

• a lens, providing a different way of viewing the world

• a significantly different view of what is happening than will other methods


A Basic Control Chart

Injuries per Month - as a Control Chart

0

5

10

15

20

25

Jan-

03

Mar

-03

May

-03

Jul-0

3

Sep

-03

Nov

-03

Jan-

04

Mar

-04

May

-04

Jul-0

4

Sep

-04

Nov

-04

Jan-

05

Mar

-05

May

-05


SPC Basics

• Common -vs- special

• System in control?

• System predictable?

• What about psychology?


Charting the Red Beads

• In the Red Bead Experiment, we reacted to the random noise from result to result.

• Rewards, punishments, ranking of the workers, feedback to the workers had no effect on the results of the process.

• The process was stable and needed to be changed!


Stable –vs- Unstable

• Stable processes contain only common cause variations and are predictable

• Unstable processes contain special cause variations and are not predictable.

• Only predictable processes may be used to plan effectively


Cause: Common –vs- special

Special Cause Variation: If a statistically significant trend occurs, find the

special cause of this trend. Use this information to correct or reinforce these special causes.

Common Cause Variation: If no trends exist, you must look at the long run

performance of the process and fundamentally change the process in order to improve the process.


Defining Trending in Charting

• One point outside the control limits• Two out of Three points two standard deviations

above/below average• Four out of Five points one standard deviation

above/below average• Seven points in a row all above/below average• Ten out of Eleven points in a row all above/below

average• Seven points in a row all increasing/decreasing


Trending Example on Control Chart


Constructing a Control Chart

• Plot the actual data by month (or whatever time interval you are using)

• Plot at least 25 points (when available)

• Calculate a baseline average rate

• Add 3 standard deviation control limits

• Incorporate a set of trend rules


Why 3 Standard Deviations?

• Dr. Shewhart established 3 standard deviations as an economic balance between failure to detect and false alarms.

• Economic Control of Quality of Manufactured Product (1931!)


Just in Case…

• Many courses incorrectly teach that the control limits cover 99.7% of the normal distribution

• Not all data are normal, “real data” can cause the rate to be as low as 95% (Dr. Wheeler)

• The Tchebychev Inequality states up to 11% can be outside three standard deviations


Breather – We OK?

• stable –vs- unstable

• common –vs- special

• constructing a control chart

(or a process behavior chart)

• 3 Standard Deviation limits


Choosing Performance Indicators


Group Activity – VI

• As individuals, take your SIPOC & 4M exercise, and– Identify a performance indicator on the input and

then the output of the process you have diagrammed and explain why you chose it

– Share and discuss your choice with your team– Choose 2 or 3 examples from the team to share

with the class– Remember: you are hunting red beads


Activity Debrief

• What indicators did you choose?

• Why?

• What actions would you take if one developed an adverse trend?


Choosing the Right Measures

“Managers who don’t know how to measure what they want settle for wanting what they can measure.”

Dr. Russ Ackoff


Performance Indicator Introduction

• It is more important how the measure is used than what the measure is

• Self-fulfilling prophecies can prevent us from gathering any data

• We are drowning in data, but little knowledge is derived

• Context and Operational Definitions are crucial


5 Critical Issues ( 1-3)

• Managers suffer from overabundance of irrelevant information.

• Managers don’t know what information they need. Need to look at the decision process to determine this.

• Even if given the information they need, decision making will not necessarily improve


5 Critical Issues (4-5)

• More communication does not necessarily lead to better performance. Information can be used destructively.

• Managers do need to know how the information system works. Just because it came from a computer doesn’t mean it is right.


Designing a Management System

• The information system should be designed as an integral part of the management system

• Most information systems are designed independently, leading to failure

• Information systems should serve management, not vice versa


PI Barriers

• higher ups will use it as a “hammer”• subjected to quotas and targets imposed from

above• fear (“accountability”) used as a “motivator”• actions and explanations as a result of

random fluctuations• perceived loss of control over portrayal of

performance• must develop “perfect” indicator the first time• use of SPC can minimize these fears


Ackoff on Performance Indicators

• We do need to know the context within which the performance indicators will be used

• Forecasting and living with the forecasted future is important, but what about designing a better future?


Context of PI

• Do not look at a chart in a vacuum

• Reconcile any differences between the data and “gut feeling”

• Combine experience and the data

• Lessons from the data should lead to insight in the field, and vice versa


PI Evolution

As a process matures, one may end up evolving the indicators used. For example, if interested in completing actions by commitment dates, one may end up using (as the process matures):

• Percent of Actions completed by due date in effect at time of completion

• Percent of Actions completed without missing any due dates during their life

• Percent of Actions completed by the original due date

• Average days Actions completed ahead of original due date


Trying for the “Perfect” PI

• When committees get together and try to table-top the perfect indicator, paralysis often sets in.

• Realize all data are flawed, there is no “true value”, indicators can always be “gamed.”

• Putting the right culture of HOW to use performance indicators in place minimizes adverse impacts.

• Gain experience with simple indicators, then move on to more complex indicators if needed.

• With proper analysis, flaws with existing data can be detected and fixed. If you never look at the data, there will never be an incentive to fix the data.


Just Do It!

• All data are flawed• Make good use of your data• Endless conference table

discussions won’t cause any data to appear

• Initial prototype successes will lead to experience, and will further the spread of the use of indicators


Data Gathering

• Plan ahead

• Establish Operational Definitions

• Check data quality

• Avoid bias

• K.I.S.S.


Data Quality

• Data should be replicable• Operational Definitions are a must• Source Data must be defined• There is no “true value” of any measure, but a

good operational definition can save much trouble in the future

• ANYONE at ANYTIME in the future should be able to apply the same operational definition to the same source data, and get the same results.


Choose a Reporting Interval

• If a trend developed, how long could you go without needing to know it?

• Longer intervals imply more risk

• Need sufficient volume of points (25)

• Costs increase as reporting interval decreases

• What is current reporting interval?


Creating a Control Chart


Creating the Baseline

• The Baseline on a control chart consists of the average (center) line, a three-standard deviation Upper Control Limit (UCL) and a three-standard deviation Lower Control Limit (LCL).

• The Baseline allows us to predict the future, and evaluate for trends.


A Good Baseline

• A “good” baseline detects future trends with a minimum of false alarms.

• If a trend is detected, we don’t want it to be due to too few data points in the baseline, causing the baseline to have been inaccurate.


To get a Good Baseline

Do show all data, but change the average and control limit calculations by:

• Dropping data off of the beginning

• Dropping data off of the end

• Dropping individual datum point(s) and circling them


Dropping 1st 3 Points


Establish Expectations

• “Stable” performance is not necessarily good

• Management needs to determine if the current stable baseline is “acceptable” or “unacceptable”

• Recall that the # of red beads was unacceptable, but process was stable and in control


Monitoring

• Update charts on the required time interval• Check for trends against the trending rules• Circle any trends, inform owning

management and look for special cause(s)• Do not shift a baseline unless there is a trend

(baseline proven guilty)


That’s Charting Performance Indicators in a Nutshell

• PI as part of an overall management system

• What makes a good performance indicator

• Data gathering• Establishing a good baseline• Establishing Expectations• Monitoring


Review Control Chart of Red Bead Experiment


Session Summary

The Red Bead Experiment Numeracy System of Profound Knowledge Finding Red beads Impact of psychology Charting red beads “What will you do on Monday?”


What will you do on Monday?

• Well???


Questions?


Wrap & Roll…

Thanks for your time and attention

Jim Clauson

[email protected]

http://jclauson.com/aice

aice mar07 deming_workshop_master

Lifestyle

red beads red beads

red white

breakthrough systems12

breakthrough systems9

breakthrough systems2

breakthrough systems6

breakthrough systems8

breakthrough systems7