critical lessons learned in the content and delivery of ... · critical lessons learned, in both...

SEI Webinar:SEI Webinar:Critical Lessons Learned in theLearned in the Content and Delivery f Si Si T i iof Six Sigma Training

Robert StoddardRobert StoddardDave Zubrow

June 30, 2010

SEI Webinar: Critical Lessons Learned in the Content and Delivery of Six Sigma Training30 June 2010© 2010 Carnegie Mellon University


NO WARRANTYTHIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERINGINSTITUTE MATERIAL IS FURNISHED ON AN “AS-IS" BASIS. CARNEGIEMELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHEREXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT, ,LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE ORMERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USEOF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKEANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROMPATENT TRADEMARK OR COPYRIGHT INFRINGEMENTPATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT.Use of any trademarks in this presentation is not intended in any way toinfringe on the rights of the trademark holder.This Presentation may be reproduced in its entirety without modification andThis Presentation may be reproduced in its entirety, without modification, andfreely distributed in written or electronic form without requesting formalpermission. Permission is required for any other use. Requests forpermission should be directed to the Software Engineering Institute [email protected] @This work was created in the performance of Federal Government ContractNumber FA8721-05-C-0003 with Carnegie Mellon University for the operationof the Software Engineering Institute, a federally funded research anddevelopment center The Government of the United States has a royalty-freedevelopment center. The Government of the United States has a royalty freegovernment-purpose license to use, duplicate, or disclose the work, in wholeor in part and in any manner, and to have or permit others to do so, forgovernment purposes pursuant to the copyright license under the clause at252.227-7013.

` SEI Webinar: Critical Lessons Learned in the Content and Delivery of Six Sigma Training30 June 2010© 2010 Carnegie Mellon University

The Co-Authors

Robert Stoddard

Senior Member of Tech Staff

Dave Zubrow

Senior Member of Tech Staff

SEI SEMA* SEI SEMA*

*SEI SEMA = Software Engineering Institute,

Software Engineering Measurement and Analysis

Polling Question 1

H did h b hi bi ?How did you hear about this webinar?

1 Email invitation from the SEI1. Email invitation from the SEI2. SEI Website3. Website with webinar calendar (ie www.webinar-directory.com)4. Social Media site ( LinkedIn, Twitter)5. Other

Today’s Talk

Introduction

Six Sigma Training Challenge

History of Six Sigma Training

Lessons Learned from SEI Research and Experience

•Training Design

•Training Implementation

SEI Six Sigma Training Solutions

Contact Info and Questions

5


Introduction

A wide variety of sources of Six Sigma training abound making it difficult for organizational leaders to identify which training and Six Sigma implementations are best for their work situations.

Additional confusion arises due to different philosophies on how to best train practitioners and domain experts in applied Six Sigma methods.train practitioners and domain experts in applied Six Sigma methods.

6


Six Sigma Training Challenge

Critical lessons learned, in both the content and delivery of Six Sigma training, have been identified by SEI researchers from their experiences with several large corporations.

The Challenge:

Successful adoption and usage of Six Sigma for tactical process improvement

versus

a failed investment in Six Sigma training that i l f l d l ti l i t d ’is less useful and less practical in today’s fast-paced business environment.

7


History of Six Sigma Training

1980’s – Traditional statistical training to manufacturing personnel; focus on manufacturing operational scorecards

1990’s – Traditional statistical training with COTS statistical tools; focus g ;moves beyond scorecards to improvement project methodology

Early 2000’s Traditional statistical training to non manufacturingEarly 2000 s – Traditional statistical training to non-manufacturing domains; focus on new product design, development and transaction

L t 2000’ A li d t ti ti l t i i ith d t l t idLate 2000’s – Applied statistical training with modern tools to a wide variety of domains; focus on training domain experts use of statistics rather than training statisticians in domain knowledge

8


Lessons Learned from SEI Research and ExperienceExperience

Critical lessons learned have been identified in both the content and delivery of Six Sigma training

Identified up to 8 years ago, most organizational deployment champions p y g , g p y pand Six Sigma training providers seem to be unaware of the lessons learned

As such, inferior training and improper use of Six Sigma methods and tools persists in the community, resulting in false starts and disappointing b i ltbusiness results

9


Polling Question 2

How would you characterize your organization’s Six Sigma methods and tools training?

1. Very good

2. Above Average

3. Below Average

4. Minimal

5. Don’t know

10


Training Design


1) Theory & Formula vs What, When & Why

Traditional Approach:

Traditional Six Sigma training forces students to learn the formulas and theory behind each techniquetechnique.

Often, this can include details of the original derivation of the technique.derivation of the technique.

Lesson Learned:

Taking advantage of both modern statistical packages and hands-onTaking advantage of both modern statistical packages and hands-on coaching, Six Sigma training should properly focus class time on connecting the business problem space with the appropriate technique or methodmethod.

Experience has shown that busy practitioners will not use the methods if their training focuses mostly on the theory and formulas

12


their training focuses mostly on the theory and formulas.

2) Statistical Purism vs Practical Use


Often, Six Sigma training still gets “wrapped around the axle” regarding statistical purism.

Thus, the training appears to ignore the fact that we are training practitioners, not college statistical students!statistical students!

Lesson Learned:

Six Sigma courses should teach only what is needed for busy practitionersSix Sigma courses should teach only what is needed for busy practitioners to use the methods, with occasional help from a coach.

An example is the on-going discussion within the Six Sigma community p g g g yabout how religious one treats the prerequisite of normality before using a number of methods, including SPC.

13


3) Theoretical Examples vs Real World Problems


Many Six Sigma courses primarily offer thought experiments and theoretical examples related to the methods and toolsmethods and tools.

This behavior actually blurs the lines of distinction between skill training vs education vs generalbetween skill training vs education vs general awareness.

Lesson Learned:Lesson Learned:

Six Sigma training should focus on skill building through a combination of real-world and realistic case studies.

Thus, the students can learn the skills in context of specific business situations and problems.

14


4) Manufacturing vs Software and Systems


Because Six Sigma training originated in the manufacturing domain, many training providers still teach the methods strictly in context of ymanufacturing.

Consequently, many organizations have seen their ft i ti th li bilit fsoftware engineers question the applicability of

Six Sigma to their work.

Lesson Learned:

Six Sigma courses should stress the application of Six Sigma methods and tools among a ide ariet of areas s ch as b t not limited to netools among a wide variety of areas, such as but not limited to: new product development, research, transaction processing and service, as well as, strategic planning and technical product marketing.

15


5) Continuous Data vs Continuous & Discrete Data


Most Six Sigma training providers only teach methods conducive for continuous data types while excluding treatment of discrete or categorical datatreatment of discrete or categorical data.

This arises because the original manufacturing application had primarily continuous data (Interval or Ratio).

Lesson Learned:

We recognize that software-intensive systems complex systems andWe recognize that software-intensive systems, complex systems and systems-of-systems development need to deal with all types of data. In fact, experience has shown that a majority of the data to be analyzed ma be categorical (Nominal or Ordinal) in nat remay be categorical (Nominal or Ordinal) in nature.

Examples: Defect type = Nominal; Severity score = Ordinal

16


6) Domain Agnostic vs CMMI & Domain Specific


Some Six Sigma training providers create training materials to be reused across industries and domainsindustries and domains.

As a result, the concepts and skills are not taught in the domain language of the student!taught in the domain language of the student!

Lesson Learned:

Six Sigma students should learn skills applicable within their specific domainSix Sigma students should learn skills applicable within their specific domain and, beneficially, against a backdrop of the CMMI constellations (Development, Acquisition, and Services).

Case studies from a number of domains should be included to ensure coverage of all students’ backgrounds.

17


Polling Question 3

To what degree would these training content lessons-learned benefit your organization?

1. Very much

2. Often

3. Somewhat

4. Minimal

5. Don’t know

18


Training Implementation


1) One Instructor vs Two Instructor Paradigm


A majority of Six Sigma training conducted today is via a single instructor and often via online or elearning.

As a result the learning experience is significantlyAs a result, the learning experience is significantly constrained by the single instructor’s experience and limited interaction with students, if any interaction at all!

Lesson Learned:

We believe a two-instructor approach is superior for severalWe believe a two instructor approach is superior for several reasons:

1) While one instructor is lecturing and leading the class, the th i t t lk d t h l ith t l dother instructor can walk around to help with tool use, and

2) The two-instructor approach dramatically enriches the learning experience via complementary instructor backgrounds.

20


p p y g

2) Statistical Prerequisites vs No Prerequisites


Many Six Sigma training courses have a deeply-rooted structure of prerequisite courses, often requiring college classes in statisticscollege classes in statistics.

As such, many domain experts find themselves excluded resulting in classes filled mostly with students from theresulting in classes filled mostly with students from the quality function.

Lesson Learned:

Because domain knowledge has been shown to be more important than statistics in driving process improvement , we have seen organizations prioriti e the class seats for the domain st dents ho ma ha e no priorprioritize the class seats for the domain students who may have no prior knowledge of statistics.

21


3) Manual Calculation vs Modern Tools


Most Six Sigma training offerings are still locked in the past, e.g. 1980’s.

They still teach students to perform heavy calculations on calculators or program formulas in spreadsheets.

Lesson Learned:

A healthy set of modern tools are now available to conduct applied statistical analysis and report the results.

A such, we believe that the teaching focus on manual calculations is a serio s distraction from the real foc s connecting the problem spaceserious distraction from the real focus: connecting the problem space with methods and tools!

22


4) Lecture vs Hands-on Practice


Much of today’s Six Sigma training is still rooted in traditional classroom lecture, or unassisted elearningelearning.

Both approaches have been shown to be inferior to hands-on, coached skill training.to hands on, coached skill training.

Lesson Learned:

We believe a combined use of lecture, individual and group exercises, and interactive use of modern tools is superior for skill building.

A seemingly superior class design guide is to limit lecture to no more than 30-40% of the class time.

23


5) “Use it or Lose it” vs “Self-Refresher” Screen ShotsShots


Traditional Six Sigma training often includes training materials that do not stand alone.

As such, students often find the need to retake the training later on when the opportunity to apply a given method or tool arises!given method or tool arises!

Lesson Learned:

Six Sigma training should include rich information, decision flowcharts and actual screen shots of the step-by-step process for each method using a modern tool.

In this fashion, students may interactively follow along during class, as well as, self-refresh on topics later on when needed.

24


6) Memorization vs Laminated Color Job Aids


To this day, most Six Sigma training involves memorizing formulas, as well as, which techniques to use in different situations based on the type of available datasituations based on the type of available data.

This memorization requirement continues to stall Six Sigma deployments.deployments.

Lesson Learned:

A number of Six Sigma courses use quick reference job aids whichA number of Six Sigma courses use quick reference job aids, which embody most of the decisions related to techniques based on the situational factors.

At the SEI, we have created several laminated, color job aids that are in great demand by our students.

25


7) Standard Data Sets vs Additional Student Data SetsSets


It is common for Six Sigma training to provide data sets to be analyzed as part of the skill building.

However, these data sets are often narrowly constrained and unrealistic to the situations that the student will encounter!the student will encounter!

Lesson Learned:

Students should not only utilize prepared data sets to practice the different techniques, but should be encouraged to bring their own organizational and/or project data to analyzeand/or project data to analyze.

In this manner, students can build compelling predictive models for their workplace needs during the class!

26


p g

8) Book Knowledge vs Self-Sufficiency


Regretfully, many training providers, coaches and practitioners seem to be only book knowledgeable of Six Sigma techniquesSix Sigma techniques.

As such, they are disadvantaged wrt the operation and strength of the different techniques.strength of the different techniques.

Lesson Learned:

Six Sigma training should be designed to build self-sufficiency withinSix Sigma training should be designed to build self-sufficiency within students through the repeated practice of individual analytical techniques and tools.

We believe skill building of this nature creates more self-sufficient students who need less coaching and are more confident in applying the techniques within the workplace

27


techniques within the workplace.

9) One time event vs On-going coaching


Most Six Sigma training is designed as a one-time training event, in which the students must go back to the workplace and apply the concepts on their ownthe workplace and apply the concepts on their own.

Even the best training suffers without the appropriate “hand-holding” during first real use.hand holding during first real use.

Lesson Learned:

Six Sigma students should be encouraged to solicit coaching for on-goingSix Sigma students should be encouraged to solicit coaching for on-going advice and help, as this psychologically promotes their use of the methods without fear.

Superior Six Sigma deployments achieve coaching structures within each of the major domains in their organization, such that practitioners are coached by someone in their domain

28


coached by someone in their domain.

Polling Question 4

To what degree would these training implementation lessons-learned benefit your organization?

1. Very much

2. Often

3. Somewhat

4. Minimal

5. Don’t know

29


SEI Six Sigma Training Solutions

The SEI offers two courses that embody the lessons learned described in this talk.

• Improving Process Performance using Six Sigma (IPPSS)Improving Process Performance using Six Sigma (IPPSS)

• Designing Products and Processes using Six Sigma (DPPSS)

30


IPPSS Course Details

Title: Improving Process Performance using Six Sigma (IPPSS) (5 days)

SEI url: http://www.sei.cmu.edu/training/p49b.cfm

Aka: Green Belt training for Software and Systems Engineers

Topics included:

1. Full treatment of the Six Sigma DMAIC process

2. Analysis using ANOVA and Hypothesis Testing

3. Sample size determination

4. Predictive modeling using Regression modeling

5. Role of these methods specifically related to the following CMMI-Dev 1.2 Process Areas: MA, QPM, OPP, CAR and OID

31


DPPSS Course Details

Title: Designing Products and Processes using Six Sigma (DPPSS) (5 days)

SEI url: http://www.sei.cmu.edu/training/p56b.cfm

Aka: Black Belt training for Software and Systems Engineers

Topics included:1 Treatment of the Six Sigma DMADV process1. Treatment of the Six Sigma DMADV process

2. Voice of Customer techniques including KJ and Kano analysis

3 Monte Carlo simulation and Optimization modeling3. Monte Carlo simulation and Optimization modeling

4. Predictive modeling using Dummy Variable and Logistic Regression modeling

5 Process Modeling using Discrete Event Simulation5. Process Modeling using Discrete Event Simulation

6. Design of Experiments for practical, short experiments, and

7. Software Reliability Growth Modeling and prediction of latent defects.

32


y g p

Polling Question 5

Which of these SEI courses seem attractive and timely to meet your organization’s needs?

1. Improving Process Performance Using Six Sigma (IPPSS)

2. Designing Products and Processes Using Six Sigma (DPPSS)

3. Both of the above

4. Neither of the above

5. Don’t know

33


Open Question

Any additional questions or comments on the critical lessons learned in the content and delivery of Six Sigma training?

34


Thank You for Your Attention!

Robert StoddardEmail: rws@sei cmu edu

U.S. mail:Software Engineering InstituteEmail: [email protected]

Dave ZubrowE il d @ i d

Software Engineering InstituteCustomer Relations4500 Fifth Avenue

Email: [email protected], PA 15213-2612USA

World Wide Web:i d

Customer RelationsE il twww.sei.cmu.edu Email: [email protected]: +1 412-268-5800SEI Phone: +1 412-268-5800SEI Fax: +1 412-268-6257

35


Are you measuring the data you need?y g ySEI Measurement and Analysis training

i ith th kill d dequips you with the skills needed.

SEI Trainingwww.sei.cmu.edu/training

36


Analyzing Project Management IndicatorsThree-day course teaching managers and practitioners how to use measurement as the foundation for making informed decisions about projects.

July 13 – 15, 2010 (SEI Arlington, VA)

Register Now:www.sei.cmu.edu/training/p07d.cfm

37


D i i P d t d PDesigning Products and Processes Using Six Sigmag gFive-day course taking a hands-on, measurement-oriented approach to product development, providing participants with a toolkit of analysis methods and leveraging industry best practicesmethods and leveraging industry best practices.

December 13 – 17, 2010 (SEI Arlington, VA)

Register Now:www.sei.cmu.edu/training/p56b.cfm

38


Implementing CMMI for High Performance, and Executive SeminarOne-day introductory course for executives and middle managers covers the key concepts and strategy behind the SEI’s new fast track to process improvement from a management perspectiveto process improvement from a management perspective.

October 18, 2010 (SEI Arlington, VA)


39


Implementing Goal-Driven MeasurementThree-day course teaching managers and practitioners how toThree-day course teaching managers and practitioners how to identify and define indicators and measures that directly support an organization’s business, process improvement, and project goals.

September 21 – 23, 2010 (SEI Arlington, VA)

Register Now:www.sei.cmu.edu/training/p06.cfm

40


Improving Process PerformanceImproving Process Performance Using Six SigmaFive-day course taking a hands-on, problem-solving approach to data analysis, providing participants with a toolkit of analysis methods and leveraging industry best practices such as Six Sigmamethods and leveraging industry best practices such as Six Sigma and Goal-Driven Software Measurement.

August 16 20 2010 (SEI Pittsburgh PA)August 16 – 20, 2010 (SEI Pittsburgh PA)


41


Start here

Webinar start

42


critical lessons learned in the content and delivery of ... · critical lessons learned, in both...

Documents