Michael Barger

Benchmarking For Best Practice

What is Benchmarking?• A tool for continuous improvement

• Compliments Lean and Six Sigma activities

• Lean and Six Sigma projects develop improved processes

• Benchmarking finds best processes already developed

Benchmarking’s Roots• The term “benchmarking” was

originally a land surveyor’s term used in topographical surveys

• A benchmark was a distinctive mark made on a rock, a wall, or building that served as a reference point in determining one’s current position and altitude

• This was a sighting point from which additional measurements could be made

Definitions• Benchmarking: A strategic and analytic

process of continuously measuring an organization’s products, services and practices against a recognized leader

• Best Practices: A process or a practice that causes performance as measured by a specific metric to be superior

Best Practices Benchmarking• Best Practice Benchmarking is straight forward

for an organization that already uses metrics to track processes

• Best Practice Benchmarking can be used instead of a Six Sigma Project or can compliment one

Types of Benchmarking• Process benchmarking - investigates business

processes

• The goal to identify the best practices from other benchmark firms

• Activity analysis used to benchmark cost and efficiency

• Often applied to back-office processes

• Financial benchmarking - performing comparative financial analysis to assess overall competitiveness and productivity

• Performance benchmarking - competitive analysis by comparing products and services against target firms

Types of Benchmarking (continued)

• Product benchmarking – focused on product improvement

• Can involve reverse engineering which is taking apart competitors products to find strengths and weaknesses

• Strategic benchmarking - observes how others compete

• Usually not industry specific

• Best to look at other industries

• Functional benchmarking - focus on a single function to improve its operation

• Complex functions such as human resources, finance, accounting and IT are unlikely to be directly comparable

• May need to focus on sub-processes to make valid comparison

The Four Phase Benchmarking Model• Phase I: Plan

• Phase II: Do

• Phase III: Act

• Phase IV: Track

I: The Plan Phase

The Plan Phase• Select the process to be benchmarked

• Select and prepare the benchmarking team

• Identify benchmarking partners• Research information sources for best

practices• Rank potential partners [1-50]• Select the final partners [1-5]• Know & use the benchmarking guidelines for

ethical conduct

Establishing a Benchmarking Project• Motivation

• Team selection and building

• Framing the problem

• Select the process to be benchmarked

• Senior management buy-in

• Resource allocation

• Identify benchmarking partners

Motivation• Justification: Where is the pain?

• Resources

• Return on investment

• Result of current path

Building the Team• Establish the team and the role of the team to

align with the overall goals and objectives

• Identify formal and informal roles

• Team building process

Team Building Process• Careful member selection (skills, motivation,

personality, availability)

• Establish Ground Rules: The team’s role and responsibilities

• The organization’s role and responsibilities

• Other key issues in the team building process

• Develop a Team Charter

• Adopt the characteristics of a high performing team

A High Performing Team• Clear vision

• Full participation

• Clear Communication

• Process improvement plan

• Clearly defined roles

• Collaborative behavior

• Well-established Decision Process

• Established Norms

• Team Management Process

Team Introductions• Name

• Job title & position

• Professional/Career Goals

• General Background information (hobbies, marital status, family information, sports etc.)

• Knowledge/experience with benchmarking activities

• Two-minute interviews & briefings by colleagues

Formal Roles• Champion

• Team leader

• Team member

• Facilitator

• Senior management representative

• Information manager

• Technical expert

Team Process Roles• Team Leader

• Facilitator

• Scribe

• Timekeeper

Roles can be rotated

Team Documents• Team Charter

• Working Agreement

• Code of Conduct

• Team Roles and Processes

• Communications Plan

Communication Plan• Communication Plan is essential

• Establishes roles and responsibilities

• Addresses both internal and external information

• Documents how information will be communicated

(meetings, issues, deliverables, resources, project status, reports, etc.)

• Documents to whom information will be communicated

Communication Plan (Continued)

• Informs the leaders and others on what to expect in terms of how the teams BM project may impact other projects-efforts within the organization

• Creates an a sense of shared commitment & ground rules within team

• Encourages a culture of openness, trust and acceptance, and provides an opportunity to exchange information, ideas and information in a timely manner

Frame the Problem• Develop a problem statement

• Use an Ishikawa Diagram or 5 Why methods

• Establish a root cause

• Avoid presupposing the solution

• Establish a clear vision of the outcome

• Build consensus

Stakeholders• Identify who the stakeholders are in your

organization

• Reach a group consensus

• From the group consensus identify which stakeholders will support your benchmarking team

• Plan to brief these individuals on the approach, progress and the benchmarking findings at critical milestonesThese individuals can help determine the

success of the benchmarking effort.

Develop an Information Briefing• Vision

• Plan

• Methods

• Expected results

Selecting Benchmarking Partners• Research information sources for possible

partners• Dun and Bradstreet• Business lists• Personal knowledge• Current business partners

• Rank potential partners [1-50]• Establish selection criteria• Poll or survey the potential partners• Score information against selection criteria

• Select the final partners [1-5]

Partner Selection CriteriaSample criteria for identifying benchmarking partners:

• Is the benchmarking partner comparable financially (similar revenues, sales, profits)?

• Is the benchmarking partner comparable size (similar number of employees)?

• Does the partner engage in comparable functions (similar work process, methods, practices)?

• Does the partner have comparable outputs (similar products & services)?

Partner Selection Criteria (Continued)

• Does the partner have similar customer expectations?

• Does the partner have comparable organization and divisional structures?

• Does the partner have comparable inputs (supplies & ingredients)?

• Is the partner part of a comparable market sector (public, private, government)?

• Does the partner have comparable logistics (similar set-up & workflow)?

II: The Do Phase

The Do Phase• Determine the data collection plan

• Collect and analyze the data

• Train team members in skills & tools as necessary

• Determine performance gaps and strengths

• Analyze performance gaps & strengths

• Produce a benchmarking report

How will Data Be Collected?• What metrics will provide the data to compare

current and benchmarked processes?

• What kind of information or measurement is necessary? (accuracy, quality, customer satisfaction, speed, etc.)

• What data exists on the internal process?

• How should the team collect the internal data?

• Who on the team will collect the data?

How will the Data Be Collected? (Cont.)

• How will the team check its results?

• How much time will be needed to collect the data?

• How will the data be consolidated and analyzed?

• How will the satisfaction of the stakeholders be evaluated during the process?

• What method(s) should be used to collect data from the benchmarking partners?

Research Methods• Internet-based information searches

• Interviews/focus groups

• Archival data

• Observation

• Surveys

• Selecting and establishing metrics

• Internal experts

Some Examples of Metrics• Productivity, by transactions per unit

• Accuracy, by the error rates

• Speed, by the cycle time

• Product stability, by the number of engineering change orders per month

• Process financial contribution, by the value-to-cost ratio

• Product availability, by fill rate

• Product quality, by first-pass yield

• Capacity, by volume managed

• Service, by the on-time delivery

Considerations for Metrics• Reliability - consistent measurements

• Validity – measuring the right things

Reliability of MetricsEUI* for Five Manufacturing Plants for Three

MonthsPlant Month 1

Month 2

Month 3

Mean

A 5.5 6.2 3.6 5.1

B 3.2 5.0 2.9 3.7

C 5.1 4.3 5.8 5.1

D 1.3 4.2 4.5 3.3

E 5.1 2.0 1.1 2.7*Energy Use Intensity (Kwh/sf/month)

Note that the best plant cannot be determined. Analysis of Variance indicates that most of the variation is due to measurement error.

Validity of Metrics• Does it make sense to the experts?

• Does it have a strong relationship with other metrics that are similar?

• Does it have weak or dissimilar relationships with metrics that are different?

• Is it free from other influences that might be causing the results?

Sample Considerations• Sample Selection

• Nearest Neighbor Bias

• Success Bias

• Selection Bias

Sample Selection• Sample should be representative

• Processes should be comparable

• Should represent a range of practices

• Objective: Discover what differentiates best from the rest

Nearest Neighbor Bias

Look beyond organizations that you already know about. Include organizations that are unfamiliar, but relevant, if possible.

“There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns -- the ones we don't know we don't know. And if one looks throughout the history of our country and other free countries, it is the latter category that tend to be the difficult ones.”

-Donald Rumsfeld

Success Bias• Avoid selecting only highly successful firms

• A range of success is needed

• Differentiation is the key

Selection Bias• We are often unaware that we are sampling

the survivors

• This produces a biased sample

• The results may be misleading

• Finding results for organizations that did not survive can help understand what is differentiating a best practice

Data Collection• Establish the purpose for the data

• Define the type of data to be collected

• Determine when and how will data be collected

• Determine who will collect the data

• Control Charts

• Multi-attribute Utility Analysis

• Triangulation

• Errors to Avoid

• Check Sheets

• Histograms

• Pareto Analysis

• Scatter Diagrams

• Fishbone Analysis

Process and Tools

Some Pitfalls to Avoid• Small sample bias

• Attribution bias

• Correlation vs. causation

• Confirmatory Bias

• Groupthink

• Hidden Profiles

Small Sample Bias• Conclusions drawn from a small sample of

observations

• The observations may be outliers or random fluctuations

• Larger samples are needed to draw stable conclusions

Attribution Bias• An incorrect conclusion that A is causing B

because it stands out

• Example: innovation is attributed to a strategic planning process when the planning process has little to do with it

• Collecting more data and seeking the root causes can help avoid this bias

Correlation vs. Causation• Two variables are related but A does not

necessarily cause B

• Need to have a better root cause explanation than simply observing the relationship

• Seek out a clear rational explanation for the relationship to ensure causality

Confirmatory Bias• Once an initial idea is accepted we tend to

seek out only confirmatory information

• We avoid or disregard disconfirmatory information

• Can lead to invalid conclusions

• Facilitator and team should be aware of this tendency

Groupthink• A feature of groups, particularly when they are

cohesive

• Need a “devil’s advocate” to challenge too ready acceptance of ideas and conclusions

• Self-censorship and majority view does not necessarily indicate unanimous consensus

Hidden Profiles• Team members often bring different

information to meetings

• In many cases not all of the relevant information is shared

• Good facilitation can overcome this bias

Determine Gaps and Strengths• Analyze the gaps/strengths in your current

business process against your benchmarking partners and determine areas to target for improvement

• Do a performance gap analysis with a detailed comparison of the current process to the best-in-class

• List the best practices and the strengths where benchmarking partners display superior performance showing parity where there are no significant differences

Determine Performance Gaps (Cont.)

• Describe where your internal practices are superior to the benchmarked partners

• Produce the analysis necessary for the benchmarking report and prepare to make recommendations to the process owners based on that analysis

• Determine reasons for the gaps projecting any future competitive gaps

III: The Act Phase

ACT Phase - Implementation• Develop an Action Plan for implementation

• Communicate the Action Plan

• Execute the Action Plan

• Recalibrate benchmarks

Transfer/Implementation• Strategically and tactically plan how you will

incorporate best practice within your organization and record, analyze and evaluate findings

Implementation & Transferability Plan• Build awareness and communication

• Ensure that there are sufficient resources

• Build relationships between the source and the recipient of the knowledge

• Assure Timely implementation

• Link to a strategic plan

Develop an Action Plan

Plan How to:

• Achieve desired results

• Measure the results

• Monitor feedback on the process changes

• Identify tasks necessary to implement the process changes

• Identify necessary training

Obtain Stakeholder Approval The following should be asked before implementation:

• Is the plan clear?

• Does the plan show how gaps in performance will be closed?

• Will it lead to sufficient changes?

• Has the management supported the team and recognized its accomplishments?

• Is the organization ready to implement successfully the changes to the process?

• Is the process in place to implement the plan?

• Are there sufficient resources for the plan?

Implementation of Best Practices• Build awareness and communication

• Ensure sufficient time and resources

• Build relationships between the source and the recipient of the knowledge

• Adhere to a suitable schedule

• Strategically and tactically plan the implementation of the best practice

• Record, analyze, and evaluate findings

IV: The Track Phase

Track Phase – Monitoring the Implementation

• Review the Effectiveness of the Best Practice Implementation

• Modify the Action Plan as Necessary to Improve Implementation

• Review the Effectiveness of the Best Practice

• Continually Monitor Means for Improvement

Evaluation of Implementation• Follow-up Surveys

• Quasi-Experimental Designs

• Single-group interrupted time series design

• Interrupted time series design with control groups

Recalibrate Benchmarking• Need for advancement still exists

• Reset targets

• New values become internal measurements for the next benchmarking effort

• Establish a new process baseline

• Continue to monitor your current best practice against others

Monitor the Benchmarked Process • Confirm strategic alignment

• Continually monitor customer satisfaction

• Determine if any additional world-class process has emerged

Repeat the Cycle• Conduct ongoing visits with benchmarking

partners

• Consider recalibrating annually

• Perform the steps of the benchmarking model

Continuous

Improvement

PLAN

DO

ACT

TRACK