NEW EXAMINER ORIENTATION

Presented by The Partnership for Excellence

With special thanks to our hosts:

Citizens Energy Group, Indianapolis, INOCSEA, Westerville, OH

AGENDA

Morning

Welcome and Introductions

Course Overview

Process Overview and the

Examiner’s Role

Code of Ethical Conduct

Begin with the End in Mind

Structure of Criteria Item

Importance of Key Factors

Program and Pre Work Q&A

Afternoon

6 Step Evaluation Process

Pre-work Instructions

BOSS Demo

Wrap-Up

WELCOME &

INTRODUCTIONS

TPE EXAMINER’S ROLE & PROCESS

SEE TAB 2

Oct/Nov/Dec ’15 Dec ’15/Jan ’16 Late Jan/Feb March Apr/May May/Jun 1.1 Examiner

Applications due

September

2.1 Independent Review

TLs contact applicant’s POC to set

dates for Site Visit

3.1 Begin Consensus Review. TL

assigns both Category Lead and

Category Back-up responsibilities to

each examiner on the team.

4.1 Plan SV (draft

SVI worksheets, list

documents to examine,

walk-about questions,

interviews, etc.

5.1 TL and/or

Scorebook Editor

insert comments &

scores into FBR

template

6.1 TPE finalizes

FBR and sends to

applicant

1.2 Applicants

notified of

acceptance within 72

hours

2.2 Review the Framework

Booklet so that you are very

familiar with its contents. Assure

you are using the correct Booklet

for your application (i.e. Health

Care for Health Care, etc.)

3.2 For your assignment as

Category Lead, read the KFs, IR

Strength & OFI, evidence,

evaluation factors, comments,

scoring range/score from all team

members

4.2 Participate in 3-

day SV. Review

documents, interview

assigned staff, verify

and clarify SVI. Work

each SVI worksheet.

5.2 When & where

requested by TL,

Category Leads

contribute to

feedback report

6.2 Examiners are

honored at the Quest

for Success

Conference in

September

1.3 If a 1st time

examiner, attend

New Examiner

Orientation in Ohio

or Indiana in October

2.3 Do a first read of the

application: make notes in

margins, underline, highlight, etc.

(or in BOSS) of any important

information that relates directly to

the Criteria

3.3 Using strongest draft comments,

further develop a 4- 6 FB-ready

Strength & OFI comments for each

assigned Item. Re-score both range

& score using appropriate scoring

guidelines. Enter in BOSS by (date)

4.3 Re-write

comments concisely,

as needed, using

Comment Guidelines.

Walk-the-Wall.

5.3 TL completes

Score Summary

Worksheet

KEY

BOSS-Baldrige

Online Scorebook

Solution

1.4 Prework – Oct

(Time to complete

prework: 25-35

hours.) Examiners

submit completed

prework at ex trng.

2.4 Read the applicant’s OP and

identify KF for P1a, P1b, P2a,

P2b, and P2c. Develop a list of KF

in a word doc. Forward to TL.

3.4 Read the consensus comments

and scoring for Items in your backup

Category assignment and provide

feedback to the Category Lead by

email, phone, or in BOSS by (date).

Review the draft list of KT.

4.4 Draft SV

scorebook before

leaving site. Return all

applicant documents.

Sign Scorebook.

5.4 TL discusses

FBR and scores with

Lead Judge

ELRC – TPE’s

Examiner Learning

Resource Center

FB - Feedback

FBR-Feedback

Report

1.5 If invited as a

Team Leader, attend

Team Leader

Training late

October.

2.5 TL will consolidate KF into a

single list and return to you to

enter the KF in BOSS by (date).

Team members will use the

consolidated KF list going

forward.

3.5 Read the feedback provided by

your Category backup and revise

your Strength & OFI comments and

re-score as appropriate in BOSS by

(date)

4.5 Participate in

closing meeting after

which there are no

further questions or

interviews with

applicant.

5.5 TL makes

revisions per

discussion with

Lead Judge

IR -Independent

Review

KF – Key Factors

KT-Key Themes

OFI-Opportunities

for Improvement

1.6 Attend Examiner

Training at one of 5

locations (3 in Ohio,

1 in Indiana, and 1 in

West Virginia) in

November

2.6 Analyze the applicant’s

response to each Item. In BOSS,

select KF, Item reference, record

Strengths & OFIs, evidence, &

ADLI/LeTCI. Draft 1 Strength &

1 OFI comment for each Item.

Write concisely. Use NERD and

Comment Guidelines for help.

3.6 Once all comments are posted,

read all consensus comments &

scoring for all Categories. Provide

feedback to the Category Lead in

BOSS by (date).

4.6 Very important:

Hold all paperwork

and notes from SV

until instructed to

destroy

5.6 Judges meet to

decide award levels

and make

recommendations to

Board of Trustees

OP-Organizational

Profile

RS-Resource Sheet

SV-Site Visit

SVI-Site Visit Issue

TL-Team Leader

CODE OF ETHICAL CONDUCT

• Protect the integrity of the award process

• Exhibit professional conduct at all times

• Protect the promise of confidentiality

• Protect the program’s intellectual property

SCENARIO 1: THE ANSWER IS ……

Examiners should not exploit their selection to the Board of Examiners

(BOE). There are no “secrets” for receiving the award. The program

prefers to understate the “winning” aspect of the process; it encourages

a focus on continued self-improvement and the sharing of best

practices to help organizations achieve performance excellence.

Examiners may not use the MBNQA logo in any advertising/promotion.

Principles: Protecting the integrity of the award process, the program’s

intellectual property, and the reputation of the BOE.

SCENARIO 2: THE ANSWER IS ……

Any assistance with an organization’s TPE application would result in

a conflict of interest for an examiner. The examiner would be

considered an employee of the organization and should list the

organization on the Disclosure of Conflicts of Interest form. If the

organization reapplies, the examiner would not be permitted to

evaluate the application.

Examiners advising or participating with an organization in preparing

an award application shall not reveal or discuss that participation

with other examiners during training or at any other time.

Principles: Protecting the promise of confidentiality and the integrity of the award process.

SCENARIO 3: THE ANSWER IS ……

No. At no time during Independent Review or Consensus Review

should an examiner contact the applicant. Examiners shall not

communicate with the applicant organization or in any manner seek

additional documentation, information, or clarification. This includes

Internet searches or other external sources of information.

Examiners shall safeguard the confidences of all parties involved in

the judging or examination of current or former applicants.

Principles: Protecting the promise of confidentiality and the integrity of

the award process.

SCENARIO 4: THE ANSWER IS ……

No. Examiners must avoid disclosures that may in any way influence the award integrity or process, currently or in the

future, and treat as confidential all information about the

applicant. While this information has been publicly released, it is not part of the application, and, therefore, it is not relevant to

the evaluation. While potentially interesting, this information should not be discussed and must not be considered in the

evaluation.

Principle: protecting the integrity of the award process.

SCENARIO 5: THE ANSWER IS ……

Yes. Examiners may use the Internet and other sources to

familiarize themselves with common or generic terminology.

Examiners shall not communicate with the applicant

organizations or in any manner seek additional documentation,

information, or clarification specific to the applicant. This includes Internet searches or other external sources of information. If

questions arise, contact TPE.

Principle: protecting the promise of confidentiality.

BEGIN WITH THE END IN

MINDCollins Technologies Excerpt of Feedback Report

STRUCTURE OF CRITERIA ITEMS

WHAT IS A KEY FACTOR?

A key factor is an attribute of an organization or its

environment that influences the way the organization

operates and the key challenges it faces. It indicates

areas of strategic importance.

Examples may include the organization’s mission, vision,

and values; strategic challenges; workforce groups and

segments.

Key factors do not include descriptions of processes.

HOW DO EXAMINERS USE KEY FACTORS?

To focus assessments on what is strategically

important to the applicant.

To activate understanding of the applicants, its

business, and the environment in which it operates,

resulting in applicant-specific, evidence-based,

value-added feedback.

To use in juxtaposition with Criteria items.

Key factors are selected thoughtfully in the

context of the specific Criteria item.

DIRECTIONS TO EXERCISE

Independently, read the Criteria requirements in P.1a-Organzation

Environment.

Independently, read P.1a from your organization’s Organizational Profiles.

As a table, discuss the significance of the Key Factors identified for your

organization on the matrix and compare them to the KFs for the other two

organizations.

As a table, identify additional KFs that might facilitate better

understanding of the organizations.

PROGRAM & PRE WORK

QUESTIONS

SIX-STEP EVALUATION PROCESS

STEP 1: GROUND YOURSELF IN THE CRITERIA ITEM

STEP 2: SELECT THE RELEVANT KEY FACTORS FOR THE ITEM

USE KEY FACTORS IN TAB 4

STEP 3: READ THE RELEVANT SECTION OF THE APPLICATION

STEP 4: DRAFT AROUND SIX STRENGTHS & OPPORTUNITIES FOR IMPROVEMENT

SEE TAB 6

FOR SAMPLE

STEP 5: DRAFT FEEDBACK-READY COMMENTS

SEE TAB 8

STEP 6: DETERMINE THE SCORING RANGE AND SCORE FOR THE ITEM

UNDERSTANDING “SYSTEMATIC APPROACH”

PRE WORK INSTRUCTIONS

Cover Letter – Tab #10 in Training Manual

Prework Instructions – Tab #11

STEP-BY-STEP TO

INDEPENDENT REVIEW

Tab #12 in Training Manual

BOSS OVERVIEW & DEMO

See Tab 13 – BOSS Tip Sheet

REFLECTION ………..

• What did this day mean for you?

• What did you learn?

• What are your strengths and opportunities?

• What are your next steps? What will you change?

• What questions need to be answered?

2-DAY EXAMINER TRAINING DATES

November 2 – 3, 2015, PRO-TEC Coating Company, Leipsic, OH

November 4 – 5, 2015, CAMC, Charleston, WV

November 9 – 10, 2015, Battelle, Dublin, OH

November 12 – 13, 2015, VHA, Indianapolis, IN

November 19 – 20, 2015, The Christ Hospital, Cincinnati, OH

SURVEY

Please complete the survey

found at Tab #9 in the

training manual

THANK YOU!

QUESTIONS?

For questions/information:

Margot Hoffman

Margot.Hoffman@partnershipohio.org

614-425-7157