2016 baldrige new examiner orientation · pdf filefurthermore, board members enhance and...
TRANSCRIPT
2016 Baldrige New Examiner Orientation
This 2016 Baldrige New Examiner Orientation Participant Guide was developed and produced by the Baldrige Performance Excellence Program at the National Institute of Standards and Technology in Gaithersburg, MD.
Participant Guide
3
Course Overview
Purpose
As part of the on-boarding experience for first-year examiners, this classroom event will enable first-year examiners to confidently and proficiently participate in the Examiner Preparation course.
.
Learning Objectives
After completing this course, participants will be able to do the following:
Demonstrate working knowledge of the program, the evaluation process, and their role in the process.
Develop an Independent Review (IR) Worksheet for an award application using the Six-Step Evaluation Process, which encompasses the following: – Identifying and using an applicant’s key factors (KFs)
– Analyzing and synthesizing an applicant’s response to the Criteria, using the Approach, Deployment, Learning, and Integration (ADLI) and Levels, Trends, Comparisons, and Integration (LeTCI) evaluation factors
– Providing relevant, feedback-ready comments
– Using the Scoring Guidelines to holistically score an item to align with analysis
Identify personal strengths and opportunities for development based on feedback received during the course from peers and experienced examiners.
Participant Guide
5
Agenda
Morning
Welcome and Introductions
Course Overview
Process Overview and the Examiner’s Role
Begin with the End in Mind
Code of Ethical Conduct
Importance of Key Factors
Program and Pre Work Q&A
Afternoon
Reconfigure Class Groups
1-on-1 Coaching
Wrap-Up
Participant Guide
8
Examiner Development Model: On-Boarding
0–1 month
1–2 months
2–7 months
5–7 months
Prepare for 2nd year
Coaching ProcessPreparation Application Evaluation
Selection Notification
Baldrige Program
Orientation
Evaluation Process
Orientation
Case Study
New Examiner Orientation
Examiner Preparation
Team Assignment Planning for Year 2
Receive welcome e-mail.Select training week & wave preference.Participate in LinkedIn discussions for new examiners.
Complete online modules.Review sample feedback report . Read about Baldrige Award recipients.Participate in LinkedIn discussions for new examiners
Complete online modules.Complete “Are You Ready?” Participate in LinkedIn discussions for new examiners.
Read the Criteria.Complete prework.Participate in LinkedIn discussions for new examiners.
Complete classroom training.Peer learning, reflection, & feedback
Complete classroom training.Prepare for the awardapplication evaluation.Peer learning, reflection, & feedback
Complete IR Scorebook for assigned applicant.Peer learning, reflection, & feedbackTeam leader coaching
Reflect on experience.Seek feedback from examiner peers & team leader.
What are you learning?What are your strengths and
opportunities?What are your next steps?
Points to Remember
Participant Guide
9
Code of Ethical Conduct
Principles 2011
Baldrige Performance Excellence Program | www.nist.gov/baldrige
Code of Ethical Conduct
• Protect the integrity of the award process
• Exhibit professional conduct at all times
• Protect the promise of confidentiality
• Protect the program’s intellectual property
Points to Remember
Participant Guide
11
Code of Ethical Conduct Members of the Board of Examiners for the Malcolm Baldrige National Quality Award pledge to uphold their professional principles in the fulfillment of their responsibilities as defined in the administration of Public Law 100-107, the Malcolm Baldrige National Quality Improvement Act of 1987. The Board of Examiners Code of Ethical Conduct establishes accountability for these principles.
Four principles are the foundation of the Code of Ethical Conduct. These principles empower examiners to make effective decisions with great confidence.
Principle 1: Protect the Integrity of the Award Process
Examiners will make sound decisions related to conflicting or competing interests, as well as do their part to ensure that all applicants are evaluated consistently. Violations of this principle include, but are not limited to, the following examples:
Representing conflicting or competing interests or placing themselves in such a position where their interest may be in conflict–or appear to be in conflict–with the purposes and administration of the Baldrige Award. This includes being employed by, being a supplier or customer of, having a financial interest in, or having a consulting arrangement with a competitor or competing interest, present or future.
Approaching an organization they have evaluated for personal gain or accepting employment from an organization they have evaluated within five years of the evaluation
Using information gained from sources other than the award applicant, such as information gathered from the press, websites, social media, examiners, or any other outside sources
Intentionally communicating false or misleading information that may compromise the integrity of the award process or decisions therein
Principle 2: Exhibit Professional Conduct at All Times Examiners will conduct themselves professionally, guided by truth, accuracy, fairness, respect, and responsibility in all their interactions. Violations of this principle include, but are not limited to, the following examples:
Sharing their number of years of service as an examiner or promoting/advertising their services while performing as an examiner
Independently giving feedback to applicants regarding scoring or overall performance
Using Baldrige Program/Award logos in advertising or promotion. Note: Upon completion of the Examiner Preparation Course, examiners may use the following designation for one year, except on business cards: Examiner, Malcolm Baldrige National Quality Award (MBNQA).
Failing to respect the climate, culture, and values of organizations being evaluated when participating in site visits
Participant Guide
12
Principle 3: Protect the Promise of Confidentiality
Examiners will safeguard the confidences of all parties involved in the judging or examination of present or former applicants so that the integrity of the Baldrige Award or award process is not compromised. Violations of this principle include, but are not limited, to the following examples:
Disclosing the identity of or other information about the applicant to anyone other than the examiner’s team, the judge involved in the examination, or the Baldrige staff members involved in the examination during or at any time after the review process.
Copying applicant information of any kind. Note: Copying information from inside the Baldrige Program’s encrypted, secure database to other documents within the database is permitted as necessary.
Retaining applicant information beyond the Consensus Review (or Site Visit Review, if applicable)
Communicating applicant identity or other applicant information through e-mail, social media, or any other electronic or written means outside of the Baldrige Program’s secure database
Communicating applicant identity or other information via cell phone, VOIP, or wireless devices unless authorized by the applicant
Using or adapting applicant information subsequent to the review process, unless the information is publicly released by the applicant
Communicating with the applicant during Independent Review and/or Consensus Review
Communicating directly with the applicant during site visits about matters other than verifying and clarifying information in the consensus scorebook, unless the examiner is the team leader or backup team leader.
Requesting/reviewing individual customer, stakeholder, or workforce member data and information during the Site Visit Review process
Principle 4: Protect the Program’s Intellectual Property
Examiners will make sound decisions related to the use of Baldrige Program materials, trademarks, logos, and information contained within the program’s website. Violations of this principle include, but are not limited to, the following examples:
Establishing links from their own website to the National Institute of Standards and Technology (NIST) or Baldrige Performance Excellence Program (BPEP) website without making it clear that users will be taken to official NIST websites
Failing to acknowledge the use of trademarks owned by NIST, including those for NIST, the Quest for Excellence, and the Malcolm Baldrige National Quality Award, along with the statement indicating the trademark is registered by NIST
Making unauthorized copies of the Baldrige Criteria for Performance Excellence or any other Baldrige Program publications
Participant Guide
13
Furthermore, board members enhance and advance the Malcolm Baldrige National Quality Award as it serves to stimulate U.S. organizations to improve quality, productivity, and overall performance. All board members pledge to abide by this Code of Ethical Conduct.
Furthermore, board members enhance and advance the Malcolm Baldrige National Quality Award as it serves to stimulate U.S. organizations to improve quality, productivity, and
overall performance. All board members pledge to abide by this Code of Ethical Conduct.
Participant Guide
15
What Would You Do?
Table activity.
Your Table:
Your Scenario(s):
At your tables, discuss the dilemma facing the examiner. Select a timekeeper and a reporter. Discuss the following questions:
What were the issues in this situation?
Which Code of Ethical Conduct principles apply to this situation?
What would you do?
You have 5 minutes to complete this activity.
Scenario #1:
I received a mailing from someone who claims to have been a Baldrige examiner for five years. For only $500, we can attend his one-day seminar on “How to Win the Baldrige Award.” His brochure promises ten secrets for winning. Because his approach to winning the Baldrige Award is so successful, he has been given permission to use the Award logo on his publications. He guarantees that our organization will receive at least a site visit, and the brochure implies that he has terrific connections. I definitely think we should sign up, don’t you?
Participant Guide
16
Scenario #2:
I received an award application from the Baldrige Program for an organization that I assisted in writing its Organizational Profile. After we finished writing the Organizational Profile, staff members felt they could finish writing the application without my further assistance. I don’t think I have a conflict with this applicant since I did not write the rest of the application. Since the information I learned about the organization is limited, I think that I will share it with other examiners in my class during training to help them better understand the process applicants use to write an award application. What do you think?
Scenario #3:
The Performance Management System of the application I am reviewing is extremely complex and difficult to understand. I think I should contact a friend of mine who is an expert in human resources. She has said that she has consulted with some of the best organizations, and perhaps she can help me understand this applicant’s approach. Or, if she’s not available, maybe I’ll just contact the applicant’s official contact point (OCP). He will definitely be able to explain it. What do you think? Is this a good idea?
Participant Guide
17
Scenario #4:
Part 1: While waiting for the rest of the team to join the teleconference for consensus, a team member shares information with you about the applicant your team is evaluating. She tells you she heard this information during a professional association luncheon. Apparently, the applicant has begun negotiations with a supplier to require adherence to the organization’s new “green” standards. The supplier does not agree but feels pressure to conform. Should the two examiners be discussing this information?
Part 2: When the consensus call begins, the team leader opens by informing the team that the applicant has just made a public announcement that it will be requiring new environmental standards for its suppliers. The process for deploying these standards can be found on the applicant’s Web site. Should this information be considered in the evaluation process?
Scenario #5:
Last year, I evaluated a large manufacturing organization that is outside my area of expertise. The application contained a number of terms unfamiliar to me, such as Kaizen, Pareto charts, PDCA, and Failure Modes and Effects Analysis (FMEA). How can I do a proper evaluation if I do not know what these terms mean? Can I do any kind of research to familiarize myself with these concepts?
Participant Guide
21
Importance of Key Factors
Slide 2
Key Factors Definition
A key factor is an attribute of an organization or its environment that influences the way the organization operates and the key challenges it faces. It indicates areas of strategic importance. Examples may include:
The organization’s mission, vision and values
Strategic challenges
Workforce groups and segments
Key factors do not include descriptions of processes.
Slide 3
How To Use Key Factors
Participant Guide
22
Table activity: Directions.
1. Independently, read the Criteria requirements in P.1a – Organizational Environment (from the 2013-2014 Health Care Criteria for Performance Excellence)
2. Independently, read P.1a from your assigned organization’s Organizational Profile.
3. As a table, discuss the significance of the Key Factors identified for your organization on the matrix and compare them to the Key Factors for the other organization.
4. As a table, identify additional Key Factors that might facilitate better understanding of the organizations.
5. Appoint one person to report to the whole group, listing the key Factors identified by your table group.
How could such disparate Key Factors result in all three of these organizations receiving the Baldrige Award in one year?
Participant Guide
25
1-on-1 Coaching
The goal of this module is for new examiners (and non-examiners) to explore the use of the Six-Step Evaluation Process with senior and alumni examiners. Discussions with experienced practitioners of the evaluation process will help new examiners better understand how to apply the process.
Process Steps
New Examiners and BETE participants are expected to share their pre work for item 5.2, describing their output and the rationale behind it. Senior and alumni examiners are expected to listen actively, ask questions about the rationale used, and provide guidance as needed, using their own examples if needed.
You have 3 hours and 15 minutes to complete this activity (includes a break).
Table introductions between new and senior/alumni examiners.
Six-Step Evaluation Process
– Step 1: Review the Criteria item. – Step 2: Select relevant key factors. – Step 3: Read the relevant section of the application. – Step 4: Identify around six strengths and OFIs. – Step 5: Draft feedback-ready comments.
Step 6: Determine the scoring range and score.
Materials Needed
Criteria booklet and case study Prework: Key Factors Worksheet and completed IR Worksheet for item 5.2 Reference tools: Step-by-Step for Independent Review and Comment Guidelines Easel and flipchart paper
Participant Guide
27
Step 1: Review the Criteria Item
You have 20 minutes to complete this activity.
Learning Objectives:
Understand that “Ground yourself in the Criteria item” does not mean a rote reading of the requirements. It includes interpreting and studying the requirements in order to apply them appropriately.
Review your work with your coaching partner and receive feedback
Identify changes to this part of the evaluation practice that you will make, as appropriate.
Process Steps
Pair up with a coaching partner.
Explain what you think the main points in Criteria Item 5.2 are and why they are important, and then share your approach and rationale for accomplishing this step.
Ask for feedback.
Seniors will listen actively and ask questions to understand your thinking, and to encourage you to think about how your approach accomplishes the objective of the step and overall goal, pros and cons of different approaches, etc.
Take notes on your learnings and things you will do differently next time around.
Prepare to share your learnings with the large group
Participant Guide
29
Step 2: Select the Relevant Key Factors
You have 20 minutes to complete this activity.
Learning Objectives:
Understand the importance of the thoughtful selection of KFs and their eventual use in constructing feedback.
Review your work with your coaching partner and receive feedback.
Identify changes to this part of the evaluation practice that you will make, as appropriate.
Process Steps
Pair up with a coaching partner.
Explain to your coaching partner the key factors you chose and why.
Ask for feedback
Seniors will listen actively and ask questions to understand your thinking, and to encourage you to think about how your approach accomplishes the objective of the step and overall goal, pros and cons of different approaches, etc.
1. Take notes on your learnings and things you will do differently next time around. 2. Prepare to share your learnings with the large group.
Participant Guide
31
Step 3: Read the Relevant Section of the Application
You have 20 minutes to complete this activity.
Learning Objectives:
Understand that “read the application” is not intended to be at the multiple requirement level, but rather is a holistic review.
Review your work with your coaching partner and receive feedback.
Identify changes to this part of the evaluation practice that you will make, as appropriate.
Process Steps
Provide a brief overview of your approach to reading and analyzing and rationale, along with the results of your analysis of the application, asking for feedback.
Ask for feedback
Seniors will listen actively and ask questions to understand your thinking, and to encourage you to think about how your approach accomplishes the objective of the step and overall goal, pros and cons of different approaches, etc.
1. Take notes on your learnings and things you will do differently next time around. 2. Prepare to share your learnings with the large group
Participant Guide
32
Step 3: Read the Relevant Section of the Application
(continued)
Points to Remember
Participant Guide
33
Step 4: Draft around Six Strengths and Opportunities for Improvement (OFIs)
You have 20 minutes to complete this activity.
Learning Objectives:
Understand that strengths and OFIs incorporate the Criteria and the applicant’s operating environment and approaches (or results).
Review your work with your coaching partner and receive feedback.
Identify changes to this part of the evaluation practice that you will make, as appropriate.
Process Steps
Refer to the sample to item 5.1. This may be a useful document to refer to in your coaching discussion.
Provide your coaching partner a brief overview of your approach to determining strengths and OFIs, and your rationale.
Ask for feedback.
Seniors will listen actively and ask questions to understand your thinking, and to encourage you to think about how your approach accomplishes the objective of the step and overall goal, pros and cons of different approaches, etc.
1. Take notes on your learnings and things you will do differently next time around. 2. Prepare to share your learnings with the large group.
Participant Guide
34
Step 4: Draft around Six Strengths and Opportunities for Improvement (OFIs) (continued)
Points to Remember
Participant Guide
35
Step 5: Draft Feedback-Ready Comments
You have 20 minutes to complete this activity.
Learning Objectives:
Understand the use of the Comment Guidelines.
Review your work with your coaching partner and receive feedback
Identify changes to this part of the evaluation practice that you will make, as appropriate.
Process Steps
Refer to the Comment Guidelines (see page 37). Your comments should be evaluated against this document.
Explain to your coaching partner your approach to this step and rationale.
Review your comments.
Ask for feedback.
Seniors will listen actively and ask questions to understand your thinking, and to encourage you to think about how your approach accomplishes the objective of the step and overall goal, pros and cons of different approaches, etc.
1. Take notes on your learnings and things you will do differently next time around. 2. Prepare to share your learnings with the large group.
Participant Guide
37
Step 6: Determine the Scoring Range and Score for the Item
You have 20 minutes to complete this activity.
Learning Objectives:
Understand that scoring is a holistic process of determining the range most descriptive of the applicant’s achievement level for the entire item, rather than for individual processes.
Review your work with your coaching partner and receive feedback
Identify changes to this part of the evaluation practice that you will make, as appropriate.
Process Steps
The process will be the same as in previous steps. In this step you will provide your scoring range and score, along with a brief overview of your approach to scoring and rationale, and ask for feedback.
Refer to the “Scoring Guidelines in the Criteria Booklet.
Ask for feedback.
Seniors will listen actively and ask questions to understand your thinking, and to encourage you to think about how your approach accomplishes the objective of the step and overall goal, pros and cons of different approaches, etc.
1. Take notes on your learnings and things you will do differently next time around. 2. Prepare to share your learnings with the large group. 3.
Step 6: Determine the Scoring Range and Score for the Item (continued)
Points to Remember
Participant Guide
39
Wrap-Up
Reflection and Assessment
You have 10 minutes to complete this activity.
What did I learn? What did this day mean to me?
What are my strengths and opportunities?
What are my next steps? What do I need to change?
For Examiner Preparation: what questions still need to be answered?
Points to Remember