? -

USE OF MODIFIED BALDRIGE CRITERIA FOR TEAM PROJECTS:

APPLICATION TO A RESEARCH AND DEVELOPMENT LABORATORY

Suzanne H. Weissman Sandia National Laboratories

P. 0. Box 5800 Albuquerque, NM 871 1 1-0724

SUMMARY Sandia National Laboratories has developed a quality award program for teams based on the Malcolm Baldrige National Award Criteria. Applicants describe how they applied Total Quality Management principles to achieve customer focus, process management, and results. From the applications, Baldrige-based feedback and scores are given; and high-scoring teams receive awards.

KEY WORDS Total Quality Management, TQM, quality, Baldrige, awards

INTRODUCTION As in many research and development (R&D) organizations, incorporating Total Quality Management (TQM) concepts into Sandia’s culture is slaw. In order to both promote TQM concepts and to recognize Sandians who follow these principles, Sandia’s President, Al Narath, requested that a team design and implement a new award at Sandia National Laboratories based on quality processes. His requirements were that this award have the following characteristics:

Based on the Malcolm Baldrige National Quality Award Criteria; Focus on results AND the process used to achieve those results; Support Sandia’ s corporate values of teamwork, quality, integrity, leadership, and respect for the individual; Minimize competition among Sandians by having applicants compete against a minimum standard and not against each other; Annual award to be presented each year during National Quality Month by the President of Sandia National Laboratories; Be non-monetary.

This award is called the Sandia President’s Quality Award (PQA) and was developed by the PQA Design Team. Prior to developing the criteria, the PQA Design Team benchmarked award programs at five companies (AT&T Bell Laboratories, Pacific Bell, FMC, Motorola, and Digital Equipment Company). We continue benchmarking as appropriate throughout the process. The initial application format was released in the spring of 1993, The first awards were presented in a celebration in November 1993. The following two years the application and criteria were improved, and awards were presented in October 1994 and will be presented in October 1995. Each individual on an award-winning team receives a distinctive diamond desk ornament etched with one’s name and the year. The award level is shown on an embedded medallion. There is no limit to the number of awards given each year at either of three levels. The highest level is gold,

Portions of this document may be megible in electronic image products. Images are produced from the best available original document.

which represents superior performance in all three categories: customer focus, process management, and results. The next level is silver, which represents sound approaches with excellent performance in at least two of the three categories. The lowest award level is turquoise, which represents defined approaches with very good performance in at least two of the three categories.

ANNUAL AWARD CYCLE Three key groups participate in the PQA process: the PQA Design Team, which is responsible for the overall process; the examiners, who score each application and provide feedback to the applicants; and the applicants, who are members of the teams submitting applications. Quality Coordinators are specialists located throughout Sandia whose primary purpose is to continually improve Sandia’s processes and achieve customer satisfaction by: 1) advocating and modeling quality principles and 2) assisting Sandia employees in applying TQM. Sandia’s Quality Coordinators assist in all stages of the PQA process; and, particularly, in applying the PQA criteria to a team’s activities and in writing a team’s PQA application. The annual award process is shown in Figure 1. The PQA Design Team improves the process annually.

FIGURE 1 - High Level Flow Chart of PQA Process POA Design Team Applicants Examiners

Lessons Learned & Customer Satisfaction to PQA Design Team @- I Lessons Learned & Customer

Satisfaction Analyzed

I Improvement Areas Selected I I New Schedule Established I

+ Criteria & Examination Process Revised

I Application Guidelines Distributed Intent to Submit Forms Completed I 4

Applications Completed Applications Examined 1 Clarification Visits Conducted

I

Awards Announced

I Feedback & Scores Sent to Teams 4

I Awards Celebration and Winning Applications Shared I * Lessons Learned and Customer Satisfaction Comdeted

@J Note: Quality Coordinators assist throughout process.

Page 2

The primary customer for the overall process is Al Narath, the President of Sandia Labs, who provides high level requirements for the overall PQA process. Applicants, Division Quality Coordinators, and Examiners are also important customers of subprocesses, who provide detailed requirements and lessons learned to the PQA Examination Team on subprocesses. Types of subprocesses include criteria, examination process, and recognition process.

The annual cycle begins by reviewing lessons learned from the previous year and feedback from our customers. Certain areas are selected for improvement and the schedule for the coming year is determined. The new criteria and important information about the award cycle are published in a pamphlet (The PQA Guidelines) that is widely distributed throughout Sandia. A team indicates its Intent to Submit an application by completing a simple form. This enables the PQA Design Team to determine how many examiners are needed. Applications are due about 7 weeks after the PQA Guidelines were distributed and the awards are announced two months later.

The PQA Design Team improves the criteria annually based on input (lessons learned and customer satisfaction) fiom applicants, Quality Coordinators, and Examiners. Changes in the Malcolm Baldrige criteria, assessments of Sandia’s progress in quality, and the usellness of the PQA criteria are also used to change the PQA criteria. For example, during the initial cycle, the body of the application was three pages, which limited the amount of data that could be included in Category 3 (Results). The following year, an optional two page data summary was initiated. In 1994 there were two types of applications: product (one-time activities) and process (repetitive activities). For a process, applicants indicated how the process was improved; while, for aproduct, applicants indicated how learnings from the product were transferred to other products at Sandia. In 1995, the distinction between product and process was eliminated. Also, during 1995, clarification visits, which are much like Baldrige site visits, were added. Clarification visits are planned for those applicants that achieve the highest scores and for all applicant teams being considered for the gold award level.

The examination process was also improved during the three years since the award was established. During the first year examiners were encouraged to have Baldrige training prior to training on the PQA criteria. In later years, Baldrige training was a requirement for examiners. Also, in the second year a computerized tool was developed for examiners. The third year the computerized examination tool was improved and the length of the examiner training was increased. During the third year our examiner training was designed and taught with the assistance of a Senior Examiner for the Malcolm Baldrige award process.

CRITERIA The criteria have always emphasized those parts of the Baldrige criteria which are applicable to teams, as shown in Table 1. During the last 2 cycles, there have been 3 examination categories:

1 .O - Customer Focus 2.0 - Process Management 3.0 - Results

Page 3

TABLE 1 . Comparison of the Content of Malcolm Baldrige National Quality Award Criteria

Management of Supplier Performance

Product & Service Quality Results I Yes 3 .1

Company Operational and Financial Results Somewhat 3.1

Supplier Performance Results No

Customer and Market Knowledge Somewhat 1.1

Customer Relationship Management Somewhat 1.2

Customer Satisfaction Determination Yes 1.3

I 7.4 lcustomer Satisfaction Results I Yes 1-32 I 7.5 ]customer Satisfaction comparison I Somewhat I 3.2 I The description of the criteria that follows is that of the 1995 criteria. As with the Baldrige criteria, the total possible score is 1000 points.

Customer Focus (240 points): The first item (1.1) is Understanding the Customer (120 points). This item deals with how customer requirements are determined and prioritized and how a team determines how the customer would use the product or service. Applicants are requested to describe how the team evaluates and improves its process for determining customer requirements. The second item (1 2) is Customer Relationship Management (60 points) and deals with how customer relations are built and maintained. The third item (1.3) is Customer Satisfaction Determination (60 points) and deals with how customer satisfaction is determined. A list of key customer satisfaction indicators is also requested.

Page 4

Process Manaaement (280 D O ~ ~ S ) : The first item (2.1) is Design and Management of the Production and Delivery Process (240 points). This item asks for a description of how key customer requirements are used to drive the design of the process for producing and delivering the product or service. Applicants are requested to describe how the production and delivery process is monitored, maintained, and improved. Applicants are also asked to indicate how each team member is enabled to contribute effectively to meeting the team’s goals. The second item (2.2) is Management of Supplier Quality (40 points). This item requests a list of key suppliers and a description of how the quality of the supplier goods and services was ensured.

Results (480 D O ~ ~ S ) : The first item (3.1) is Product or Service Results (220 points). This item asks for a description of current quality levels and trends for the outputs and for in-process measures that predict the quality of the output as well as comparisons with results for other organizations that are seen as leaders. The second item (3.2) is Customer Satisfaction Results (260 points), which asks for a summary of current levels and trends of key customer satisfaction indicators as well as comparisons with other organizations. Core Team: Another important concept in Sandia’s PQA is the Core Team, which focuses on the primary contributors to an activity. Core team members have the responsibility and accountability for the entire scope of the project. It is the critical few, typically 3 to 12 people at any one time. Often, SNL projects involve a central coordinating team and many subteams; all of which can submit separate PQA applications. The PQA Design Team encourages that the core team members recognize others who contributed to a team’s success and a PQA award.

EXAMINATION PROCESS As in the Baldrige process, PQA applications are scored based on three evaluation dimensions:

Approach - How the applicant addresses the requirements 0 Deployment - The extent to which the team’s approaches are applied to all requirements

Results - The outcomes in achieving the purposes of the examination item. Categories 1 and 2 are approacWdeployment items; category 3 is a results item. Our scoring guidelines are equivalent to those used by the Baldrige process. As with the national Baldrige process, Case Studies are used in training of our examiners.

RESULTS OF THE PQA PROCESS The types of applications received over the last three years represent the variety of work at Sandia National Laboratories. Projects include administrative and technical activities; short term (less than 2 months) and long term (over 20 years). Some projects have only internal customers while others are being conducted for external customers. Tables 2 and 3 show some data on the PQA process. As can be seen the number of applications received has increased every year, as has the number and level of awards.

Page 5

TABLE 2 - Participation in PQA Process Metric FY93 FY94 FY95

Number of Applications Received 86 111 120 Number of Intent to Submits Received 177 16 1 Number of Team Members on Applications 1200 1600 about 1700 Number of Examiners 16 45 43 Number of Students Completing PQA Course -- 127 __

TABLE 3 - Number of PQA Awards Given Level of Award FY93 FY94

Gold 0 0

Silver 5 10 Turquoise 11 4

USE OF PQA CRITERIA TO MANAGE PROJECTS Our primary purpose in developing the PQA process is to increase the awareness of quality as a critical element of Sandia’s success and to enable Sandians to understand the requirements for performance excellence. During late 1994, Sandia initiated a new activity associated with the PQA-a course designed to help teams focus on using the PQA criteria to manage projects. Our training and development organization recommended that teams (two to three people per team) take the course together. In the six months that this course has been offered, 127 Sandians have taken the course. Course attendees have used the knowledge gained from the course to strengthen their use of TQM principles and to apply for an award. Participants from award- winning teams also assist other teams in applying PQA concepts to other projects.

RESULTS OF THE PQA PROCESS Applicants are surveyed at the end of each year and over 85% of respondents indicate that the PQA Application is a usefbl tool to the team for analyzing their project or process. Teams are eligible to apply for a higher award level each year, and some teams that win a Turquoise or Silver award will work very hard to win a Silver or Gold award the following year. One member of a team that won a Silver award in 1994 commented that, “I now realize that if we had done things differently, we would have received a better product for less cost and effort.”

CONCLUSION Sandia’s PQA process is a contributor to improving the quality level, as measured by the Baldrige criteria, of our technical and administrative processes throughout the laboratory.

ACKNOWLEDGMENTS I want to acknowledge the many members of the PQA Design Team, especially those who have been on the team since its inception: Allison Kane and Cliff Yokomizo. I also want to acknowledge all who participated in the process as members of the PQA Design Team, as

Page 6

Examiners, as members of the Applicant Teams, or as Quality Coordinators assisting the PQA Process. This work performed by Sandia National Laboratories for the US Department of Energy under contract DE-ACO4-94AL85000.

REFERENCE LIST Malcolm Baldrige National Quality Award: 1995 Award Criteria. Gaithersburg, MD: National Institute of Standards and Technology.

DISCLAIMER

This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsi- bility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Refer- ence herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recom- mendation, or favoring by the United States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof.

-

Page 7