lessons learned from an initiative for improving … · lessons learned from an initiative for...

Proceedings of the 29th Annual Hawaii International Conference on System Sciences - 1996

Lessons Learned from an Initiative for Improving Software Process, Quality and Reliability in a Semiconductor

Equipment Company

Herb Kramer, President Krasner Consulting 7907 Ringtail Ridge Austin, TX 78746

Phone: (512) 328-4264 email: [email protected]

Abstract:

Improving software in semiconductor manufacturing equipment was targeted by SEMATECH in the early ‘90s due to numerous reports of problems, failures, complexities and growing needs. The SEMATECH SPI Project has been working with equipment supplier companies on focused software improvement initiatives. This report will describe the key accomplishments and lessons learned from the software process and quality improvement initiative that has been going on in Applied Materials for the last 2 years. Conclusions and recommendations for companies preparing to embark on such an improvement program are described, as well as, the plans and challenges for Applied Materials in 1995 and beyond.

1.0 Introduction

SEMATECH is a consortium that opened in Austin, Texas in early 1988. Its mission is to solve the technical challenges required to keep the US number one in the global semiconductor industry. It has been reported to have been a major contributor that allowed US companies to regain the lead in worldwide semiconductor equipment sales in the early ‘90s. The consortium is sponsored by: AMD, Motorola, National Semiconductor, AT&T, Digital, IBM, NCR, Hewlett Packard, Rockwell, Intel, Texas Instruments, and DOD ARPA. For more information about SEMATECH please refer to [8]. Affiliated with SEMATECH is SEMI/SEMATECH, an organization representing some 150 companies which supply equipment and software to the manufacturing units of the SEMATECH member companies.

Applied Materials is a Fortune 500 global growth company and the largest producer of wafer fabrication system, processes, and services for the global semiconductor industry. The Company delivers systems

Gregory Scott, SEPG Leader Applied Materials

3320 Scott Boulevard Santa Clara, California 94084

Phone: (408) 583-1522 email: [email protected]

and services for chemical vapor deposition (CVD), physical vapor deposition (PVD), epitaxial and polysilicon disposition, plasma etching, and ion implantation.

In addition to corporate facilities in Santa Clara, California, Applied Materials maintains research, development and manufacturing centers in the United States, Europe, Japan and Israel. To support a growing worldwide customer base, sales and service offices are located in North America, Europe, Japan, South Korea, Taiwan, Singapore, and the Peoples Republic of China.

Applied Materials global infrastructure is vital to maintaining close working partnerships with its customers and their worldwide operations. This partnering with customers is essential in bringing together the experience and resources needed to develop advanced semiconductor-manufacturing systems.

1.1 The Software Problem in Semiconductor Manufacturing

Software’s role in the semiconductor manufacturing industry has increased in scope and importance because of changes in the equipment/operator interface, increased automation of material handling, and integration of process steps into equipment clusters and cells. As a result, the quantity of software in semiconductor fabrication (FAB) facilities is expanding greatly. That software also is functionally more complex and frequently is embedded in equipment control systems or other time- and reliability-critical operations. As this trend continues, FAB and information system managers are using more suppliers (both commercial and in-house) and the frequency of software updates and new releases is increasing. This explosion of software content in the semiconductor industry has created increased demand for software that is functional; reliable; cost-effective; and easy to install, use, integrate and maintain.

693 1060-3425/96 $5.00 0 1996 IEEE

Proceedings of the 1996 Hawaii International Conference on System Sciences (HICSS-29) 1060-3425/96 $10.00 © 1996 IEEE

Information collected by SEMATECH indicated that many suppliers of semiconductor equipment are dealing with sharp increases in the volume of software embedded in the products they sell. For example, one representative piece of consists of 2 million lines of code and another is quickly approaching that size. Such trends mean that software complexity as well as the number of software engineers required is increasing and will continue to do so for some time to come. In some supplier organizations, the software portion of the engineering budget already exceeds 50%.

In 1992, a series of workshops sponsored by the Semiconductor Industry Association (SIA) observed the following about software problems in the US semiconductor industry [6]: l Software capability, availability and quality is

considered a “show stopper” technology, which if not addressed, will decrease the nation’s capability in integrated circuit technology and production.

. Software failures are a major problem for today’s FAB equipment, and over 50% of process equipment failures may relate to errors in the embedded control software.

l A viable software supplier infrastructure is needed that includes the creation of a skill base related to semiconductor manufacturing.

l Software development and management guidecmes must be established which create a technical discipline comparable to that in the process technology domain.

l Equipment users, through their purchasing behavior, must require equipment/software suppliers to follow appropriate software standards. While there is much intuitive and anecdotal evidence

of software related problems in this industry, there is very little hard data available. In an attempt to find out more about the reasons such problems were occurring, SEMATECH undertook a survey of the software issues troubling SEMI/SEMATECH companies in early 1993. A key conclusion from that survey[l] stated that most reporting companies had immature software processes. The report went on to recommend comprehensive strategic actions through which the industry could address those problems.

Software functionality and quality are major issues for FAB decision makers and equipment supplier company executives. Reports of equipment failure in FABs due to software [e.g.,91 have indicated that the situation is critical. Generally speaking, current equipment control software appears to be deficient in areas such as: handling of exceptions to normal operation, fault tolerance characteristics, documentation, design for maintainability, design of user interfaces, and reliability due to excessive software defects.


VLSI Research identified the ten best semiconductor process equipment supplier companies for 1994 [lo]. These ten have been rated by their customers as the best in VLSI Research Inc.‘s annual customer satisfaction survey for semiconductor manufacturing equipment. However, the overall rankings were generally lower in 1994 then in 1993. Across the board for these companies, persistent customer criticisms continue to focus on problems with the quality and support of their control software. As has been the case in prior years, software support is the lowest rated item for all semiconductor industry segments, thus identifying a major opportunity for wide scale improvement.

A recent analysis of the value of software improvement within a typical FAB was done by one SEMATECH member company. It showed that a 20% improvement in the software would result in a savings of up to $1.4 million per FAB per year. This was a very conservative estimate based on software outages less than 3% of the time. Proprietary studies conducted by SEMATECH at the request of member companies indicate the actual range of software-related FAB outages to be 22-60%. Obviously the potential savings would be much higher in such cases.

1.2 The SEMATECH SPI Project

SEMATECH’s Software Process Improvement (SPI) project was created in 1992 in response to these emerging concerns in order to help improve the quality and reliability of software provided by US suppliers of equipment to the semiconductor manufacturing companies. Since its inception the project has completed two major phases, is currently in the middle of its 3rd phase, with a fourth phase planned to commence in 1996. Phases are occurring roughly yearly. These are shown in Figure 1.

The project’s initial phase (phase I), completed in December 1993, planned and established the project, assembled and pilot tested an approach to software process improvement tailored to suppliers in the semiconductor manufacturing industry. The project’s beta test phase (phase 2), launched in late 1993 and completed at the end of 1994, refined the approach, beta tested it at 5 supplier sites, and prepared a deployment strategy and the infrastructure required to proliferate software process improvement initiatives throughout the US semiconductor manufacturing industry. The initial proliferation phase (phase 3) commenced in 1995, evolved the approach and initiated improvement programs in 10 more early adopter companies. Phase 4 will commence in 1996, and will feature extended proliferation.

The major goals of the project are to:

694



l Develop, demonstrate and deliver as an integrated approach to improvement - a method, model and tools (called the SPI Guidelines for Improving Software, see [2]) for increasing the maturity of the software process within a supplier organization;

l Proliferate successful uses of the approach among all US suppliers;

l Establish and sustain a semiconductor industry-wide inti-astructure that supports emerging software process improvement initiatives;

Verify and publicize evidence that increased process maturity results in higher quality software components, products and systems. 2. PROJECT STATUS

2.1 Phase 1

The first phase was initiated in May 1992 and completed in December 1993. Phase 1 initiated the project, created interest, and garnered enough resources for developing a basic approach and trying it out on one supplier company. The supplier company selected as the alpha test site, met SEMATECH’s criteria for having equipment products significant to the semiconductor industry, relevant software project needs, and a strong corporate commitment to TQM.

In phase 1 the project accomplished the following: l Assembled a critical mass of extended project staff to

work on the problem l Created Project Awareness and Sponsorship in the

customer sets

l Developed and alpha tested the SPI Methodology l Built a core SPI service supplier infrastructure. l Established lack of basic software measurement as a

major issue l Established essential collaborations (e.g. Sandia, SEI) l Developed a beta test strategy

2.2 Phase 2

Objectives

Building on the results of phase 1, phase 2 was initiated in the fall of 1993 and completed at the end of 1994. The primary objectives of this phase included: . Refinement of the software problem definition based

on the collection and analysis of additional US industry dam;

l Development and dissemination of the SPI Guidelines 3.0, which contain the technical core of SEMATECH’s approach to software process improvement;

l Adoption of the SPI initiatives by the five participating suppliers who serve as leaders and examples for the industry;

. Identifying and qualifying commercial sources for SPI services;

. Developing a semiconductor industry baseline for SPI measurement;

Phases 1992-3

Figure 1 - The Major Phases of the SPI Project

695



l Refining and prioritizing options within a strategy for wider-scale proliferation of SPI through the rest of the industry. In phase 2, the project staff worked with 5 US

supplier companies, 4 SPI service companies, one of the SEMATECH member companies, and Sandia National Labs. All were involved in the beta test of the enhanced SPI approach at the 5 beta test suppliers.

Phase 2 Overall accomplishments 4. Teradyne, Inc., Semiconductor Test Division in

Following are tifieen specific accomplishments for 1994 that represent the complete range of project activities in phase 2: 1.

2.

3.

4.

5.

6.

7.

8. 9. 10

Six suppliers built continuous software improvement programs. The SPI Project acquired an experienced and capable staff. Commercial providers performed improvement services. Member companies participated through a project technical advisory board. Six key areas of practice were targeted for improvement. Core metrics were defined for software. The core metrics are as follows [7]: 0 Software size - as measured in source lines of

code (SLOC) 0 Efirt - as measured in staff-hours l Schedule/progress - as measured by planned and

actual dates associated with such project milestones as reviews, audits, deliverables, and exit criteria

l Quality - as measured by software problem and defect counts

Release 3.0 of the SPI guidelines was developed and sent to all SEMI/SEMATECH suppliers [2]. A baseline was established for indicating progress. Member companies participated in the SPI Project. Sandia collaborated with the SPI Project. ‘.

11. The SE1 supported the SPI project. 12. Two Case studies in software improvement were

developed and distributed. 13. Software’s role in semiconductor manufacturing was

clarified [ 12, 131 14. The project used various channels of communication

to reach the suppliers. 15. Additional suppliers showed interest in SPI.

3.0 SPI RESULTS at Applied Materials

During the beta test phase, the SPI Project worked extensively with five supplier companies as pilot sites for software improvement initiatives. The five were: 1. Applied Materials in Santa Clara, CA, and Tel Aviv,

Israel 2. Eaton Corp., Semiconductor Equipment Division, in

Beverly, MA 3. Plasma and Materials Technology, Inc. in

Chatsworth, CA

Agoura Hills, CA 5. Varian, Thin Film Systems in Palo Alto, CA

These 5 were selected from 12 interested companies at the beginning of the beta test phase. The pilot sites were chosen in order to be representative of all semiconductor equipment suppliers, and thus, varied significantly in their size, scope, company growth profile, etc. A sixth pilot test company, PIU Automation of Billerica, MA, continued its work on software improvement that began during the project’s alpha test phase.

In each of the pilots, an executive manager or senior staff person sponsored the software improvement initiative, while an experienced software engineer or manager served as the SPI internal leader. They and other participants from each internal organization formed management steering teams, assessment teams, software engineering process groups, and technical working groups. An SPI expert consultant worked with each pilot. Four expert consultants were selected by competitive bid from a set of 17 bidders on the basis of their experience in helping organizations improve their software processes and quality. The consultant and a member of the extended SPI Project staff supported work at each site.

Twice during the year, in March and in September, the SPI Project held on-site reviews of the beta tests with the entire leadership team participating. In December 1994, all of the beta test company leaders met with the SEMATECH staff to review their work, completed as well as planned, in a workshop attended also by representatives from au additional 7 supplier companies considering SPI initiatives. The five pilot suppliers began their SPI initiatives in the 44 1993. The SPI initiatives progressed roughly through the following stages: l Assessing the software capabilities in terms of a

tailored SE1 capability maturity model . Developing action plans for improvements based on

the assessment findings and then executing the plans . Building infrastructures within the organization for

continuously improving software

696



) Planning &! 7 eking 1

I Product Release Control

I

Figure 2 - SPI Process Architecture

l Facilitating improvements by injecting knowledge about best practices, relevant standards, tools, methods; and through the design of small scale trials leading to wide scale deployment.

l Evaluating improvement progress against stated objectives. In the beta test phase, the SPI Project identified six

common problem areas that they felt an organization must address early in its effort to improve software. These 6 areas represent a blending of the CMM V 1.1 Repeatable Level KPAs [ 1 l] and key Quality Engineering activities are: 1. Configuration Management 2. Test 3. Peer Technical Reviews/ Inspections 4. Project Planning and Tracking 5. Requirements Management 6. Defect Tracking.

3.1 Applied Materials relative to other companies.

At the end of the beta test phase, the five suppliers assessed their progress in these six areas and forecasted further improvements for 1995. Figure 3 shows that as a group, they made significant overall progress in 1994 and that considerable progress was anticipated by them in 1995.

At the December 1994 and May 1995 SEMATECH Suppliers Workshop, each of the attendees were requested to rate their company’s SPI progress in six areas. Each of the six elements were broken down into 6-8 subelements. Each of the subelements were scored by representatives

of the supplier companies using a rating scale of blank, 0, 1 or 2 where: l Blank meant Don’t Know (not used in score) . 0 meant that no effort was being expended in this

area . 1 meant that this area was partially implemented . 2 meant that this area was fully implemented.

An average score for each area for each supplier was determined and then averaged with other supplier’s scores to produce Figure 3. As can be seen, most of the supplier companies had little or no SPI activities implemented in January 1994 and most companies expect to achieve or close to achieving SE1 Repeatable Level (level 2) by January 1996.

An interesting side effect of this is that as progress in the five beta test companies became visible to other SemiKEMATECH member companies in early 1995, these other companies initiated their own SPI programs. SEMATECH has responded by expanding its roll in member company activities by coordinating training sessions, providing consulting services and an on line process assets library in Lotus Notes.

3.2 Highlights of Accomplishments at Applied Materials

In January and February 1994 SEMATECH conducted a Software Process Capability Assessment at Applied Materials facilities in Santa Clara and Tel Aviv. In response to the assessment findings, Applied Material began an internal SPI initiative headed by the manager of Software Quality Assurance and aided by two co-

697



champions assigned to each of the six major findings. Each of the six co-champion pairs generated an action plan for their area of responsibility and began implementing the action plan. The six action areas are as follows: 1. Requirements Management. 2. Project Planning and Tracking. 3. Software Testing. 4. Product Configurations. 5. Customer-Supplier Relationships. 6. Leadership in Software.

4.0 Lessons Learned at Applied Materials

Some generalizations can be drawn from looking at the progress of Applied Materials and other SEMATECH supplier companies: . SPI progress rate is independent of organization size. . Infrastructure is everything. You cannot make

progress without it. l Training Program may be a Defined Level Key

Process Area but you cannot achieve a Repeatable process without one.

0 You cannot have a Training Program without an Organizational Process Focus

. Subjective measurement can be used in place of objective measurement.

l Configuration Management and Defect Tracking are receiving most of the SPI attention at supplier companies.

. Project Planning and Tracking and Requirements Management are receiving the least attention at supplier companies..

4.1 Rate of Progress

Revisiting Figure 3 reveals that the rate of change appears to be independent of organization size. The size of organizations that contributed to the survey ranged from the very small organization with less than five software engineers and managers to large organizations with over two hundred software engineers and managers. Working on the assumption that 70% or a subjective score of 1.4 is successful implementation, albeit not yet institutionalized, none of the key areas were implemented in January 1994. Even in January 1995, when everything WaS partially implemented, only Configuration Management was working satisfactorily for the participating companies. However, all companies expect to have all of the critical areas working by January 1996.

1.8

1.6

1.4

1.2

1

0.8

0.6

0.4

0.2

0 l/1/94 l/II95 * Ill/96

Figure 3 Progress Indicators for Supplier Companies including Applied Materials.

4.2 Infrastructure meeting varied levels of success. Status reports indicated that many of the CMM Key Process Area (KPA)

When the author began working at Applied Activities for Repeatable Level were being attempted but Materials, one year after the organization had undergone a large percentage of them were meeting with failure. a SEMATECH-lead Software Process Assessment (SPA). A quick audit revealed that the Commitment to An action plans had been written and six co-sponsor pairs perform and Ability to perform Common Features of were attempting to implement process improvement but every Repeatable KPA were not implemented. The audit

698



was halted at that point with the finding that none of the basic infrastructure was in place to enable successful software process improvement.

A new action plan was developed. An off site meeting was held with the software engineering leads to developed and document the basic Repeatable software engineering infrastructure. Working groups were formed during the off site to become process action teams with the charter to correct difficulties found during implementation. However, the immediate output of the off site meeting was the discovery of the first pot hole on the road to process improvement. You cannot build an infrastructure without a training program.

One long aside before returning to lessons learned. This lesson contains a warning for defense contractors rushing to shed DOD standards and adopt commercial practices for software engineering, be careful about what you throw out. Table 1 characterizes lessons learn at Applied Material contrasting the authors’ experiences of SPI in a DOD contractor environment with the environment found at Applied Materials.

Most DOD contractors have documented in excruciating detail every aspect of product development.

A typical software development effort follows DOD- STD-2167A which requires a software plan consisting of: a documented process; software development plan and it’s relationship to the overall project plan; staffing plan; configuration management plan; and a software quality assurance plan (which may be documented separately in accordance with DOD-STD-2168). Not to mention all of the software documentation required to support as software development effort, such as: Software Requirements Specification (SRS), Software Design Document (SDD), Interface Control Specifications (ICS), and Software Test Plans (STP). In short, at the beginning of a government funded project, every aspect of the effort is documented and a suite of documents generated to capture the entire effort to be used later during product operations and maintenance.

As software organization and processes mature Software Engineering Process Groups are formed and staffed and SPI efforts are incorporated into the project organizations [15, 161. It is not until this point that organization process focus and organizational process definition begin to become important to a DOD contractor.

Table 1. Comparison of DOD Contractor and Commercial Development Documentation.

Document DOD Contractor Environment Commercial Environment Corporate Quality Manual Rarely exists in this environment because of Almost an essential document for conducting

the focus on DOD standards and not customer- business, especially if pursuing IS0 9000 supplier relationships certificate.

Standard Operating Detail documentation of every aspect of the Primarily a collection business relationships Procedures (SOP) organization usually containing one or more and HR policies. Rarely contains product

chapters devoted to engineering. development processes or engineering policies.

Software Engineering DOD-STD-2 1267A compliant, documenting Non-existent as a unified document. May Manual software engineering procedures and exist as a disjointed collection of isolated

standards in great detail. procedures and standards, usually tailored to a specific project.

Project SOPS Required by contract. ad hoc, at best, organization usually does not have a process focus or organizational process from which to generate project SOPS.

4.3 Training Program

Policy, Standard and Procedures Manuals are A second pot hole was quickly discovered. There are shelfware which should come as absolutely no surprise to very few quality training vendors that are willing to anyone. The difference between good shelfware and bad provide customized training programs. They will be shelfware is training. Many managers and project leads at happy to sell you all kinds of off-the-shelf courses but a Applied Materials, when asked to find something in the training course designed around your companies specific Policies, Standards & Procedures (PS&P) Manual greeted needs is just not part of their business plan. the request with confusion. Most of them had signed a Software Engineering Process Group (SEPG) located form stating that they had read the PS&P Manual but a contractor to develop and deliver an Applied Materials

never expected that anyone would really ask them where to find something in the manual. Clearly, a class covering the contents of the PS&P manual was in order.

699


Proceedings of the 29th Annual Hawaii Inte national Conference on System Sciences - 1996

Software Policies and Procedures training course. During the requirements definition phase of course development, a third pot hole in the road to process improvement was discovered. You cannot teach policies and procedures without providing the students some context in which to understand the rationale for the policies and procedures.

This is where a vendor with off-the-self courses is an asset. A one day CMM overview course was combined with a one day IS0 9001 overview course to produce a Software Engineering Quality Standards Overview training course. This exposure to quality standards gives the software engineers and managers a framework in which to understand the policies and procedures.

At each step of the improvement process another training requirement is uncovered. What had started out as a way of educating software managers and engineers in how to survive a quality audit has become a complete training program. Which leads to the conclusion that you cannot have a training program without an SEPG.

4.4 Software Engineering Process Group (SEPG)

The organization my not be call an SEPG but someone must be responsible for SPI activities. Some of the early challenges to SPI success at Applied Materials was the inability of the co-sponsors to provide enough time to achieve success. Which leads to the assertion that process improvement cannot happen unless someone is given ownership of the processes to be improved.

The real reason is a bit deeper than that. An Initial Level organization, especially a successful one, sees process change only in context of past successes or failures. Improvements that lead to the production of faster, better, cheaper products are not usually within the

paradigm of the organization. Someone with a different point of view must be placed in a position to effect change. In variably this person’s first task is to begin a training program targeted at expanding the viewpoint of the organization.

4.5 Subjective measurements

An Initial Level organization is not mature enough to establish an effective measurement program. Because of the ad hoc nature of the organization measurements will vary to the point of being meaningless. But some form of progress measurement is absolutely necessary.

Taking a tip from Fred Langner of SEMATECH (see Figure 3) the author assembled a questionnaire using the goals from five of the six Repeatable Level KPAs. Each month all of the key software managers were asked to rate how their organizations performed to each of the goals using a scale of one to ten where ten was a perfect score. Figure 4 shows the results of the first four months of the survey.

Based upon experiences with this form of measurement two generalizations can be made. The normal trend from subjective measures is positive. That is, the scores improve with time. The second observation is that training causes a decrease in scores. Either way the very act of asking the questions for the survey increases awareness of software process improvement issues, which is the real purpose of the survey.

4.6 Challenges Encountered

6.00

7.00

6.00

5.00

4.00

3.00

2.00

1 .oo

0.00 Jun-95 Jul-95 Aug-95 sap-95

Figure 4. Subjective Progress against Repeatable Level KPAs.

700



The road to improvement was not necessarily smooth in all cases. This report would not be complete without identifying some of the major challenges that were encountered. The following additional potholes were found along the road to improved software: l Staff shortages of software engineering,

management, and SPI positions l Loss of key technical skills due to employee turnover . Lack of technical software engineering training l Lack of commitment of responsible and continued

leadership for software improvement l Making software improvement sufficiently high

priority l Invisible, wavering or sporadic senior management

sponsorship . Lack of focus on change as a part of the engineering

job l Lack of will ingness and discipline to change l Turf politics l Lack of definitive software quality goals

4.7 SEMATECH SPI - Working with Suppliers in 1995 and Beyond

A supplier’s commitment to improve software involves a good bit more than simply saying “Let’s do it.” There must be a clear understanding of the work involved, some preparation, and some minimum level of readiness. An organization that is ready has the following elements in place or is willing to implement those it does not yet have: l Sponsorship for SPI by senior management l A management steering group that owns

responsibility for improvements . A baseline of its current software process and

performance l A quality improvement goal and business incentive

for achieving it l A budget for staff skill growth in software

engineering and management l One or more focused working groups, staffed and

with charters l A percentage of time for software engineers and

managers to work on SPI Other readiness considerations relate to the

organization’s culture, its interactions with its customers, and its will ingness to contract with an SPI service provider if there is a lack of experience within the organization in effective organizational maturation/change tactics. Clearly, a commitment to improving software is viewed as a serious matter.

5.0 CONCLUSIONS

Many companies in many different business sectors are reporting successful software improvement programs [3]. They report ROI figures between 5 to 1 and 9 to 1, with some lag between the investments and the observable benefits. The amount of lag usually depends on organization size, type and scope of improvements attempted.

Several notable systematic software improvement programs have emerged which may serve as models for the remainder of the US software industry [e.g. 4, 51. However, many formal such programs which attempted to start in the late ‘SO’s faltered soon after a formal SEI-style assessment was performed (2/3 reportedly died), are now in decline or perhaps wiIl soon fail. This may be happening because of: a f lawed strategy, a lack of commitment, lack of follow through, not measuring improvements, or lack of crisp improvement objectives that may not have been tied to business objectives. Newer software improvement programs may be more likely to succeed because they have learned from the mistakes of the pioneers, explicitly represent that process maturity is a means to an end, and have more focused objectives.

A defined software improvement payoff function must tie business, engineering/development and personnel improvement objectives together in a meaningful way. SPI works best when payoff is demonstrated in all three of these areas simultaneously (Win, Win, Win). A payoff function which demonstrates the pursuit of higher quality software seems to do that.

The following high level payoff impact areas should be considered when setting software improvement program objectives: . Improving profitability, market share, time to market,

competitiveness, productivity, competence, etc., . Delivering ever-improving software quality to

customers/users, l Maximizing the overall predictability, productivity

and effectiveness of the software development process.

l Reducing the amount of crisis-driven chaos by managing growth and change in scope, size and complexity,

l Increasing the pride in workmanship, use of new skills learned and personal empowerment in decision making. Management indicators within this payoff function

would include measures of: product quality, process quality (e.g. rework, productivity), project predictability, skill base growth, and customer satisfaction. Some of the biggest payoffs of improvement are expressible in human terms, not dollars. They might involve: better job

701



satisfaction, pride in work, an increased ability to attract, retain and grow experts that will innovate, etc.

For organizations that aren’t fully mature yet, the existing data suggests that there is no reason not to start a well focused, well designed software improvement program. Crisp objectives should be tied to corporate business success factors. Executive sponsorship and demonstrated commitment by management will help motivate the program. Measurement discipline is a key to success - quantitative baselines must be established and then measured against. A focus on improved software quality seems to accelerate an improvement program, and should be targeted. Starting quickly with specific improvements that demonstrate early results raises the level of confidence in the program. Periodic re- appraisals/process assessments reinforce and stimulate the program. Patience is a virtue, since payoff measures resulting from culture changes may not be visible for some time. Setting realistic expectations of upper management are important if the improvement program is to be perceived as credible.

The question of how to get started with a program is often asked. The advice usually given is to start small, demonstrate a success and then build momentum. Improving the basic understanding of software issues in small, low maturity organizations is achievable within a few months. A formal systematic improvement program should then be pursued in expanding cycles that consist of: establishing sponsorship for the program, baselining project performance, process maturity and current “as is” process definition, setting improvement goals, formulating and executing an improvement road map, measuring progress, adjusting the program as needed and continuing until a “best in class” capability is achieved.

6.0 REFERENCES

[l] Sofnvare Process Improvement Issues Survey Findings Report, 93011489A-TR, Presents results, conclusions and recommendations of semiconductor industry survey

[2] SEMATECH SPI Guidelines for Improving Sofmare: Release 3.0, SEMATECH Technology Transfer 94092541A-ENG, October 3 1, 1994

[3] Krasner, H. (1994) The Payoff for Software Process Improvement: What it is and How to get it, in IEEE Technical Committee on Software Engineering Newsletter, September, 1994

[4] Krasner, H. (1994), A Case History of the Space Shuttle Onboard Systems Project, SEMATECH Technology Transfer 9409255 1 A-TR, October 3 1, 1994

[5] Kramer, H. (1995), A Case History of Process Improvements at the NASA Software Engineering Laboratory, SEMATECH Technology Transfer 94 122662A-TR, January 3 1, 1995

[6] Moore, G., et al., SIA Semiconductor Technology, An Agenda for American Cooperation - Workshop Conclusions, 1993, Semiconductor Industry Association

[7] Kramer, H. (1993), Implementing A Basic Software Metrics Program in SEMI/SEMATECH Supplier Companies, Krasner Consulting TR 93.3, July, 1993.

[S] SEMATECH, 2706 Montopolis Drive, Austin, Texas, 78741-6499, Attention: Communications Department; Phone: (512) 356-3137

[9] Maxion, R. (1994), Preliminary Report on Software Failures, SEMISEMATECH distribution of SEMATECH ETAB Report, May 12, 1994

[lo] Steele, L. (1994) Ten-Best Process Equipment Companies in 1994, VLSI Research Inc., San Jose, CA

[ll] Two publications that document the Capability Maturity Model for Software are: M. Paulk, B. Curtis, M. Chrissis, and C. Weber, Capability IMaturity Model for Software, Version I. I, Software Engineering Institute, Pittsburgh, PA, CMU/SEI-93-TR-024; and M. Paulk, C. Weber, S. Garcia, M. Chrissis, and M. Bush, Key Practices of the Capability Maturity Model, Version 1. I, Software Engineering Institute, Pittsburgh, PA, CMU/SEI-93-l-R-025

[ 121 The 1994 version of the National Technology Roadmap for Semiconductors (SIA) is the product of all sectors of the US semiconductor technology base industry, government, universities, customers, and suppliers. It updates the first National Roadmap, which was prepared in 1992-93.

131 E. Neacy, S. Brown, and R. McKiddie, MMAC Survey and Interview Results, Technology Transfer #94052374A-XFR, June 30, 1994, SEMATECH, Austin, TX.

141 Sofhoare Process Improvement (SPl) 1995 Project Plan, Technology Transfer #94 122653A-TR, January 3 1,1995, SEMATECH, Austin, TX

151 Care and Feeding Costs of a Maturing Sofiware Process, Gregory J. Scott and Ann L. Hughes, Proceedings of 44th National Aviation Electronics Conference (NAECON-91).

[ 161 Can Software Engineering Afford Process Improvement?, Gregory J. Scott, ACM SIGSOFT Notes, April 1992.

702


lessons learned from an initiative for improving … · lessons learned from an initiative for...

Documents