Reaching Business Goals with Value Adding CMMI Assessments

Download Reaching Business Goals with Value Adding CMMI Assessments

Post on 23-Jan-2015




0 download


This paper describes how to apply the CMMI in a business-focused way, by doing frequent small assessments. Goals are input to the assessments, and a matrix is used to assure both organisation and process area coverage. The results of assessments integrate smoothly with operational targets of existing groups within the organisation.


  • 1. Reaching Business Goals with Value Adding CMMI Assessments, by Ben LindersEricsson EuroLab Netherlands (ELN) is a company within Ericsson, the worlds leading supplierin telecommunications. They have facilities in the south (Rijen) and in the east (Enschede, andEmmen). In total about 1100 employees are working for this company, almost all are directlyinvolved in software and hardware development. The scope of this article is about the design unitin Rijen, and in particular about the Charging Competence Centre department. This departmentemploys about 160 persons, developing charging systems for telephone exchanges. Thetechnology is UNIX and NT based systems, using UML based Object Oriented development withC++ and JAVA, deploying mostly Rational toolsThis paper describes how to apply the CMMI in a business-focused way, by doing frequent smallassessments. Goals are input to the assessments, and a matrix is used to assure both organisationand process area coverage. The results of assessments integrate smoothly with operational targetsof existing groups within the organisation.Practical tools support the frequent assessments, and experience with the CMMI is used tooptimize the assessment and improvement cycle, resulting in continuously improving theperformance of the organisation based on its goals.1History: CMM as a rating modelCMM assessments have been done for quite some years within Ericsson. This paragraphdescribes the history, a more detailed description can be found in a separate paper on this subject[ref 1].Ericsson was one of the early adopters of the CMM model. The process focus, and the concept ofmeasurement based improvement were the unique selling points why the model was chosen. Thefirst assessment, back in 1994, showed Ericsson Rijen was on level 1, which was no surprise. Animprovement program was initiated, which established a process support organisation that rolledout all level 2 processes in the projects. The next assessment in 1995 showed that the Researchand Development Unit of Ericsson Rijen was on level 3, the first company within Ericssonworldwide.Reaching level 3 was a big reward for the organisation. As a result, the focus changed, andoperational goals such as meeting deadlines, combined with many vacancies demanded their toll.Projects became less focused on processes and measurements. Since the process supportorganisation remained, as did the culture to improve in most of the departments, the projects stillmanaged to live up to the expectations. But things didnt always go that smoothlyOne department in Ericsson Rijen, the Charging Competence Centre, kept on believing in theCMM and its usefulness. It started their own program to come to level 4, by definingmeasurements and applying them in their projects [ref 2]. In June 1998, they made their firstpreparations towards a new assessment. Bill Curtis, associated with the Software EngineeringInstitute, stated that the Unix development department could be "among the world elite in oneyear". Finally, the department was re-assessed in November 1998, where it was found to be aprofessional "solid as a rock" level 3 organization, with parts of level 4 institutionalised.Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders Page 1 of 12

2. Having worked for quite some years on process improvements, and not getting the reward ofbeing fully at level 4, came somewhat as a disappointment to this department. And it alsoconfused them, as they didnt just implement the practices of the CMM model. The approach hadbeen from the goals and concepts of the model, implemented it in such a way that it made senseto do it. Though no process areas were excluded, not all practices were implemented. For somethere were alternative solutions. But most of all they were confused because in their opinion theimprovement programs had delivered results, being an organisation able to perform projects in acontrolled way, and capable of improving where business goals required it.As a next step, the department didnt start a new improvement program to fill the gaps found inthe assessment. Instead, the approach focussing on business goals and selecting improvementsthat contribute to them was continued, regardless of CMM levels involved. This focus on CMMmatched perfectly with operational goals set by the department, in the sense that CMM wouldcontribute directly to the business.Along with this focus, the assessment approach had to change also. Full assessments were nolonger useful, instead there was a need for small focused assessments which would come up withdetailed findings. These assessments should be an instrument to be used next to audits andimprovement sessions in projects.2 Approach: Business Focused CMMI AssessmentsA new assessment method has been defined: Business-focused CMMI assessments. Such anassessment is a class C type assessment [ref 3], where the main purpose is to find strengths andweaknesses. The assessment consists of the following steps: Assign Orderer and Customer Select Goals Define Organizational Scope Determine Process Areas Organise Assessment Team Plan Assessment Perform Assessment Support ImprovementsThe purpose of the first six steps is to make an assessment proposal. This document fully definesthe assessment, and after approval it is used to perform the assessment and support theimprovements. The steps will be discussed in detail in the following paragraphs.2.1 Assign Orderer and CustomerThe first step is to define the interface between the assessment and the organisation. Thisinterface consists of two persons: An orderer and a customer. They will play an essential role.The orderer has to be a member of the highest management of the organisation. The orderer isresponsible for one or more goals that will selected for this assessment, for that reason (s)he isparticularly interested that a good assessment will be done. Also, this person arranges funding forthe assessment, including hours of the interviewees. It is also the person who has endReaching Business Goals with Value Adding CMMI Assessments, Ben Linders Page 2 of 12 3. The customer is the person who needs the result from the assessment: The findings. This personis responsible for implementing improvement actions. It can be the same person as the orderer. Itmust be clear, before the assessment is started, that the customer is able to take over the results ofthe assessment, and arrange the improvement actions. If there is no room in the organisation toimprove, then there is no need to assess either.Both the orderer and customer must be clearly stated in the assessment proposal. They will alsoattend kick-off and final meeting, where they personally show their stakes in this assessment andwhat they require the assessment to deliver.2.2 Select GoalsFor every assessment, there must be a business reason to do it. This reason has to do with one ormore goals of the organisation, or with strategies performed to reach the goal(s). This step resultsin a list of goals and strategies that have to be assessed.For instance, an organisation can have the goal: All projects must be finished on the planned enddate. To reach this goal, there are strategies such as improvement programs for project planning,and lead-time improvement. When it is uncertain if the strategies will assure that the goal will bemet within the defined time frame (usually one or more calendar years), then an assessment isuseful to investigate it.Goal setting is a complex, but vital activity. This assessment approach requires that the goals areclear, but sometimes it turns out that when discussing them, additional clarification is needed.Also priorities between goals are not always sufficiently defined. It is of the utmost importancethat there are no more uncertainties about the goals before the assessment planning is started.An assessment focuses on a subset of the goals, in order to come with findings specifically relatedto reaching those goals. Assessing against all goals of an organisation results in too manyfindings, where priorities will have to be set after an assessment. In such a case, findings will bethrown away and valuable assessment time has been wasted. More important: An expectation wasgiven that the problems would be solved. Not living up to this expectation can seriously hamperfuture assessments and improvement programs.The experience is that assessments at the Charging Competence Centre have been focussing on 1to 2 main goals. Further on in this paper it will become clear how it was assured that all the goalswere covered by assessments over time.2.3 Define Organizational scopeThe result of the previous step is a list of goals and strategies. In this step the part of theorganisation involved in reaching these goals or performing the strategies must be identified. Theresult will be a list of groups and roles/functions, which have to be interviewed.Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders Page 3 of 12 4. A goal is assigned to a responsible person or group in the organisation. Also, other persons andgroups are performing work that relates to the goal. The boundary of the assessment must bedefined such that it includes the right parts of the organisation, nothing more, and nothing less.A misunderstanding is that in an assessment, every group of the organisation has to berepresented. This should not be the case. If groups are not involved in activities related to thegoals to be assessed, then there is no need to interview them. Any findings from this group wouldbe put aside and considered irrelevant, because they do not endanger the goal. So why waste theirtime, and yours?So the scope includes only groups giving input to the work, performing the work, and requiringoutput from the work, where the work is that part of all work done in the department that isrequired to reach the goal or perform the strategy.2.4Determine Process AreasNow that the scope of the assessment is defined from the organisations perspective, lets look atthe scope from the models perspective. That is, given the organisational scope there is usuallyneed to assess only a selected set of process areas from the CMMI model.Looking at the same example that the organisational goal to investigate is lead-time precision,and the groups involved would be two product development teams running projects, and a staffgroup on configuration management. It was decided in the organisational scope step that theassessment should only interview the project management staff, and not the engineers. In thatcase, it would be useful to select process areas mainly from the project management cluster, andadditionally add configuration management from the support cluster. There is no need to assessprocess areas from engineering or process management clusters, since no activities from theseprocess areas are performed within the organisational boundaries of the assessment. Again thisprevents coming up with findings which are not goal related.Some experience from the Charging Competence Centre department: In the assessments done in2000 and 2001, the number of process areas varied a lot. There were assessments with only 1(Decision Analysis and Resolution) or 2 (Requirements Management/development) process areas,but there have also been assessments with 7 areas. The latter is about the maximum, when itbecomes more then one should return to the first step and try to reduce the number of goalsinvolved. There has been no assessment covering all process areas on a specific level.The result of this step is a list of process areas that will be assessed. With this step, the total scopeof the assessment is defined.2.5Organise Assessment TeamUntil now, there has only been an Assessment Team Leader. This person has done all previoussteps, together with persons from the line organisation. Now the team can be assembled, so thatthey can take the next steps together.A team consists of a team leader, and 1 to 2 assessors. The minimum relates to the fact that it isessential to have at least two views on the findings. At any time an assessor can discusspreliminary findings, and do a peer review before presenting them to the interviewees. TheReaching Business Goals with Value Adding CMMI Assessments, Ben LindersPage 4 of 12 5. maximum relates to the number of process areas and communication; adding more people (unlessthey are expert in a certain area) doesnt improve the quality nor the time needed.In an assessment team there has to be at least 1 member from the assessed organisation. The roleof this local member is to assure that the right people are interviewed, and that potentialfindings within the goals are derived, which are formulated using terminology known and usedwithin the organisation. This helps coping with the so-called CMMI speak problem, where anorganisation doesnt understand the questions and cannot use the findings, because of terms usedfrom the CMMI model that they do not understand.Assessment team members must have excellent communication skills, have knowledge both inthe way an assessment is conducted and in the process areas being assessed, and must havesufficient time available within the time frame. The latter is very important, as an assessment hasa fixed time schedule from start to finish. You cannot postpone a consensus or a final meeting ifyou run out of time, re-planning meetings with all the people involved will almost always meanthat an assessment slips, and then loses the momentum in the organisation.Roughly the time needed in an assessment is 40-60 hours for a team leader, and 20-40 hours foran assessor. This depends of course on experience, and on the scope of the assessment.2.6 Plan AssessmentThe next step now that the part of the organisation to be interviewed and the process areas aredefined, is that the assessment team makes a detailed plan of the assessment. This plan includes: Select Interviewees Define Timeplan2.6.1Select IntervieweesThe interviewees have to be selected within the defined organisation scope (groups androles/functions) defined earlier. The purpose of selecting interviewees is assuring that everyprocess area is sufficiently covered. That is, the group of people interviewed should containsufficient persons that perform the activities of the process areas from several angles.Not only persons doing the work should be interviewed, but also persons ordering (and beingresponsible for) the work, and persons depending on the outcome of the work, should beinterviewed. This can be compared to a 360-degree appraisal, where a persons boss, customersand people below him/her have to give input on the performance.Given the previous example (lead-time in project management teams and configurationmanagement staff, with process areas from project management and configuration management)the following roles could be interviewed: Project managers Performing project management activities Team leaders Idem, but for the design team...