1.264 Lecture 5

System Process: CMMI, ISO

Next class: Read UML chapters 1 (skim), 2 (skim), 4, 8, 9 Make sure you have Visual Paradigm installed. We start using it Monday in class.

Homework 1 solutions posted Monday

1

Discussion: Case studies

Chemical inventory case study List the errors made Outline resource estimation steps that should have been

used Outline process steps that should have been followed

RFID case study What lifecycle model would you use? How would you give a time estimate? List major project risks and how to mitigate them

2

Capability Maturity Model Integration

Developed at Software Engineering Institute (SEI), Carnegie-Mellon University (www.sei.cmu.edu)

Method for process assessment and improvement Five level model of maturity and capability

1: Initial/Performed 2: Managed 3: Defined 4: Quantitatively Managed 5: Optimizing

Predictability, effectiveness and control of software processes improve as organization moves up these levels

One old reading on CMM/CMMI is on course Web site Current material is at http://www.sei.cmu.edu/cmmi/

3

CMMI Motivation

CMMI covers three interlocking areas: Development Services Acquisition

CMMI Development: 20 years of unfulfilled promises about productivity and

quality gains from new software technology Databases, object oriented programming, Internet,

Organizations realized fundamental problem is the inability to manage the software process

CMMI provides guidance on how to evolve toward a culture of software engineering and rational management

4

CMMI Maturity Level 1: Initial

Ad hoc, occasionally chaotic Few processes defined Success depends on individual effort and

heroics This is what youre taught to do in school

5

CMMI Maturity Level 2: Managed

Basic project management processes established to track cost, schedule, functionality

Discipline in place to repeat earlier successes on projects with similar applications

Key processes focus on basic project management controls Requirements management: initial spec, change control Software project planning: resource estimation based on past

performance Software project tracking and oversight Software subcontract management Software quality assurance: code reviews, test plans, tracking Software configuration management

At level 2, you can measure whats going on, and that helps understand future projects

A project or group can assess at level 2 6

CMMI Maturity Level 3: Defined

Software process for management and development is documented, standardized and integrated into an overall process for the organization

All projects use approved, tailored version of standard process

Key process areas focus on institutionalizing effective process Organization process: spiral model, tracking, control Training program: managers, analysts, developers Integrated software management: lines of code, defects, time, Software product engineering: alternative designs, scopes Intergroup coordination: Web, database, analyst, coder, customer Peer reviews: initiate at requirements, design, QA. Continue code

reviews At level 3, you begin to have some control; you can actually

predict times/costs for new projects and make some choices on how to approach them (fastest, most efficient, nominal)

The entire organization must assess at level 3; you cant do it for an individual project or group

7

CMMI Level 4: Quantitatively Managed

Detailed measures of software process and product quality are collected, in addition to product measures

Process and products are quantitatively understood and controlled

Key processes focus on quantitative understanding of process Quantitative process management: measure and manage time

at all stages: requirements, design, code, QA Software quality management: reviews at all stages, tests and

traceability at all stages

At level 4 you have real control: you can measure and manage all aspects of the project

8

CMMI Level 5: Optimizing

Continuous process improvement through quantitative feedback

Piloting innovative technology and ideas Key process areas focus on continual process

improvement Defect prevention (extreme programming, refactoring,

reviews) Technology change management (new tools, training, pilots,

tests, knowledge sharing) Process change management

At level 5, you not only have control but are efficient

9

10

/$/

/$/

/$/

/$/

5

4

3

2

1

Time ....

Time ....

Time ....

Prob

abili

ty

Time ....

With well-defined processes, performance improves in Level 3 organizations

Plans based on past performance are more realistic in Level 2 organizations

Schedule and cost targets are typically overrun by Level 1 organizations

PROCESS CAPABILITY AS INDICATED BY MATURITY LEVEL

Based on quantitative understanding of process and product, performance continues to improve in Level 4 organizations

Target N-y

Target N-x

Target N+a

Target N

Performance continuously improves in Level 5 organizations Target N-z

Image by MIT OpenCourseWare.

11

Image by MIT OpenCourseWare, adapted from Carnegie Mellon University, Software Engineering Institute.

Not Given Initial Managed Defined Quantitatively Managed

Optimizing

3600

32003400

2800260024002200200018001600140012001000800600400200

0

3000

Num

ber

of O

rgan

izat

ions

Based on March 2011 appraisal of 5346 organizations

Process Maturity Profile by All Reporting Organizations

12

Image by MIT OpenCourseWare, adapted from Carnegie Mellon University, Software Engineering Institute.

100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%25 or fewer 26-50 51-75 76-100 101-200 201-300 301-500 501-1000 1001-2000 2001 +

0.8%

0.4%

0.7%

4.6%

0.5% 1.1%

1.4% 4

.9%

0.7% 2.

4% 3.1% 6

.4%

21.6

%

3.3% 5

.9% 7.9%

0.9%

20.9

%

20.7

%

3.6%

9.4%

9.0%

1.6% 5

.3%

5.4%

0.3%

10.4

%

17.0

%

4.8%

16.0

%

8.0%

0.8%

14.3

%

5.2%

23.9

%12

.1%

24% 5.6

%

0.8%

7.1%

1.2%

1.2%

47.3

%

45.5

%

65.1

%

63.1

%

62.8

%

57.1

%

52.9

% 55.4

%

47.8

% 51.6

%

27.4

%

51.8

%

38.8

%

27.3

%

25.8

%

% o

f or

gani

zatio

ns b

y si

ze

Maturity Profile by Organization SizeBased on the total number of employees within the area of the organization that was appraised

Based on 5301 organizations reporting size data as of March 2011

Not Given

Initial

Managed

Defined

Quantitatively Managed

Optimizing5.

0%

13

Image by MIT OpenCourseWare, adapted from Carnegie Mellon University, Software Engineering Institute.

40

30

20

105

20

28

28.5

Num

ber

of m

onth

s to

mov

e to

ne

xt m

atur

ity le

vel

Time period of initial appraisal

2006 to March 2011

Level

Orgs

1-2 2-3 3-4 3-5

7 227 50 46

75th Percentile

Median

25th Percentile

Time to Move Up

Exercise

Briefly assess the pros and cons of using spreadsheets (Excel) as the major analysis method in running a business or managing a project

14

Solution

Pros: Spreadsheets are very quick to develop Most staff know how to write and use them

Cons: Incorporate decision making that is not known to central

organization Hard to pass on spreadsheet to successor or other user Spreadsheets rarely documented or tested well Spreadsheets are rarely completely correct No business rules, like a database

Heavy use of spreadsheets indicates a low process maturity organization

15

ISO 9001:2008 International Standards Organization (ISO)

National standards bodies from 100+ countries ISO 9001:2008

First major modification of ISO standard in 20 years occurred in 2000, and was updated in 2008

No longer primarily a manufacturing standard, with military origins, implemented as books of procedures

New standard addresses all products and services in all industry segments

The new standard requires 8 principles be implemented: Customer focus Leadership Involvement of people Process approach System approach to management Continual improvement Factual approach to decision-making Mutually beneficial supplier relationships (The old standard didnt really require anything) 16

ISO 9001:2008 cont

ISO 9001 certification important to do business in Europe, and becoming so in Pacific Rim and eventually Americas

General approach Create a quality steering team in management Create project plan for ISO 9001 certification Perform gap analysis (good process vs current process) Write needed system development, quality procedures

Often takes 6-12 months Train employees in new procedures Use new materials, retain records 3 certification options: self, clients or independent body

17

Exercise

Steelmaker: current software Steelmaking process control software: custom Substantial raw material and outbound logistics: custom Enterprise resource planning software: custom and SAP Customer relationship mgt software: software-as-service Demand forecasting: best-of-breed from small vendor

Proposed project: Integrate demand forecasting and logistics

Current organization large, not integrated (silos) Steelmaking process is ISO 9000, nothing else is ISO/CMMI

What process suggestions do you make? Which basis: CMMI or ISO? At what CMMI level is the organization likely to be now? List a set of initial steps to take How long will it take to see substantive process change?

18

Solution (one of many)

What process suggestions do you make? Which basis: CMMI or ISO?

Could justify either ISO 9001: involves major parts of enterprise, not heavy on

system development. CMMI: choose to focus on systems aspect

At what CMMI level is the organization likely to be now? Level 2, since there is ISO 9001 certification Perhaps level 1, since most parts are not certified

List a set of initial steps to take ISO 8 principles, or CMMI level 2 steps, to ensure process Then do requirements, design, etc. in a spiral

How long will it take to see substantive process change? ISO often 6-12 months CMMI time to move up from level 1 to level 2 is 5-10 months

and to level 3 is 20-24 months

19

Intelligent Transportation Systems (ITS) project management process

Source: ITS Systems Engineering Guidebook for Intelligent Transportation Systems, version 3.0

Public domain (US Federal Highway Administration and Caltrans).

20

ITS project management: Vee

Source: ITS Systems Engineering Guidebook for Intelligent Transportation Systems, version 3.0

Public domain (US Federal Highway Administration and Caltrans).

21

Comparison of CMMI and ISO

CMMI focus: Software engineering practices Program/project management practices Increasing maturity levels Appraisals when going to next maturity level

ISO 9001:2008 focus: Customer satisfaction Process discipline across entire organization Continual improvement Principled management

ITS: http://www.fhwa.dot.gov/cadiv/segb Transportation-specific, includes construction process

22

MIT OpenCourseWarehttp://ocw.mit.edu

1.264J / ESD.264J Database, Internet, and Systems Integration TechnologiesFall

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.