inf5181: process improvement and agile methods in systems development

INF5181 / Lecture 08 / © Bente Anda 2011, Slide 1

INF5181: Process Improvement and Agile Methods in Systems Development

Lecture 08, 3.11.2011:

Software process improvement in practice:

Estimating software development effort

Bente Anda

Senior advisor, Norwegian Tax AdministrationAdjunct associate professor, IFI

email: [email protected] 2011

mailto:[email protected]

mailto:[email protected]


Software process improvement (SPI)

• A process is a series of steps, that is, what we do, to produce a result

• A software process is the set of activities, methods, and practices used in the development and maintenance of software

– This lecture focuses on one sub-process, the estimation process

• Software process improvement is change of some kind to the way software development and maintenance is conducted

• Organizations do software process improvement to reduce defects, improve productivity, lower costs…

• But, changing work habits is difficult, and SPI frequently fails…


• Appr. 320 people working on software development as well as a large number of consultants

• Appr. 12 ongoing software development projects• Several hundred change requests to existing systems each

year• Methods:

Software development at Skatteetaten

Smidig / Scrum

UP/UML

SMART

Project management Software development

PMBOK


Effort estimation

• Definition of estimate: Most likely effort to accomplish a software development task

• Estimates of software development effort are necessary to manage scope, effort and resources Scope

Time Resources

• A challenge of defining an estimation process and methodology to improve effort estimation is that estimates often contain an element of magic

X?


Effort estimation dilemma

• An effort estimate is often produced early when there is a large uncertainty with respect to the software product and software development project

• An estimate may have a large impact on our priorities (and consequently on what we deliver) and on how well we succeed in delivering it


The role of effort estimates

Process for estimationProcess for estimation

Estimates Estimates

Management (prioritization and descision) Management (prioritization and descision)

Project

Predictability resources

Predictability resources

Predictability delivery

Predictability delivery

Predictability scope

Predictability scope

scope

time resources

Portefolio

strategy/goal

time resources

Control strategy/goal

Control strategy/goal


Motivation for software process improvement within effort estimation

• Not all software development is estimated

• The estimates are not always used in prioritization, resource allocation and planning

• Estimates often have a very large risk buffer

• Little documentation of prerequisites and assumptions for estimates

:


Motivation cont.

• Too high estimates mean – unnecessary costs since estimates of time and resources

tend to be fulfilled, and– productivity is too low

• Too low estimates mean – other projects/change requests that would have given better

cost-benefit are not prioritized, and – additional costs when consultants must be hired to “save”

projects• Given the size of the organization, consequences are high

Stages of SPI – the IDEAL model

IDEAL SEI`s approach to software process improvement


Keys to successful SPI 1. Define and communicate the problem to be solved through the SPI

initiative throughout the organization

2. Align the SPI initiative with business strategy and goals

3. Be sensitive to the organizational context

4. Gain commitment from key stakeholders

5. Dedicate resources to manage the implementation and coordinate the process improvement activities

6. Invest in the necessary tools/infrastructure

7. Document key metrics before and after the process change

8. Publish evidence of SPI success

9. Learn from the feedback results

10.Be aware that SPI is change management


Challenges of SPI

Technological challenges:• Many methods, techniques and tools for effort estimation to

choose from

Organizational challenges:• Many stakeholders• Complex related processes that frequently change• How to measure improvement


Challenges of SPI - Technological

• Very diverse tasks need to be estimated

1. Types of task: Small change requests, user stories… large projects

2. Requirements may be known or uncertain

3. Development platform may be familiar or unknown

4. Resources may be known or uncertain

…

• Technology is complex and a moving target, and there are many estimation techniques:

1. Estimation by analogy2. Parametric estimation3. Use case based estimation4. Wideband Delphi5. Successive estimating6. Bottom-up estimation based

on a work breakdown structure

7. Planning poker…

Which technique should be used when?


Challenges of SPI – Organizational

• Skills – ”I don`t know these techniques, so how can we use

them?”• The nature of effort estimation

– “An effort estimate is nothing more than a guess anyway”, and

– “A number can be guessed very easily without any technique, so why should I bother to use a technique”

• Measurement –– “We will not reveal bad practices”


Stages of this SPI project

Training

Baseline

Development of methodologyBest practice

Goals

Sponsorship

Motivation

Iteration 1

Pilot 1

Pilot 2

Pilot 3

Iteration 2 Iteration 3 Iteration 4

Measurement

Infrastructure


Stages of SPI project - Initiating

Stimulus for improvement: Gartner report

Set goals for the improvement

Establish sponsorship

Establish improvement infrastructure

Goals

Sponsorship

Motivation

Infrastructure


Qu

anti

fiab

le

Financial Non-financial

No

n-

qu

anti

fiab

leSet goals

• Better investments caused by better prioritization

• Better use of resources

• Better possibilities for assessing offers from external suppliers

• Less need to hire additional resources to ”save” projects

• Less effort required to produce estimates

• Estimates of higher quality

• Less need for re-planning caused by budget overruns

• More satisfied employees

• Better reputation in the Department of Finance

• Improved predictability with respect to project completions

• Improved understanding of uncertainty in estimates and of risk

• Improved project management through – Improved understanding of cost drivers– Routines for collecting effort data

throughout the projects

• Documented practice that enables continuous improvement


Establish sponsorship • The SPI project was part of large project to renew IT function

– Goals

• Reference group with:– Member of IT staff– Project director– Manager of architecture team– Senior manager

Improved strategic management and more efficient deliveries

of IT services

More efficient maintenance and evolution, right quality

on service deliveries

Increased capacity for development

Simplified and renewed portfolio of systems

Free resources Realize

Reduced costs Reduced development time


Establish improvement infrastructure

• Responsibles for the SPI initiative:

– one senior project manager

– one expert on SPI and effort estimation

• Wiki


Stages of SPI project - Diagnosing

Establish a baseline of current practice for effort estimation

Develop recommendations based on best practice

Baseline

Best practice


Techniques used in SPI

• Interviews– Interviews are used to access to tacit knowledge– We used interviews as a primary means to gather a baseline and information

about best practice of effort estimation in Skatteetaten and in comparable organizations

• Workshops– Workshops are conducted to do some work and create a product– We used a workshop to

• describe best practice from the interviews and the collected material• make the first version of the estimation methodology

• Literature reviews– Literature reviews are conducted to establish evidence-based best practice– We used a literature review to identify relevant best practice


Interviews

• Interviewees: Project and group managers with extensive experience from estimating software development effort (internal and from a number of external organizations)

• Duration: 1.5 hours• Semi-structured interviews

• Interview guide:1. What are the estimation units?

2. What is input to estimating?

3. Who are responsible for estimating and who participates?

4. Which estimation techniques and tools do you use?

5. Do you have historical effort data?

6. How are earlier experiences with effort estimation used?

7. When do you estimate and re-estimate?

8. How do you document the estimates?

9. Positive and negative experiences with existing estimation practice?


Workshops

• Participants: – Facilitator, documenter (SPI team), IT staff, project manager, 2

managers, 2 developers (all very experienced)– Participants should be chosen according to their influence in

the organization, their skills and interest in the topic, and to what extent they will be impacted by the outcome of the workshop

– Duration: 5 hours• Preparation:

– Informed about focus of workshop and expected results

– Distributed all material from interviews – Room: Walls for notes

Best practice for effort estimation at Skatteetaten-excerpt


Literature review

• Identify, evaluate and put together

relevant studies to one whole

• Protocol for literature review

1. Select relevant databases for search

(www.scholar.google.com)

2. Search for studies

(Best practice software effort estimation)

3. Determine their quality

4. Put together the relevant information from the studies of sufficiently high quality

INF5181 / Lecture 08 / © Bente Anda 2011, Slide 25 25

General best practice for effort estimation1. Understand the functional size of the software task to be estimated

(the most important cost driver)

2. Understand the uncertainty in product and project and include this in the estimate

3. Use group estimation

4. Combine estimates

1. produced by different people

2. and/or by using different technique

and look at where they are different and where they converge

5. Check that the estimate takes all requirements (also the non functional and implicit requirements) in to account and includes all necessary activities

6. Separate estimate from expectations regarding resources


Stages of SPI project - Acting

Development of the methodology in four iterations

Testing of the methodology in pilot projects

Establish measurement scheme

Development

Testing

Measurement


Ideas Prioritization Project Realizing benefts

Early estimatesEarly estimates Planning estimatesPlanning estimates Re-estimationRe-estimation

Forslags-fase

Analyse-fase

Evaluerings-fase

Forstudie-fase

B1 - KvalifiseringB2 - PrioriteringB3 - Vedtak

B3

U1

B2B1

U2

Development

Different estimation processes at different stages


Early estimatesEarly estimates Planning estimatesPlanning estimates

Task estimatesTask estimates

Re-estimationRe-estimation

Development cont.

Small tasks with low uncertaintyEstimation techniques:• Planning poker• Checklist based

Early project suggestionsEstimation techniques:• Wideband delphi• Successive estimating• Use case based estimation

Requirements specification availableDevelopment platform and resources knownEstimation technique: • Bottom-up based on work breakdown structure


ExperienceDB

Estimation template

Historical effort data

Analyze project

Adapt work breakdownstructure

Choose estimation process and techniques

Log estimation data

Choose project methodology and

software development methodology

WBS

Development

Configuration of estimation processes


Design of Wiki (v0.1)


Piloting

1. Workshops where Planning Poker was used to identify important cost drivers for early estimation

2. Adapted templates for early estimation

3. Tested techniques and templates for effort estimation and re-estimation, particular focus on Wideband delphi

4. Tested routines for collecting historical effort data, including use of estimation units according to RUP


Examples of tasks to be estimated with PP to identify cost drivers

• Task 1: Establish connection between single owner company and owner in a specific time period

• Task 2: Make new star for MVA to access historical transactions

• Task 3: Make it possible to access 3 tables from a known source in the control mart.


Measurement

•

Goal Metric

1. Estimates of higher quality Actual effort – estimate before and after estimation method is introduced

2. Less effort required to produce estimates Effort spent on estimation

3. Improved understanding of uncertainty and risk Impression

4. Better use of resources Less need to hire additional resources to ”save”

projectsNOK on unplanned external resources

5. Better assessments of external offers Impression

6. Improved predictability with respect to project completion

Less need for re-planning caused by budget overruns

Actual delivery date – planned delivery# Cases of replanning

7. Better reputation in the Department of Finance Impression


Summary• Effort estimation is a starting point of software development and effort estimates are vital in prioritizing and planning• There are many techniques for effort estimation• SPI is often difficult as there are both technological and organizational challenges

– Technical understanding, – alignment with business goals, – understanding of the relevant organizational processes, and– commitment and involvement from stakeholders is important

• Models and frameworks for SPI may serve as tools for planning your own SPI initiatives• To learn more about effort estimation:

inf5181: process improvement and agile methods in systems development

Documents