Download - Industry - Estimating software maintenance effort from use cases an industrial case study
Estimating Software Maintenance Effort from Use Cases: an Industrial Case Study
Yan Ku, Jing Du, Ye Yang, Qing Wang
Institute of Software, Chinese Academy of Sciences
2011-09-29
*This work is supported by the National Natural Science Foundation of China under Grant Nos. 90718042, 60873072, 60903050 and
61073044; the National Hi-Tech Research and Development Plan of China under Grant Nos. 2007AA010303, 2007AA01Z186 and
2007AA01Z179.
Agenda
� Motivation
� Industrial Setting
� Methodology
� Case Study
� Discussion
� Tool Implementation
22
Motivation
� An anecdote in software engineering domain*:
� Elephant: Discipline
� Monkey: Agile
� Elephant & Monkey: Practical
33
*Barry W. Boehm, Richard Turner, Balancing Agility and Discipline: A Guide for the Perplexed, Aug, 2003
*Picture Source: www.image.baidu.com
Motivation
� “Speaking of the problems we are facing during software lifecycle, hmmm, there is no way a short list. … I hate to, but I have to say that effort estimation especially for maintenance project, if not totally impossible, is really a big challenge
-----A Project Manager
� Formal estimation models: time-consuming, requiring
enough information and data---- the elephant
� Expert Judgment: expert-dependent, easily lead to
thumb-up---- the monkey
44*Picture Source: http://www.zcool.com.cn
Industrial Setting
� The industrial research is inspired by the estimation
dilemma a mentioned before.
� The problem occurred in developing the leading product
of a medium-size software enterprise with CMMI ML4
in China.
� The product, named QONE, is a commercial software
process management tool.
� It has contributed to the process improvement for more
than 300 small and medium-sized software companies
and organizations in China.
55
Industrial Setting (Con)
� Since 2004, QONE has released several major versions as well as branches for special customized ones in succession.
� Several of the evolving versions are maintenance projects.
� Expert estimates were mainly used in the past effort estimation of QONE.
� The estimation results are not so stable due to the objectivity and other issue.
� The actual effort and other data including use case documents have been accumulated by QONE itself.
66
2009-3-202008-9-1v7
2008-8-212008-3-20v6
2008-7-312007-12-10v5
2007-10-312007-5-28v4
2007-3-302006-1-16v3
2005-11-302005-7-11v2
2004-11-152004-10-8v1
End DateBegin DateVersions
Methodology
� Goal: Achieving the balance of simplicity, early-estimating and accuracy in one effort estimate .
� Methodology Principles:
� Apply use cases as the size metric and introduce requirement elaboration factors to make the estimate in advance
� Introduce adjusted factors as few as possible in order to reduce the complexity
�Take advantage of the history data to help improve the estimation accuracy.
77
Maintenance task type is not distinguished
due to the difficulty for effort classification
Modeling Process
88
Get the lowest level
requirementsUse the same
data unit
Count Data
� Use case: number of use cases � newUC: new-added
� modUC: modified
� reuUC: reused without modified
� delUC: delete
99
Modeling Process
1010
Count Data & Construct Model
� Weight � Wmod/ Wreu / Wdel :effort ratio of
modified/reused/deleted use case to a new-added one
1111
Sizeadjusted = newUC + Wmod * modUC
+Wreu * reuUC + Wdel * delUC
Effort = A * (Sizeadjusted) B
Where
Effort is the maintenance effort;
Sizeadjusted is the adjusted product size;
A is the multiplicative calibration constant;
B is the exponential calibration constant
Modeling Process
1212
Validation
� Metric Definition
� MRE = |predictive effort – actual effort| / actual effort
� Referred Measures
� MMRE: mean magnitude relative erro
� MdMRE: median magnitude relative error
� PRED25: the % of the data points with RE<=0.25
� PRED30: the % of the data points with RE<=0.30
1313
Prediction Process� Data collection
� Prediction
� To apply in different phases during lifecycle,
elaboration factors are referred to estimate size input.
� Sizere-adjusted = EF * Sizeadjusted,where EF is the elaboration factor between higher and lower-level requirements
� whereNRi is the number of higher-
level requirements and NUCi is the number of lower-leverlrequirements in ith data point
1414
∑=
=
n
1i iNRiNUC
n
1EF
*Picture Source: Alistair Cockburn, Writing Effective Use Cases
Case Study
� Historical data are the projects mentioned in
slice 6.
� Requirements are described in three level:
� capability goals (CG, least detailed)
� capability requirements (CR)
� use cases (UC, most detailed)
1515
16164366815Use cases(UC)
05133Capability requirements(CR) 7166
01830Capability goals(CG)
v7
33113037Use cases(UC)
032202Capability requirements(CR) 14903.6
01281Capability goals(CG)
v6
153013112Use cases(UC)
233181Capability requirements(CR) 7477.5
17121Capability goals(CG)
v5
172296150Use cases(UC)
417333Capability requirements(CR) 10340.1
36131Capability goals(CG)
v4
81349486Use cases(UC)
0123012Capability requirements(CR) 30945
03154Capability goals(CG)
v3
0207227Use cases(UC)
026160Capability requirements(CR) 3941
07110Capability goals(CG)
v2
0216103Use cases(UC)
03570Capability requirements(CR) 2284.5
01350Capability goals(CG)
v1
Actual Effort
(person/hour)
Number of
deleted
Number of
reused
Number of
modified
Number
of newRequirements LevelsVersions
Case Study
� Weight:
� Wmod = 0.2, Wreu = 0.05, Wdel = 0
1717
Sizeadjusted = newUC + 0.2 * modUC +0.05 * reuUC
0.913862Adjusted R2
0.928219R2
0.000481P-value
1.192651B
96.9396A
ValuesParameters
Effort = 96.9396 * (Sizeadjusted) 1.192651
Validation Result
� Leave-one-out cross validation is applied.
� Elaboration factors are referred from A. A. Malik’s research* since the data used there is the subset of our dataset.
1818
*A. A. Malik, B. W. Boehm, Y. Ku, and Y. Yang, “Comparative Analysis of Requirements Elaboration of an Industrial Product,”
Proceedings of the 2nd International Conference on Software Technology and Engineering(ICSTE 2010), Oct. 2010, pp. 46-50
0.85710.71430.5714PRED30
0.71430.57140.4286PRED25
20.01%24.13%27.09%MdMRE
26.94%26.18%36.87%MMRE
UCCRCGMetrics
UC yields the best
result
Validation Result (Cont)
1919
7.506628.5125716634.9v7
20.0111921.178614903.658.55v6
17.946136.34557477.533.25v5
86.3019263.978010340.173.65v4
24.7823276.772730945111.5v3
4.393767.9859394121.75v2
27.682916.89252284.515.8v1
MRE (%)Predictive EffortActual EffortAdjusted SizeVersions
1. COTS has been used
2. Increased productivity through
requirement management
Method Comparison
� Analogy � Database used for analogy is from China Software
Benchmarking Standard Group (CSBSG)
� 999 software project data from 140 organizations distributed in 15 regions across China
� COCOMO2000
2020
0.85710.714320.01%26.94%Use case
0.57140.571415.47%32.47%COCOMO
0.28570.285731.59%33.09%Analogy
PRED30
PRED25
MDMREMMREMethods
Discussion
� Lessons Learned:
� Use Case Metrics
� Requirement Elaboration Factors
� Advantages of use-case based estimation
� Linear vs. Exponential relationship between effort and use case
� Threats to Validity
� Internal threats:
� Weak outlier tolerance
� Complexity of use cases
� External threats:
� Use case weight
2121
Tool Implementation
2222
Than You!Our Lab: http://itechs.iscas.ac.cn