improving measurement plans from multiple dimensions: exercising with balancing multiple dimensions...

Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005

1

Improving Measurement Plans from multiple dimensions: Exercising with Balancing Multiple Dimensions (BMP)

11stst Workshop on Workshop on «« Methods Methods for for Learning MetricsLearning Metrics »»19 September 2005, Como (Italy)19 September 2005, Como (Italy)

ÉCOLE DE TECHNOLOGIE SUPÉRIEURE – MONTRÉAL - CANADA

Luigi Buglione & Luigi Buglione & AlainAlain AbranAbran


2

• Introduction• Multidimensional Analysis in Project Management

ü Why is it needed?ü Some Multidimensional Modelsü What should be measured and analysed?ü Which set of indicators to select?

• BMP: Balancing Multiple Perspectivesü The proposed measurement procedureü An example using 4 dimensionsü Measuring project performances from multiple views

• Applying BMPü BMP-Q: the questionnaireü BMP-Q: the metricsü First Results & Feedbacks

• Conclusions & Prospects

Agenda


3






Agenda


4

Introduction

• Growing interest in Integrated Software Measurement…ü E.g. BSC, EFQM, MBQA

• …but still few documented industrial implementationsü Usually Time and Cost dimensions used

• Other possible dimensions of analysis (eg: Quality, Risk, …) are not often taken into accountü Q: how much does it cost project monitoring & control?ü Q: how many measures/indicators are usually tracked during

the project lifecycle? And from which perspectives?• Objective:

• to optimize the cost for tracking & control projects, balancingthe number of measures/indicators used by each perspective of analysis useful to the project


5






Agenda


6

Multidimensional Analysis in PMWhy is it needed?

• Loss of project control is one of the most frequent causes of failure in Project Managementü Prevention: detailed analysis of content and quality of project

tracking • Some basic questions: ü Right number of perspectives?ü Right number of indicators?ü What about hypothesis of relationships among processes?

At least, 3 dimensions:ü Managementü Usersü Technical


7

Multidimensional Analysis in PMSome examples


8

Multidimensional Analysis in PMWhat should be measured and analysed?

STARSTAR taxonomy: a broader view on Measurement & Analysis


9

Multidimensional Analysis in PMWhich set of indicators to select?

• Q: What is the right number of indicators to use? – The Miller’s “magic number” 7 ± 2?

• General suggestions to avoid the misbalance in selecting the measures critical to success, whatever the numberØ “Select a small suite of key measures that will help

you to understand your group’s work better, and begin collecting them right away, measuring several complementary aspects of your work, such as quality, complexity, and schedule.”(Karl Wiegers)


10






Agenda


11

BMP: Balancing Multiple PerspectivesGeneral issue

• Q: how can a proper balance of perspectives and indicators be selected when managing a portfolio of projects?

PEANUTS © United Feature Syndicate, Inc.

The problem is not to reduce the cost of measurement, but optimising it against the informative value provided by the number

of measures/indicators balancing them by each perspective of analysis


12

BMP: Balancing Multiple PerspectivesThe proposed Measurement Procedure

å Determine the dimensions of interest in the projectdimensions of interest in the projectç Determine the list of the most representative measures measures

associated with each dimensionassociated with each dimensioné For each of the measures selected, identify which other

control variables might be impacted negativelyè Figure out the best combination of indicators and the best combination of indicators and the

causal relations between themcausal relations between them in order to build a build a measurement planmeasurement plan for the project


13

BMP: Balancing Multiple PerspectivesA Generic four-dimensional BMP


14

BMP: Balancing Multiple PerspectivesAn example with 4 dimensions

å Determine the dimensions of interest in the projectdimensions of interest in the project

A: Time, Cost, Quality & Risk


15


ç Determine the list of the most representative measures associated measures associated with each dimensionwith each dimension – Note: excerpt from the PSM Guide


16


ç Determine the list of the most representative measures associated measures associated with each dimensionwith each dimension – Note: excerpt from the PSM Guide


17


é For each of the measures selected, identify which other control variables might be impacted negatively


18


è Figure out the best combination of indicators and the causal best combination of indicators and the causal relations between themrelations between them in order to build a measurement planbuild a measurement plan for the project


19


è Figure out the best combination of indicators and the causal best combination of indicators and the causal relations between themrelations between them in order to build a measurement planbuild a measurement plan for the project


20

BMP: Balancing Multiple PerspectivesMeasuring projects’ performances from multiple views

• Q: What about a possible usage of BMP for measuring the overall project value within a BSC logic?– Pros: BSC helps in managing multiple perspectives – Cons: BSC does not provide the integrated measurement

results

• A: a joint usage of QEST nD model with BSC framework

• A: BMP can help as a tool for considering the counter-productive impacts of a possible control action in a project by each BSC perspective

SourceSource: A.Abran & L.Buglione, A Multidimensional Performance Model for Consolidating Balanced Scorecards, International Journal of Advances in Engineering Software, Elsevier Science Publisher, Vol. 34, No. 6, June 2003, pp.339-349


21






Agenda


22

Applying BMP BMP-Q: the Questionnaire http://www.geocities.com/lbu_measure/qestlime/bmp.htm

http://www.geocities.com/lbu_measure/qestlime/bmp.htm


23

1a 1 Respondents profile by project role (# and %)

2 Experience profiles for current project role (# and %)

1b

3 # analysis viewpoints (OLD)

1c 4 # analysis viewpoints (NEW)

2 1 # selected measures (OLD)

2 # selected metrics (NEW)

3 # affected viewpoints (NEW)

4 Avg of measures by viewpoint (# and %)

5 Ranking of selected measures by: abs value, respondent project role, analysis viewpoint

3a 1 List of causal relationships among measures

2 Ranking of relationships by: abs value, respondent project role, analysis viewpoint

4a 1 % respondents knowing amount of costs for m&c (monitoring & control) activities

4b

1 Max, Min, Avg and Med for the returned values (%) – OLD

4c 1 Max, Min, Avg and Med for the returned values (%) – NEW

Applying BMP BMP-Q: the Metrics http://www.geocities.com/lbu_measure/qestlime/bmp.htm

http://www.geocities.com/lbu_measure/qestlime/bmp.htm


24

Applying BMPFirst Results & FeedbacksTeaching viewpoint:• simplify the way for explaining the ISO/IEC 15939:2002 Information Model

and introducing the cause-effect linkage among development processes, as a as a useful foundation for ICT Balanced Scorecards (BSC)

• help in pointing out to practitioners and students that there are plenty of well-proven and established measures within the Software Engineering domain, but also that each Measurement Program must be built having in mind that unique situation and that every measure should be designed and verified to be effectively goal-driven

Industrial viewpoint:• increase awareness about cost-benefit analysis in implementing (or not) a

certain measure/indicator within their current QMSs. ü Stimulus for industry people to know more and more about the effort/cost splitting of

their projects by SDLC phase (in many cases the answer is: don’t know)ü Proposed the Test Selection Algorithm: each measure has its own cost from its

definition till the project closure, within a budget constraintü Key point: balancing the current available amount of budget and not to simply cut

controls. And in that eventuality, the point to face was: but which drawbacks of deleting or adding some controls?

• Proposed to re-label such measures and indicators as project “controls”, to provide a better match with managers’ responsibilities


25






Agenda


26

Conclusions && Prospects• Project managers often considers only two dimensions for tracking & control

of their projects (Time, Cost)• At least, the Quality perspective should be also taken into account; further

perspectives (eg: Risk) could be also be useful if considered from the planning phase on. Even challenging, a multi-perspective approach – as in the BSC – is suggested

• It does not exist a “magic number” of indicators to track, but the goal is to optimize costs and informative value derived from that amount of indicators, establishing also the causal relationships among their related goals

• BMP (Balancing Multiple Perspectives) proposes a 4-step procedure to select an appropriate balance of indicators from the various perspectives taken into account (e.g. Time, Cost, Risk and Quality) and focus on the core indicators from each of them, thereby helping the project manager in tracking and control activities

• Due to its inner multidimensional nature, future joint usages with methods, tools and frameworks taking into account concurrent dimensions (eg: QEST/LIME) will be investigated


27

Q && A


28

[email protected] , [email protected]

Thank you!

mailto:[email protected]

mailto:[email protected]