peter baxter, distributive management hampton roads incose decision analysis conference - nov 2009 u...
TRANSCRIPT
![Page 1: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/1.jpg)
Peter Baxter, DISTRIBUTIVE MANAGEMENT
Hampton Roads INCOSE Decision Analysis Conference - Nov 2009
USING MEASUREMENT AND THE SAATY METHOD TO CHOOSE THE BEST DECISION ALTERNATIVE
![Page 2: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/2.jpg)
2 OBJECTIVES
Learn: How to construct and weight evaluation
criteria. How to make pair-wise comparisons of
alternatives. How to expand the example for more
complex, nested types of criteria. How a measurement process can support
the Saaty method.D I S T R I B U T I VE M A N A G E M E NT
![Page 3: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/3.jpg)
WHAT IS THE AHP/SAATY METHOD?
![Page 4: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/4.jpg)
4 SIMPLE DEFINITION
Compute the matrix values for a and … voila!
The Analytic Hierarchy Process (AHP)
D I S T R I B U T I VE M A N A G E M E NT
![Page 5: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/5.jpg)
5
COMPARE THE SIZES
OF THESE CIRCLES
A B C D E
A B C D E
A 1 7 9 5 3
B 1/7 1 3 1/5 1/7
C 1/9 1/3 1 1/7 1/9
D 1/5 5 7 1 1/3
E 1/3 7 9 3 1D I S T R I B U T I VE M A N A G E M E NT
![Page 6: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/6.jpg)
6SCALE OF
COMPARISON Don’t need an absolute scale
(if you already know the size of all but one)
Relative scale approximates difference
D I S T R I B U T I VE M A N A G E M E NT
![Page 7: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/7.jpg)
7SIMPLE
EXPERIMENT
One test is worth 1000 expert opinionsD I S T R I B U T I VE M A N A G E M E NT
![Page 8: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/8.jpg)
8WEBER’S
EXPERIMENT
In 1846 Weber found, for example, that people while holding in their hand different weights, could distinguish between a weight of 20 g and a weight of 21 g, but could not if the second weight is only 20.5 g. On the other hand, while they could not distinguish between 40 g and 41 g, they could between 40 g and 42 g, and so on at higher levels. We need to increase a stimulus s by a minimum amount Δs to reach a point where our senses can first discriminate between s and s+Δs. Δs is called the just noticeable difference (jnd). The ratio r = Δs/s does not depend on s.
Weber’s law states that change in sensation is noticed when the stimulus is increased by a constant percentage of the stimulus itself. This law holds in ranges where Δs is small when compared with s, and hence in practice it fails to hold when s is either too small or too large.D I S T R I B U T I V
E M A N A G E M E NT
![Page 9: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/9.jpg)
9 WHY IT WORKS
People are inconsistent at providing an absolute scale to evaluate objects.
How much does Rock A weigh?
How much does Rock B weigh?
There are better at comparing pairs of objects.
Does Rock A weigh more than Rock B?
D I S T R I B U T I VE M A N A G E M E NT
![Page 10: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/10.jpg)
10APPLYING THE
TECHNIQUE
1. Select criteria(s) to evaluate
2. Define comparison scale
3. Perform pair-wise comparison
4. Check consistency
5. Calculate values
D I S T R I B U T I VE M A N A G E M E NT
![Page 11: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/11.jpg)
APPLYING THE METHOD
![Page 12: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/12.jpg)
12
EXAMPLE #1SOFTWARE
ESTIMATION A controlled test where existing software sizes are known.
Ask 30 grad students to estimate the SLOC of common data structures like stack, queue, list.
Estimate SLOC three ways:
1. Guess a number.
2. Compare to one reference structure using numeric scale.
3. Compare to one reference structure using relative scale.
Plot results
From “Establishing Software Size Using Pair-wise Comparison Method” by Eduardo Miranda
D I S T R I B U T I VE M A N A G E M E NT
![Page 13: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/13.jpg)
13EXAMPLE #1
SOFTWARE ESTIMATION
D I S T R I B U T I VE M A N A G E M E NT
![Page 14: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/14.jpg)
14EXAMPLE #1
OBSERVATIONS
Miranda’s Observations
The high variability of the “finger in the wind approach”, which is almost two to three times bigger than the corresponding paired comparisons method.
The high correlation, r = .979, existing between the relative sizes of modules independent of the estimation method employed. This seems to corroborate the premise that the human mind is better at establishing differences than at guessing absolute values.
D I S T R I B U T I VE M A N A G E M E NT
![Page 15: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/15.jpg)
15EXAMPLE #2
SYSTEM REQUIREMENTS
From an SEI study “Requirements Prioritization Case Study Using AHP” by Nancy Mead.
Given nine security requirements, decide which one(s) have greatest cost-benefit.
Benefits
“By using AHP, the requirements engineer can also confirm the consistency of the result. AHP can prevent subjective judgment errors and increase the
likelihood that the results are reliable.”
D I S T R I B U T I VE M A N A G E M E NT
![Page 16: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/16.jpg)
16EXAMPLE #2
APPROACH
Approach1. Review requirements for completeness.
2. Apply pair-wise comparison for value.
3. Apply pair-wise comparison for implementation cost.
4. Calculate AHP matrix and diagram for value & cost.
5. Use resulting diagram for analyzing requirements.
D I S T R I B U T I VE M A N A G E M E NT
![Page 17: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/17.jpg)
17EXAMPLE #2
COMPARISON SCALE
D I S T R I B U T I VE M A N A G E M E NT
![Page 18: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/18.jpg)
18EXAMPLE #2
COMPARISON
Compare one attribute of each requirement to another requirement using a relative scale.
#1 is same as #1
#1 is greater than #2
#2 is much less than #8
D I S T R I B U T I VE M A N A G E M E NT
![Page 19: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/19.jpg)
19ABOUT
CONSISTENCY
If: A > B > C
Then: C > A is wrong
AHP contains a technique to calculate the extent of pair-wise consistency, which can then be
compared to a consistency tolerance.
AHP can also indicate which pair-wise comparison (like the one above) is inconsistent.
D I S T R I B U T I VE M A N A G E M E NT
![Page 20: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/20.jpg)
20EXAMPLE #2
COMPARISON SUMMARY
D I S T R I B U T I VE M A N A G E M E NT
![Page 21: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/21.jpg)
21EXAMPLE #2
FINDINGS
Client feedback
It may be beneficial to see the consistency matrix.
Understand weight of cost and value.
Difficult to understand the motivation of each reviewer.
D I S T R I B U T I VE M A N A G E M E NT
![Page 22: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/22.jpg)
22 OTHER EXAMPLES
From Saaty:
Evaluate the best city in China for Disney to build a new theme park.
Determine optimum foreign relations policy for dealing with Iran.
Estimate market share of “super” retail stores.
Selecting a school. D I S T R I B U T I VE M A N A G E M E NT
![Page 23: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/23.jpg)
MEASUREMENT AND AHP
![Page 24: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/24.jpg)
24MEASUREMENT
AND AHPGoal is for Measurement to support AHP estimation
Estimate
Plan
Monitor & Control
Capitalize
Measurement Process
Use AHP
D I S T R I B U T I VE M A N A G E M E NT
![Page 25: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/25.jpg)
25
TYPICAL MEASUREMENT
PROCESS
Evaluate
EstablishCapability
Technical and Management
ProcessesINFORMATION NEEDS
ANALYSIS RESULTS
ANALYSIS RESULTS AND
PERFORMANCEMEASURESIMPROVEMENT
ACTIONS
Measurement Process
USER FEEDBACK
Plan Perform
Core Measurement Process
ExperienceBase
MEAS-UREMENTPLAN
![Page 26: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/26.jpg)
26 WHAT TO MEASURE?
Measurement “Requirements” = information needs Based on your business needs.
(not a pre-defined list) Information needs are:
“Input” to the measurement process.
Provided by the management and technical process that need information to perform their jobs.
Become the requirements for measurement process.
Refined into measures and then resulting information products are provided to the “users”.
D I S T R I B U T I VE M A N A G E M E NT
![Page 27: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/27.jpg)
27COMMON PROJECT
MEASURES
Cost
Staffing , Staff Hours
Functional / Requirements Size
SLOC
Defects
Estimation Project/Program Store
Factors estimated using AHP
D I S T R I B U T I VE M A N A G E M E NT
Usually there is opportunity for measurement to support AHP since there is overlap between what is estimated and what is measured
![Page 28: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/28.jpg)
28MEASUREMENT
GUIDE AHP
Measurement contains project attributes:
Lifecycle model
Principle architecture
Application domain
AHP estimation of projects with similar attributes.
Reference sizes should not differ by more than an order of magnitude.
D I S T R I B U T I VE M A N A G E M E NT
![Page 29: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/29.jpg)
29 AHP IN USE
Estimate total staff hours in person years using data from five completed projects sharing similar attributes.
D I S T R I B U T I VE M A N A G E M E NT
![Page 30: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/30.jpg)
30 RESOURCES
“Requirements Prioritization Case Study Using AHP” by Nancy Mead Software Engineering Institute
“Establishing Software Size Using the Paired Comparisons Method” by Eduardo Miranda
“Relative Measurement and Its Generalization in Decision Making Why Pairwise Comparisons are Central in Mathematics for the Measurement of Intangible Factors The Analytic Hierarchy/Network Process” by Thomas Saaty
D I S T R I B U T I VE M A N A G E M E NT
![Page 31: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/31.jpg)
31 QUESTIONS
D I S T R I B U T I VE M A N A G E M E NT
![Page 32: Peter Baxter, DISTRIBUTIVE MANAGEMENT Hampton Roads INCOSE Decision Analysis Conference - Nov 2009 U SING M EASUREMENT AND THE S AATY M ETHOD TO C HOOSE](https://reader035.vdocuments.site/reader035/viewer/2022070307/551aa1db5503466b3a8b5768/html5/thumbnails/32.jpg)
32
Peter Baxter
Distributive Management
www.distributive.com
D I S T R I B U T I VE M A N A G E M E NT