that is not the analytic hierarchy process: what the ahp is and what it is not

which is the case here. To illustrate, consider thematrix in (8) of the paper. If the criteria weightsunder A1 (resp. A2, A3) are correct, then theconventional AHP criteria weights are (48/117,33/117, 36/117) (resp. (72/235, 55/235, 108/235),(16/333, 308/333, 9/333)). Note that these threesets of weights have different ordinal orderings.The extreme inconsistency of the criteria weights isthe cause of the rank reversal, not the supermatrixmethodology.

EASE OF APPLICATION

If the ratio scale properties of the AHP are notneeded in a decision, an evaluation of the linkingpin AHP versus MAVT (including the method ofS&H) should be based on ease and accuracy of useby real decision makers. We leave it to the readerto draw a conclusion with respect to the relativeease of response and accuracy from the followingtwo typical questions:

(a) `How many times more attractive is a 2 week

vacation than a 1 week vacation?' (AHPquestion);

(b) `How many times greater is the attractivenessimprovement from a 1 day vacation to a2 week vacation than from a 1 day vacationto a 1 week vacation?' (MAVT question).

REFERENCES

Schenkerman, S., Àvoiding rank reversal in AHPdecision-support models', Eur. J. Oper. Res., 74,407±419 (1994).

Schoner, B., Choo, E. U. and Wedley, W. C., Àcomment on `Rank disagreement: a comparison ofmulti-criteria methodologies', J. Multi-Crit. Decis.Anal., 6, 197±200 (1997).

Schoner, B. and Wedley, W. C., Àmbiguous criteriaweights in AHP: consequences and solutions', Decis.Sci., 20, 462±475 (1989).

Schoner, B., Wedley, W. C. and Choo, E. U., À unifiedapproach to AHP with linking pins', Eur. J. Oper.Res., 64, 384±392 (1993).

Discussion

That Is Not theAnalytic Hierarchy Process:What theAHP Is andWhat It Is Not

THOMASL. SAATYUniversity of Pittsburgh, Pittsburgh, PA, U.S.A.

There are two basic principles that are totallymissed by Salo and HaÈ maÈ laÈ inen [R6] in theirattempt to reformulate the AHP, which is basedon ratio scales, to fit multiattribute value theory,which is based on interval scales. A theorem due toVargas [R8] that is discussed later says thatinterval scale measurement cannot be convertedto ratio scale measurement without knowing whatamounts to the entire ratio scale in the first place.We also give a counter-example which shows thatthis approach gives the wrong numerical outcome.Thus in my response I must first remind the

reader what these basic principles are and whytheir absence has caused these authors to stray farfrom the mark of making useful suggestions. Thefollowing two anecdotes tell it all and hint at my

general response. Their relevance will be clear inthe ensuing discussion.

. A man, upon meeting another in the street, saysto him `Hey, George, what have you done toyour hair, and look at that, you are alsowearing glasses. What has happened to youwith all these changes?'. The man replies Ì amnot George!'. What, and you have changedyour name too?'

. Tehran radio broadcast that a certainArmenian by the name of Humbersunian hadwon 1000 toumans in a state lottery. Theswitchboard was flooded with calls correctingthe broadcast. They said Ìt was not Mr.Humbersunian in the first place, but Mr.

324 DISCUSSION

J. Multi-Crit. Decis. Anal. 6: 320±339 (1997) & 1997 John Wiley & Sons, Ltd.

DISCUSSION 325

Hartunian. It was not a 1000 toumans, but onlya 100 toumans. It was not a state lottery, but afriendly poker game. And lastly he did not win.He lost'.

The authors spend the first part of their papersuggesting that they do not like the AHP as it isand propose radically changing it to an approachthat uses ratios of differences without a singleexample for the reader to see the problems withthat idea, and then spend the remainder of theirpaper proposing doctored-up modifications ofexisting AHP practice. They want to be countedin both ways. They are on the side of not A, butjust in case, and to be safe, they are also on the sideof A. Apparently they are not sure about thevalidity of their first proposal. Nor am I.What is missing from these authors' thinking is

a clear understanding of the fundamental differ-ence between measurement in the form of ratioscales derived mathematically from paired compar-isons, and measurement obtained by operating onnumbers assigned one at a time, their differencestaken and their ratios estimated. One has to havenumbers in mind to form differences and comparethem. Assigned numbers may be arrived atthrough a guess, a judgement, or by dividing aninterval of unit length into parts of appropriatelengths and parcelling these ordinal parts to eachof the elements being measured, or even by a wellworked-over value function. In this regard theysay in their conclusion section `Starting from thefoundations of multiattribute value measurement,we have demonstrated that pairwise comparisonsin ratio estimation should be interpreted in termsof value differences between pairs of underlyingalternatives'. Their proposal requires knowing thenumbers, e.g. for temperature because it ismeasured on an interval scale. Then one takesthe difference of two such readings and divides itby another difference. The object is to estimate thisratio when one does not know the underlyinginterval scale. But the difference in temperaturesand the strength of how we feel about thisdifference are not easily correlated in practice. Inaddition, the choice of minimum may not coincidewith the natural origin of the interval scale and onecannot make the transformation and obtain thecorrect outcome. Incidentally, these authors ask usto take the difference of non-existing measure-ments of intangibles and estimate their ratios!The AHP is a new paradigm that some people

find hard to accept on first exposure, particularly

those who have committed themselves to assigningnumbers directly to elements. But paired compar-isons, previously looked down on by many of us(including me, brought up in the tradition of puremathematics), owing to their great simplicity andto our lack of knowledge about how to use them tocreate valid measurement, are essential. Let meexplain what I understand by paired comparisons.The vernacular interpretation is that relativemeasurement is simply taking numbers from anabsolute or a ratio scale (the only kind of numbersone can add) and dividing each by their sum. Butthat is far from how relative measurementpriorities are derived in the AHP. One firstmakes the paired comparisons and then derivesthe priorities from them. Paired comparisons arethe engine for generating relative measurement. Inthe AHP one creates a hierarchy or network torepresent a decision and establishes a matrixcontaining the pairwise comparison judgementsfor the elements linked under a parent element.One then derives a priority vector of relativeweights for these elements, the principal eigen-vector of the matrix. An inconsistency measure isderived as a by-product of this process. There isone such matrix for every parent element. All thepriority vectors are appropriately weighted andsummed to obtain the overall priorities for thealternatives of a decision.

MEASUREMENT IS CREATED FROMPAIRED COMPARISONS

Paired comparisons are the intermediate fine-structured process with which one uses one's bestknowledge and understanding to determine howmany times the dominant of two elements, onsome property, is a multiple of the less dominantone taken as the unit of measurement. Thus onemust answer the question `How many times moreis the second element than the unit element?'. Theunit then has the reciprocal relation to the secondelement. The unit, as it is the smaller element inwhatever pair is being compared, is different inevery pairwise comparison. The number one usesto express `how many' belongs to the fundamentalscale of absolute numbers 1±9 of the AHP. Laterwe explain why the range of the scale must belimited. This scale is unit-free, for if the elementscan be measured in pounds or in kilograms, forexample, the second element is the same multipleof the unit element in both scales. An absolute

& 1997 John Wiley & Sons, Ltd. J. Multi-Crit. Decis. Anal. 6: 320±339 (1997)

326 DISCUSSION

scale is invariant under the identity transforma-tion, which means that no other number would doto express the comparison judgement.`The proof of the pudding is in the eating', the

saying goes, and it equally applies to the validity ofAHP applications. Paired comparisons make itpossible to ask scientific questions that aremeasurement-oriented. Making paired compari-sons does not mean assigning numbers from aninterval to each of a set of elements and thenforming their pairwise ratios. It is from all thecomparisons that one must first derive a set ofrelative weights or priorities that belong to thesame ratio scale for all the elements in that group.To say how many times more one element is thananother, taken as the unit for that comparison,when the elements are close entails some kind ofanalytical thought directed precisely to the ques-tion. It is not the same as simply calling for adifference of two numbers out of one's head andassigning it to one element, and then calling foranother such number sensing the magnitude oftheir ratio and estimating its value. It never ceasesto amaze me how people have numbers waiting inthe wings of their heads to be assigned asmeasurements or probabilities used to makedecisions. Only where an individual has longfamiliarity with physical objects that havemeasurement on some existing scale can thatindividual accurately assign numbers directlyfrom that scale. Usually only the grocery clerkcan tell when a piece of cheese is close to onepound of weight.Besides, our goal here is to extend the concept of

deriving measurement from paired comparisons tocover intangibles for which there are no scales.Paired comparisons relate one element to anotheraccording to one's perception and experience.Paired comparisons should enable us to makefairly accurate estimates and predictions ofhappenings in the real world. Even though thepreponderance of humanity has not gone to schoolto learn how to perform measurement accordingto our sophisticated theories or even deal withnumbers in some depth, yet many make pairedcomparisons with alacrity and confidence andselect the best of several options. They possesssome built-in ability that we suspect is someversion of paired comparisons using their feelingsand perceptions.In passing, we observe that theories of measure-

ment before the AHP, not having a convincingway to create practical and meaningful cardinal

measurement that ties well with measurement inthe physical sciences, attempted to move slowlyfrom ordinal scale ranking to something slightlystronger. Their approach has yielded weaker scalesthan one obtains by starting from the other end,with an absolute scale from which the very strongratio scales are derived. An advantage here is thatratio scales make it possible to incorporate greaterprecision to capture people's understanding andmake it possible to deal with decisions involvingdependence and feedback because they can bemultiplied and added.

HOMOGENEITY AND CLUSTERING

The other missing point in Salo and HaÈ maÈ laÈ inen'saccount is the absence of a recognition that peoplecan only provide judgements about things that arefairly close or homogeneous. Using the scale 1±9means that one can only compare things that arewithin an order of magnitude from each other.Thus the homogeneity requirement of the AHP.Otherwise, if they are dealing with a widelydisparate set of elements on some property, peopleare forced to guess wildly about how much toassign to the small and how much to assign to thelarge. This problem does not disappear by usingzero and one to represent the extreme contrastbecause assignment would have to be made to thenext set of widely contrasting objects calling for adiversity of estimations and guesses. I know of nomathematical or technical problem associated withputting a bound on the values of a fundamentalscale to derive relative measurement from it.If one makes paired comparisons of two

alternatives, one determines which is the smallerand estimates how many times (multiples of it) thelarger one is. If they are too far apart to compare,e.g. a grape with a watermelon according to size,one would make such a bad mistake in judginghow many grapes are equal to a watermelon that itis not worthwhile to make the comparison directly.How does the AHP measure widely separatedelements?One first compares the grape with a plum and

also with a small apple in one set, and then againcompares in a different set the small apple with alarge apple and a grapefruit, and for a third timeseparately compares the grapefruit with a melonand the watermelon. One then has a comparison ofthe grape with the watermelon. In each set ofcomparisons one uses the 1±9 scale. In the second


DISCUSSION 327

comparison set one divides by the weight of thesmall apple so it gets a `one' and multiplies by itsweight in the first comparison so it has the sameweight relative to the grape. The weights of thelarge apple and the grapefruit are also multipliedby this weight of the small apple from the firstcomparison. Again if one divides the weights in thethird set of comparisons by the weight of thegrapefruit and multiplies them by its weight fromthe second comparisons (adjusted as explainedabove), one gets the relation of the watermelon tothe grape, by simply dividing to see how manygrapes make a watermelon. One should not use thescales 1±3 or 1±5 etc., because, as we have seen,one would do much more work. In addition,people are observed to have the ability to comparehomogeneous objects that are within the sameorder of magnitude spread out from 1 to 9. Ibelieve that this order of magnitude was notchosen arbitrarily but came from observing whatpeople are able to do. I and numerous other peoplehave applied the 1±9 scale to compare manyphysical phenomena (lengths, weights, distances,brightness, etc.) and got excellent results. Inaddition, I have carried out a number of experi-ments to use numbers from other scales instead ofthe 1±9 scale and invariably failed to get goodresults. Here one has to use a verbal scale toexpress the judgements which are then replaced bythe appropriate numbers.If one is uncertain about the homogeneity of the

elements being compared, one performs thecomparisons on a few elements at a time using itas a filtering process and observes the widedeviations in the resulting vector, and then rear-ranges them accordingly into homogeneous groups.In theory there should be no problem in doing this.In practice the total set of elements may have to beincreased (possibly with hypothetical members as inthe case of the grape and the watermelon) to make asmooth transition from one set to another. Notethat the range of human interests spans a very smallnumber of orders of magnitude and this filteringprocess would not be as cumbersome as it could inphysics and astronomy.

COMBINING TANGIBLES WITH THE SAMEUNIT AND WITH OTHER TANGIBLES AND

INTANGIBLES

So far we have talked about measurement withrespect to a single attribute or criterion. In making

decisions, we deal with many criteria and mustcombine their different measurements. How? TheAHP is concerned with the measurement ofmyriads of intangibles (things for which nomeasurement scales exist) and how to bring thatmeasurement together with the measurement ofthe few tangibles there are.In their paper, Salo and HaÈ maÈ laÈ inen give an

example of the need to combine tangibles that aremeasured on the same scale proportionately byweighting according to the quantity eachpossesses. This is well known in the literature ofthe AHP, and the Expert Choice software has acommand, called Transformation, to do it. Ourpurpose in any case is more often to relate tangibleand intangible criteria, criteria that do not havethe same scale, or finally, criteria that have thesame scale but in spite of that whose measure-ments on this scale have different impacts on ourvalue system in different settings.We will show with an example how tangibles are

combined, first with like tangibles, then with othertangibles and with intangibles in the AHP. Assumethat a family is considering buying a house andthere are three houses to consider. Four factorsdominate their thinking: the price of the house, theremodelling costs, the size of the house as reflectedby its footage, and the style of the house which isan intangible. They have looked at three houseswith data shown in Table I on the quantifiables.Thus, in terms of dollars, price and remodelling

costs have weights of 1000/1300 and 300/1300respectively (Table II). Of course, the importanceof these two factors can also depend on themanner in which the payments are made, howmuch money is available at a certain time and soon, and one may have to combine these tangibleweights with other intangibles derived throughpaired comparisons. When one uses measurementfrom an existing scale, the implicit assumption isthat a unit has the same value at all levels ofmeasurement. Most people doubt this use ofcounting numbers in measurement. Thus one


Table I. Choosing the best house

Price

($1000)

Remodelling

costs ($300)

Size

(ft2) Style

A 200 150 3000 Colonial

B 300 50 2000 Ranch

C 500 100 5500 Split level

328 DISCUSSION

needs to establish priorities for different ranges ofmeasurements taken from existing scales, whichmust be done in the particular context of thedecision problem and its objectives. Here we havetried to be faithful to the arithmetic in combiningeach set of measurements as if numbers are thesame at any level. A better way would be tointerpret the significance of number ranges byusing paired comparison judgements. Numbersprovide information for analysis, they do not inthemselves contain answers to problems. We arethe ones who decide what a number means to us.Then we must combine the factors measured

with respect to the same scale, in this case price(Table III).To make relative and absolute measurement

compatible, we must first combine any factorsmeasured on the same existing scale into a singlefactor under which the alternatives are measured.We do this by multiplying and adding and not byraising to powers and multiplying as some peoplewould like to have us do. We can then proceed tomake paired comparisons among the differentfactors and among the alternatives.We see here that we also need ratio scales in a

multicriteria setting because we must trade off oneunit of one ratio scale against another unit ofanother ratio scale, and we can do this by makingpaired comparisons. We measure the relativeimportance of the criteria, whether tangible orintangible, which these units represent, on equalfooting, and use the resulting priorities to weight

the units. We can, for example, compare theintangible criterion style with the tangible criterionsize to determine which is more important to serveour higher objective of comfort. It is a truism tosay that people do this all the time and verysuccessfully. Since the total units (encounters andknowledge indicating the degree of significance tocomfort) available for each criterion are different,one compares the average or equivalently the totalof the units available for one against the averageor total for the other [R5].

THE AHP IS DESCRIPTIVE ANDPREDICTIVE, NOT NORMATIVE

At its foundation the AHP is intended to be adescriptive, not a normative, theory. Sometimes itis used to respond to the question of what ispreferred and this is usually done at the level of thealternatives. More often it is used to answerquestions about what is more important at thelevel of the criteria and what is more likely at thelevel of the alternatives, as in the presidentialelections. Its present thrust is to tell how things arerather than to prescribe how they should be, andto determine best choices in the face of current andprojected conditions. The effectiveness of thisorientation has been borne out through successfulprediction in a large number of applications. Agood descriptive theory should be able to say whatthe current situation is now and how it will be inthe future. In decision theory, when adequatestructure is provided to analyse the decision, itshould be possible to predict the best decision tosurvive the conditions of the present and thefutureÐan optimal decision. The question beforeus is: How can a theory claim scientific (not justmathematical) validity if all it does is assignnumbers to objects subject to theoretical assump-tions that appear to be eminently reasonable,unless at the same time it has some predictivecontent that can be tested in practice? The MITphysicist Slater once said `Questions about atheory which do not affect its ability to predictexperimental results correctly seem to me quibblesabout words'.Here are some references to examples of

successful predictions made with the AHP [P12]and ANP [R4, P2, P9]. Many applications havebeen made of the young and robust generalizationof the AHP to dependence and feedback, knownas the analytic network process (ANP). These

Table II. Choosing the best house

Price

1000/1300

Remodelling

costs

300/1300 Size Style

A 200/1000 150/300 3000/10500 Colonial

B 300/1000 50/300 2000/10500 Ranch

C 500/1000 100/300 5500/10500 Split level

Table III. Choosing the best house

Economic factors

(combining price and

remodelling costs) Size Style

A 350/1300 3000/10500 Colonial

B 350/1300 2000/10500 Ranch

C 600/1300 5500/10500 Split level


DISCUSSION 329

applications include rise and fall of stocks in thestock market [P8], political candidacy [P5], oilprices [P3, P4, P6], energy rationing [P7], inter-national chess competition (predicting the winnerand by how many games) [P10, P11], predictingfamily size in rural India [P13], predicting foreignexchange rates for the dollar versus the yen [P1],turnaround of the U.S. economy and the strengthof its recovery [P2] and the outcomes of the play-off games and Superbowl in the 1995±1996 foot-ball season [P9].

GROUP DECISION MAKING

Prior to the development of the AHP it wasconsidered that to develop a theory to aggregateindividuals' cardinal preferences was `chasing whatcannot be caught' [R1]. It is precisely the pairedcomparison process with its reciprocal propertythat makes this possible. In addition, we have therecent proof that Arrow's so-called impossibilitytheorem is possible with cardinal numbers [R7].Another problem with the difference methodproposed by Salo and HaÈ maÈ laÈ inen is how togenerate and aggregate judgements of a groupcorrectly.

VALUE FUNCTIONS

The approach suggested by Salo and HaÈ maÈ laÈ inenand what is done in the AHP reduce to thefollowing. In the AHP, if A � 4C and B � 2C,then A=B � 4=2 � 2 while Aÿ B � 2C. The latterdepends on C whereas the former is independentof C, and what are we going to do about it? Takingdifferences is not unit-free and resorting tocomplex arguments to make it so creates the newproblem of how to get a measurement on C in thefirst place. Again we see that the AHP is a ratioscale theory and not a `difference in magnitude'theory. The object is not to collect large or smallvalues but to have ratios derived from absolutecomparisons that provide the necessary contrastbetween elements of different sizes.Salo and HaÈ maÈ laÈ inen observe that one should

not use ratios directly in making comparisons butshould use value functions constructed laboriouslyfrom interval scale readings. Value functionsshould apply to recover measurements on a ratioscale, as Salo and HaÈ maÈ laÈ inen propose. If, tomake a decision, human beings had to make their

mental ratios based on such a theory, there wouldbe no civilization today. There has to be an easierway accessible to every person.According to Roberts [R2], suppose that K is a

set of consequences and L is a collection oflotteries with consequences in K. Any functionu : K! Re will be called a value function on K andit will be convenient to distinguish value functionson K from utility functions, which are valuefunctions on K with certain special properties,e.g. the property of preserving certain observedrelations on K.The definition of a value function does not say

how to construct such a function by assigningnumerical values to the elements being measured.According to how the assignment is made and theassumptions it satisfies, the resulting mapping tothe reals is a given type of scale. For example, ifthe admissible transformations of this assignmentare monotone, the resulting scale is ordinal. On theother hand, if the assignment is invariant underlinear transformations, then it is an interval scale.If the elements are assigned likelihoods of occur-rences (one way to do this is through lotteries),then the result leads to an interval scale under theexpected value hypothesis. If the assignment ismade through direct assessment by dividing aninterval from 0 to 100 heuristically, in pieces eachassigned to one alternative, then the result is anordinal scale.The distance from Philadelphia example was

intended to show that when numbers are available,they can be compared as ratios by applyingjudgement to their magnitude, a special case ofpaired comparisons and not the other way aroundas the authors seem to think. If one comparesestimates of differences between the alternatives,and assuming that these comparisons satisfy theaxioms of difference measurement, then the resultis an interval scale. The authors assume that thevalue function is an interval scale and take ratiosof the differences of two readings and claim thatone gets a meaningful ratio scale. The axioms forsuch a theory must assume transitivity and weakorder. Even the first is not required by the AHP.Nor does transitivity hold in most human activ-ities.Here is a fundamental observation. The authors

assume that one can go back and forth betweeninterval and ratio scales. It is true that one can gofrom a ratio scale to an interval scale because ratioscales are a special case of interval scales. Sincethey are a special case, there must be an additional


330 DISCUSSION

requirement to satisfy to convert an interval scaleto a ratio scale. That requirement has been shownby Vargas [R8] to be the presence of one`anchoring' paired comparison that must comefrom the ratio scale one is seeking. Such a pairedcomparison requires that one define the meaningof comparing two interval scale values. Once thatis done, one is able to convert all the other intervalreadings.Subtracting interval scales is not the answer.

Here is a counter-example. Assume that we havethree objects whose weights on a ratio scale are 6, 3and 1 units. The ratio of the first two is 6=3 � 2.Now let us convert them to an interval scale byusing the transformation proposed by the authors,�xÿ x��=�x*ÿ x��, where x� is the smallest valueand x* is the largest. The corresponding intervalscale values are given by 1, 2/5, 0. It follows thatthe ratio of the first two scale values is given by1=�2=5� � 2:5 and not 2. The only way one obtainsthe ratio 6/3 is if one assumes that the smallestvalue is zero, which implies that one alreadyknows the scale with which one is dealing and inthis case it is a ratio scale.They ask the question `how many times greater

is the quality improvement' in making compar-isons by relating a good-quality and a bad-qualitycar. It is obvious that the answer to how manytimes is not as they say, but requires that onealternative be given as a multiple of the other (anabsolute number as the quotient of two ratio scalenumbers). It makes no sense to ask such a questionby taking differences because the two numbersmust be known individually to form their differ-ence. This difference is not an absolute number butis scale-dependent because the numbers themselvesbelong to a particular scale. As we have seen in theexample above, one does not get meaningfulnumbers with the proposed approach, and evenif one did, their difference is taken automaticallyand does not need asking the question `how manytimes greater'. In the AHP one does so directly bycomparing the two cars using the smaller as theunit without intervening a third car as a reference.It is a bizarre and haphazard procedure to get a

number for an alternative get another number fora second alternative, get a third number for a veryunfavourable alternative, subtract it from each ofthe others and then form the ratio of the resultingtwo differences. Does this process lead to anythingmeaningful? The authors refrain from showing ushow to use differences in an intelligent way in areal-life decision with intangibles and prove

nothing about getting a meaningful result thathas something to do with known weights.Nowhere do they mention that the eigenvalueapproach is the procedure through which onesolves equations to obtain the priorities when theyare not known at the start.The authors cite other people versed in utility

theory about what constitutes acceptable proce-dure for preference elicitation. This reminds one ofall the predictions made by pundits of the pastabout man's ability to fly, split the atom, givewomen the right to vote and so on. What isapparent in all this is that they are attempting toconvolute a new idea to fit an old one. There is alarge community of scientific people who use theAHP style of preference elicitation and find itacceptable and effective. We need to focus on themanner of making paired comparisons and deriv-ing relative measurement rather than on who sayswhat from their own old point of view. And,I repeat, the authors provide no practical examplewhatsoever to illustrate the cumbersome proce-dure they propose for replacing pairedcomparisons.

OH, WHY CAN'T RATIOS BE LIKEDIFFERENCES?

The authors replace the judgement values 1 by 2and 8 by 9 on two-element comparisons andobserve that the ratio of the difference in theresulting priorities of the first component of�1=�1� r�, r=�1� r�� by changing r from 1 to 2and again from 8 to 9 is 15, to which they attachgreat significance. In fact, in paired comparisonsone cannot ignore what also happens to thepriorities of the second element. In that case thejudgements in the matrix go from 1 to 1

2and from 1

8to 1

9. If one carries through the same type of

analysis, one again gets 15. The difference of thetwo results is equal to zero, or if we take the ratioof this 15 to the other 15, we get one. Along theline of thinking of the authors, we conclude thatone need not be any more concerned that peoplemay change their judgement from 1 to 2 than from8 to 9. Because judgements are never made on oneelement at a time, adding a unit to a judgement inrelative measurement of homogeneous elementsseems to have the same effect at whatever level itis added. Again the lack of understanding ofpaired comparisons has led the authors on a wildgoose chase to next propose new scales they think


DISCUSSION 331

are better than absolute numbers to express thejudgement that A is so many times larger than B!

PROPOSING NEW SCALES TO REPLACE 1±9

Consider four line segments Si , i � 1, . . ., 4, oflengths L�Si� � i inches. When normalized, thesebecome the relative lengths 0.i. The pairedcomparison judgements from the fundamentalscale 1±9 in this case involve the values 1, 1

2, 13, 14

in the first row of the matrix and their reciprocalsin the first column. It is known that in theconsistent case the eigenvector is given by any ofthe normalized columns and hence the relativelengths are precisely given by 0.i, i � 1, . . ., 4, andonce more we see that the scale 1±9 gives back thedesired values. If instead we use the values 1, 1.22,1.50, 1.86 proposed by the authors in the firstcolumn and normalize them, we obtain for therelative lengths 0.179, 0.219, 0.269, 0.333 whosedeviation from 0.1, 0.2, 0.3, 0.4 on the AHP ratioscale metric is 1.093, which is near the upper limitof the guideline for acceptable compatibility. If, onthe other hand, we use their scale values for thelengths of the segments and normalize them, weget 0.18, 0.22, 0.26, 0.33. If we now take theirappropriate ratios and approximate them by thenearest integer from the scale 1±9, we get 0.17,0.17, 0.33, 0.33 whose deviation from that scale onthe ratio scale metric is 1.030, considerably underthe 1.1 limit for acceptable compatibility. Again,here is what the authors' proposed scale gives forareas of figures whose measurements are given inTable IV compared with the scale 1±9.

Yet a third example of the optics experiment inphysics about the relative brightness of four chairsplaced at distances of 9, 15, 21 and 28 ft from alight source yields the results in Table V.The words intended to capture three times more,

five times more and so on have been testednumerous times in practice and found in generalto give good results. There is no intrinsic numberbehind a word until people have used it sufficientlyoften to associate it with their mental experience.It is not difficult to argue that in makingcomparisons, the mind is much better trainedthrough experience to estimate integer multiples ofa given unit (the smaller of two elements) than toassign a number directly from a convenientlychosen scale. If we have several elements tocompare, we would find that the derived scalestill captures some of the lost accuracy that occursbecause integer values from the fundamental scalemust be used in verbal comparisons. One can showmathematically that reciprocal matrices are themost stable framework we have to derive ratioscales from homogeneous judgements. When theelements are close, more accurate results areobtained by comparing them with other elementsthan by invoking decimals, and in this case onemay need to expand the scale in the interval from 1to 2 to allow judgements 1.1, 1.2, . . . , 1.9. Oneshould not force an element to be at least twiceanother when in fact it is not.

DISCRETIZATION AND BOUNDS ONSCALE VALUES

The author's comments about upper and lowervalues leave out the consideration of homogeneity.It is impossible to construct a normalized examplethat results in all the values 0.1, 0.2, . . . , 0.9included, because the sum would exceed one.Normalization, or dividing each value by the sameconstant, simply yields relative values and does notdistort anything if one remembers that in any case,relative values in the eigenvector are not absolutebut ratio scale numbers. The line segmentsexample shows that the scale 1±9 does give backequally spaced answers for equally spaced stimuli.On reading the authors' account of the work byWedley and Schoner, I called Wedley on thetelephone to learn more about the experiment. Hetold me that he thinks the scale 1±9 works well anddoes not think it needs changing because ofanything he has done.

Table IV. Areas of figures

Circle Square Diamond RectangleTriangle

Actual area 0.470 0.230 0.130 0.097 0.050

Authors' scale 0.388 0.218 0.173 0.136 0.087

1±9 scale 0.484 0.236 0.123 0.108 0.049

Table V. Relative brightness of chairs

Chair 1 Chair 2 Chair 3 Chair 4

Actual scale value 0.608 0.219 0.111 0.065

Authors' scale 0.477 0.241 0.157 0.125

1±9 scale 0.620 0.220 0.100 0.060


332 DISCUSSION

The authors are troubled by the fact that puttinga bound on the scale decreases the largest valueany component of the eigenvector can have, as afunction of the number of components. This is abit of sophistry because they forget to take theeffect of normalization into account. The moreelements there are, the smaller is the value of themaximum in relative terms because they all have toadd to one. That does not mean that its ratio to theweights of the other elements may not be at itslargest possible value which is an order ofmagnitude but no more because the elements arehomogeneous even with this maximum. The otherelements would take on correspondingly smalldecimal values. Since none of these values can betoo small relative to the maximum, the moreelements there are, the smaller is the value of thelargest element. In addition, the larger the boundM, the smaller the relative values of the otherelements need to be to still satisfy homogeneity.Nothing is surprising about this outcome. It is nota problem with the upper and lower values of thebounds but needs to be considered together withhomogeneity. In comparing the area of a largecircle that is homogeneous with smaller circles, themore there are of the latter, the smaller is theproportion of the largest circle to the total, as itwould be. But the larger the circle, the greater is itsnormalized share of the total, yet it is still possibleto form ratios with areas of the other circles.Similarly, the minimum value is correspondinglysmaller. This part of the paper, along with thecomments about criteria, makes no sense inrelative measurement and ratios. Along withseveral mathematical colleagues who read thispaper, I remain open to explanations by theauthors as to what they really have in mind, andwhat relevance it has to saying that one object isfive times heavier than another, or a figure hasthree times the area of another figure, and on tocriteria and other abstractions in decision making.

RANK PRESERVATION AND REVERSAL[K1±19]; PEOPLE WITH BIG PROBLEMS SEEMOTE IN BROTHER'S EYE, NOT BEAM IN

OWN EYE

We discuss three ideas in this part: (1) that rankreversal is legitimate and should be allowed for ina mathematically precise way; (2) Salo andHaÈ maÈ laÈ inen's example of the supermatrix is thewrong one for rank preservation; (3) the super-

matrix allows for both rank preservation andreversal just as the AHP itself does.There are two kinds of rank reversal discussed in

the literature. The first has to do with lotterycomparisons in risky situations where people'spreferences for lotteries change owing to theirassessment of the risk involved, leading to achange in the outcome giving rise to rank reversal.This is what is known as the violation of theinvariance principle, a major concern in utilitytheory discussed briefly later.The other kind that is directly related to the

AHP is rank reversal that may be due to theaddition or deletion of alternatives. There are nowmyriads of real-life examples which show thatthere are cases where rank does and should beallowed to reverse and others which show thatrank should not be allowed to reverse. Cases whererank reverses may be due to copies and nearcopies, phantom alternatives, decoy alternativesand others identified by researchers, most ofwhom are practitioners of utility theory. Becauserank reversals occur in practice, a theory cannotadopt one position with regard to all thesesituations by laying down a principle to alwayspreserve rank or always to allow rank to reverse.Let us note that there are decision problems thathave identical mathematical structure and judge-ments; in one of these problems rank needs to bepreserved whereas in the other it must be allowedto reverse.A decision theory needs two methods of

synthesis to deal with these two possibilities. Inthe AHP, in both its relative and absolutemeasurement approaches, there are two suchmodes of synthesis. In relative measurement thefirst mode preserves rank from irrelevant alter-natives by comparing all alternatives with an ideal;the second mode, called the distributive mode,allows one to choose the best alternative in the setby comparing them with each other withoutreference to an ideal. In absolute measurement,the normative approach, that is comparable withutility theory, where alternatives are rated one at atime, the ideal mode preserves rank not only withrespect to irrelevant alternatives but also withrespect to any alternatives, whether relevant ornot, and the distributive mode allows it to reverse.Those who insist on always preserving rank

have committed themselves to a theory that cangive wrong answers to problems raised by theirown colleagues that cannot be resolved byphilosophical arguments but need a revised


DISCUSSION 333

mathematical approach. Some have argued that byrevising the set-up of the problem they can alwaysdeal with a decision problem in the context of rankpreservation. It is easy to see that that isimpossible because even in the simplest casewhere copies can cause rank reversal it may notbe the first copy but the 1555th copy and how isone to know that in advance without a carefulmeasurement procedure that tells one whenaccording to one's preference saturation has beenreached.There never was a mathematical theorem that

persuasively showed that rank had to be preservedunder all conditions. It was simply an assumptionthat some people thought was a reasonable oneand was supported by ambiguous speculations.The reader who desires quick exposure to theproblem should read the tortuous versions andcounter-examples provided by Luce and Raiffa intheir book [K9]. I became a sceptic when I usedutility theory in the cause of disarmament andfound these contradictory statements. Rank rever-sal is as essential to creative decision making asallowing inconsistency is to change our assessmentof the relative importance of alternatives withrespect to a single criterion. When we are inclinedto value the presence of a high level of a propertyin an alternative owing to its scarcity, we changeour minds when we find that that property is infact abundant in many alternatives. Such rankreversal does not only occur when assessing theaddition of real alternatives, but also of phantomor decoy alternatives because we assume that theycan be there to affect our thinking in choosing thebest alternative. All these are manifestations ofchange in the relative measurement of alternativesin which normalization plays an intrinsic role.Normalization takes account of the number ofalternatives (their scarcity, abundance and unique-ness) and the strength of measurement of eachalternative (its proximity and similarity to and itscontrast with other alternatives). Another impor-tant factor is whether one ranks alternativesrelative to ideal standards or whether one wantsto choose the best one in a set of given alternatives.Usually one preserves rank in the former andallows it to reverse in the latter.Those who have not dealt with relative measure-

ment have often hastened to suggest eliminatingnormalization to make things look like ratio scalenumbers used in actual measurement such asyards, pounds and dollars. Rank reversals mustbe accounted for by the theory itself and not by

watching the process of decision making with analert eye! The matter would have to be dealt withby either a clear rule or an algorithm, notaccording to one's capricious tastes. There is anabsence of awareness that mathematics is neededas in the AHP to determine the point at whichreversals can and should occur because it may notbe the first or second added alternative but themillionth one that can legitimately cause rank toreverse.The dogma about preserving rank when the

alternatives depend on each other has always beenthat anything can happen. Thus, when the criteriadepend on the alternatives, which implies thatalternatives depend on the alternatives, rank maybe allowed to reverse. However, one can have aproblem with dependence among the criteria, butwith no dependence of criteria on alternatives andrank may need to be preserved. It is clear that Saloand HaÈ maÈ laÈ inen are not familiar with the AHPliterature that says that if the alternatives areindependent of each other and the criteria do notdepend on the alternatives, then the alternativesare evaluated with the ideal mode outside thesupermatrix and their weights appropriatelysynthesized by using the limiting weights of thecriteria from the supermatrix. This procedure isdescribed on p. 249 of my 1994 book onfundamentals of the AHP [R3]. Their supermatrixexample does not prove anything because in it thecriteria and the alternatives depend on each otherand this implies that the alternatives depend oneach other. The supermatrix approach has beenapplied extensively to make predictions about theturnaround date of the U.S. economy andoutcomes of games and sports events with verysatisfactory results and hence there seems to bemore to it than fanciful number crunchingdesigned to impress people with its complexmanipulations. In fact there is an entire book[R4] written on the supermatrix theory withabundant applications and there seems to bewide demand for knowledge about dependence indecision making as I myself have experienced bothinside and outside academe.Now about the violation of the invariance

principle we mentioned earlier, Tversky et al.[K17] write `. . . a growing body of empiricalevidence questions the assumption of invariance,which is essential to the theory of rationalchoice . . . alternative framings of the sameoptions . . . give rise to reversal of preferences,and alternative elicitation procedures . . . give rise


334 DISCUSSION

to reversal of preferences. . . . Because invar-ianceÐunlike independence or even transitivityÐis normatively unassailable and descriptivelyincorrect, it does not seem possible to construct atheory of choice that is both normatively accep-table and descriptively adequate'. This says it all tothose who doubt that the phenomenon is real andcannot be brushed away with philosophical argu-ments.The AHP does not deal with lottery compar-

isons in risky situations but has a differentapproach based on direct paired comparisonsand feedback to deal with risky situations. Amajor work that has been completed and refereedwill appear in a well-known journal whichdemonstrates that the judgement process ofpaired comparisons is more general than estimat-ing probabilities as one does in lotteries and maynot lead to the difficulties of violating invariancewith appropriate posing of a decision problemunder risk.In their conclusion the authors say `This

interpretation is general and applies to all methods(including the AHP and SMART) which make useof ratio statements in the elicitation of hierarchicalmodels. In fact, when the questions in the AHP arerephrased according to the value difference inter-pretation, the AHP can be regarded as a variant ofmultiattribute value measurement. While it is stillunclear to what extent the DM's intuitiveresponses to the standard AHP questions conformto the value difference interpretation, we feel thatAHP practitioners could improve their analyses bystating the pairwise comparison questions accord-ingly'.I believe that I have shown the lack of relevance

of the author's proposal to the AHP. I have gravedoubts about statements made without rigorousarguments and mathematical proofs. My under-standing is that scientific research is based onevidence and hypothesis testing and not onspeculative statements and observations aboutexamples.

GENERAL REFERENCES

R1. MacKay, A. F., Arrow's Theorem: The Paradox ofSocial Choice: A Case Study in the Philosophy ofEconomics, New Haven, CT: Yale University Press,1980.

R2. Roberts, F. S., `Measurement theory with applica-tions to decision-making, utility, and the social

sciences', in Encyclopedia of Mathematics and ItsApplications, 1979.

R3. Saaty, T. L., Fundamentals of Decision Making withthe Analytic Hierarchy Process, Pittsburgh, PA:RWSA Publications, 1994.

R4. Saaty, T. L., The Analytic Network ProcessÐDecision Making with Dependence and Feedback,Pittsburgh, PA: RWS Publications, 1996.

R5. Saaty, T. L., Vargas, L. G. and Wendell, R. E.,Àssessing attribute weights by ratios', OmegaÐInt. J.Manag. Sci., 11 (1983).

R6. Salo, A. A. and HaÈ maÈ laÈ inen, R. P., Òn themeasurement of preferences in the analytic hierarchyprocess', J. Multi-crit. Decis. Anal., 6, 309±319 (1997).

R7. Srisoepardani, K. P., `The possibility theorem forgroup decision making: the analytic hierarchy pro-cess', Ph.D. Dissertation, University of Pittsburgh,1996.

R8. Vargas, L.G., Ùtility theory and reciprocal pairwisecomparisons: the eigenvector method', Socio-Econ.Plan. Sci., 20, 387±391 (1986).

PREDICTION REFERENCES

P1. Blair, A., Nachtmann, R., Olson, J. and Saaty, T.,`Forecasting foreign exchange rates: an expert judg-ment approach', Socio-Econ. Plan. Sci., 21, 363±369(1987).

P2. Blair, A., Nachtmann, R. and Saaty, T. L.,Ìncorporating expert judgment in economic forecasts:the case of the U.S. economy in 1992', in Saaty, T. L.(ed.), Fundamentals of Decision Making and PriorityTheory with the Analytic Hierarchy Process, Pitts-burgh, PA: RWS Publications, 1994.

P3. Gholam-Nezhad, H., `1995: the turning point in oilprices', in Didsbury Jr., H. F. (ed.), The GlobalEconomy: Today, Tomorrow, and the Transition,Washington, DC: World Future Society, 1985.

P4. Gholam-Nezhad, H., Òil price scenarios: 1989and 1995', Strat. Plan. Energy Manag., 7, 19±31(1987).

P5. Saaty, T. L. and Bennett, J. P., À theory ofanalytical hierarchies applied to political candidacy',Behav. Sci., 22, 237±245 (1977).

P6. Saaty, T. L. and Gholamnezhad, H., Òil prices:1985 and 1990', Energy Syst. Policy, 5, 303±318(1981).

P7. Saaty, T. L. and Mariano, R. S., `Rationing energyto industries: priorities and input±output depen-dence', Energy Syst. Policy, 8, 85±111 (1979).

P8. Saaty, T. L., Rogers, P. C. and Pell, R., `Portfolioselection through hierarchies', J. Portfolio Manag.,No. 3, pp. 16±21 (1980).

P9. Saaty, T. L. and Turner, D. S., `Prediction of the1996 Super Bowl: an application of the AHP with


feedback (the supermatrix approach)', Proc. 4th Int.Symp. on the Analytic Hierarchy Process, Vancou-ver, 1996, Pittsburgh, PA: Expert Choice Inc., 1996.

P10. Saaty, T. L. and Vargas, L. G., `Hierarchicalanalysis of behavior in competition: prediction inchess', Behav. Sci., 25, 180±191 (1980).

P11. Saaty, T. L. and Vargas, L. G., `Modelingbehavior in competition: the analytic hierarchyprocess', Appl. Math. Comput., 16, 49±92 (1987).

P12. Saaty, T. L. and Vargas, L. G., Prediction,Projection and Forecasting, Dordrecht: Kluwer, 1991.

P13. Saaty, T. L. andWong,M., `The average family sizein rural India', J. Math. Sociol., 9, 181±209 (1981).

RANK REVERSAL REFERENCES

K1. Buede, D. and Maxwell, D. T., `Rank disagree-ment: a comparison of multi-criteria methodologies',J. Multi-Crit. Decis. Anal., 4, 1±21 (1995).

K2. Bunge, M., Treatise on Basic Philosophy, Vol. 7 ofEpistemology and Methodology III: Philosophy ofScience and Technology Part II: Life Science, SocialScience and Technology, Boston, MA: Reidel, 1985.

K3. Corbin, R. and Marley, A. A. J., `Random utilitymodels with equality: an apparent, but not actual,generalization of random utility models', J. Math.Psychol., 11, 274±293 (1974).

K4. Farquhar, P. H. and Pratkanis, A. R., `Decisionstructuring with phantom alternatives', Manag. Sci.,39, 1214±1226 (1993).

K5. Freeman, K. M., Pratkanis, A. R. and Farquhar,P. H., `Phantoms as psychological motivation:evidence for compliance and reactance processes',University of California, Santa Cruz and CarnegieMellon University, 1990.

K6. Grether, D. M. and Plott, C. R., Èconomic theoryof choice and the preference reversal phenomenon',Am. Econ. Rev., 69, 623±638 (1979).

K7. Kahneman, D. and Tversky, A., `Prospect theory:an analysis of decision under risk', Econometrica, 47,263±291 (1979).

K8. Keeney, R. L. and Raiffa, H., Decisions withMultiple Objectives: Preference and Value Tradeoffs,New York: Wiley, 1976.

K9. Luce, R. D. and Raiffa, H., Games and Decisions,New York: Wiley, 1976.

K10. McCardle, K. F. and Winkler, R. L., `Repeatedgambles, learning, and risk aversion',Manag. Sci., 38,807 (1992).

K11. McCord, M. and de Neufville, R., Èmpiricaldemonstration that expected utility decision analysisis not operational', in Wenstop, S. (ed.), Foundation ofUtility and Risk Theory with Applications, Boston,MA: Reidel, 1983, pp. 181±200.

K12. Pommerehne, W. W., Schneider, F. and Zweifel,P., Èconomic theory of choice and the preferencereversal phenomenon: a reexamination', Am. Econ.Rev., 72, 569±573 (1982).

K13. Saaty, T. L., Fundamentals of Decision Makingwith the Analytic Hierarchy Process, Pittsburgh, PA:RWS Publications, 1994.

K14. Saaty, T. L. and Vargas, L. G., Èxperiments onrank preservation and reversal in relative measure-ment', Math. Comput. Model., 17(3±4), 13±18 (1993).

K15. Tversky, A. and Kahneman, D., `Judgment underuncertainty: heuristics and biases', Science, 185, 1124±1131 (1974).

K16. Tversky, A. and Simonson, I., `Context-dependentpreferences', Manag. Sci., 39, 1179±1189 (1993).

K17. Tversky, A., Slovic, P. and Kahneman, D., `Thecauses of preference reversal', Am. Econ. Rev., 80,204±215 (1990).

K18. Tyszka, T., `Contextual multiattribute decisionrules', in Sjoberg, L., Tyszka, T. and Wise, J. A. (eds),Human Decision Making, Bodafors: Doxa, 1983.

K19. Zeleny, M. Multiple Criteria Decision Making,New York: McGraw-Hill, 1982.

Discussion

On theMeaning of Relative Importance

VALERIEBELTONDepartment ofManagement Science, University of Strathclyde, Glasgow, U.K.

TONYGEARAston Business School, AstonUniversity, Birmingham, U.K.

As anyone who is familiar with our earlierpublications commenting on the analytic hierarchy

process would expect, we are very much inagreement with Salo and HaÈ maÈ laÈ inen who present

DISCUSSION 335


that is not the analytic hierarchy process: what the ahp is and what it is not

Documents