an essay on rank preservation and reversal

Mathematical and Computer Modelling 49 (2009) 1230–1243

Contents lists available at ScienceDirect

Mathematical and Computer Modelling

journal homepage: www.elsevier.com/locate/mcm

An essay on rank preservation and reversalThomas L. Saaty a,∗, Mujgan Sagir ba University of Pittsburgh, United Statesb Eskisehir Osmangazi University, Turkey

a r t i c l e i n f o

Article history:Received 4 June 2008

Keywords:Analytic hierarchy processAdding alternativesDecision makingPhantom alternativesPrioritiesRank reversal

a b s t r a c t

Rank preservation and reversal, so fundamental in decision making, have been anunresolved issue in the field of economics and utility theory and came into focus whenthe Analytic Hierarchy Process was developed because it uses paired comparisons thatinevitably make the priorities of the alternatives interdependent. This paper summarizesthe important issues that can play a role in rank preservation and reversal withcounterexamples to show that preserving rank in all situations is wrong.

© 2008 Elsevier Ltd. All rights reserved.

1. Introduction

This paper dealswith the question: Havingmade a ranking of alternatives that are assumed to be completely independentof one another, what happens to the order in that ranking when a new alternative is added or an existing one deleted?We say there is rank preservation if the order previously determined among the old alternatives is maintained when newalternatives are added (or deleted), and there is rank reversal otherwise. If the alternatives are dependent among themselvesthen anything can happen to rank. Traditionally people assigned the alternatives a number from a scale one at a time.Consequently, rank is always preservedwhen a new alternative is added or an old one deleted because the number assignedis unaffected by the numbers assigned to other alternatives. Of course if new criteria are added (or old ones deleted) orjudgments are changed, then the rankmay also change. As a result of this number assigning approach people came to believethat in reality, barring change in criteria or in judgment, a previously determined rank should not change on introducing ordeleting alternatives. Some, blinded by the use of the technique of assigning numbers one at a time, went on to demand itfrom any other method of measurement even if that method, like the Analytic Hierarchy Process, derived its priorities andranks by comparing the alternatives thus making them dependent on one another.The question is whether rank can in fact reverse in practice without adding new criteria or changing judgments on the

old alternatives. The answer to this is in the affirmative as numerous examples given by a diversity of people will show.There are still people from the old school who attempt to rationalize why the examples are not correct.Ranking or ordering things according to preference is a purely human activity. On the other hand, ranking according to

importance or likelihood is a more scientific or objective activity in which one attempts to project what can happen in thenatural world. Nature has no predetermined rank for the preference of alternatives on specially chosen criteria of its own.It is people who establish the criteria and make their ranking on these criteria.Ranking alternatives on a single criterion involves use of the senses and elementary reasoning and scientific

measurement. To rank alternatives on several criteria which they have in common, we need to not only evaluate thealternatives with respect to each criterion but also evaluate the criteria themselves with respect to higher criteria or directly

∗ Corresponding author.E-mail addresses: [email protected] (T.L. Saaty), [email protected] (M. Sagir).

0895-7177/$ – see front matter© 2008 Elsevier Ltd. All rights reserved.doi:10.1016/j.mcm.2008.08.001

http://www.elsevier.com/locate/mcm

http://www.elsevier.com/locate/mcm

mailto:[email protected]

mailto:[email protected]

http://dx.doi.org/10.1016/j.mcm.2008.08.001

T.L. Saaty, M. Sagir / Mathematical and Computer Modelling 49 (2009) 1230–1243 1231

with respect to a goal and then use their derivedweights to synthesize the resulting individual rankings of the alternatives toproduce an overall ranking. The ranks of the criteria are not usually known in advance, nor are they somehow hidden in ourintuitive understanding. We must reason analytically in order to surface and create their priorities. By making comparisonswe are concerned with the strength or multiplicity of dominance of one element over another (the smaller or lesser one)taken as the unit with respect to a give property or criterion. It is necessary that the elements be close or homogeneous inorder for judgments to be accurate. Otherwise, they are put into homogeneous clusters of successive orders of magnitudewith a common pivot from one cluster to an adjacent cluster. The cognitive psychologist Arthur Blumenthal [1] describesin his book that making comparisons is an intrinsic capability of the mind. It is not an invention by man like all scales ofmeasurement which of course each has an origin and an arbitrarily chosen unit based on which the measurements must beinterpreted with ‘‘subjective judgment’’ as to their importance in each application.Ranking belongs to the field of order topology inmathematics in contrast tometric topology used in science, engineering

and economics measurement that is concerned with how close differences in the numbers are. In ranking and derivingpriorities, closeness depends more on the ratios of the numbers (how much more important is one criterion than another)and not on their numerical differences. It turns out that to derive numbers for which ratios are meaningful does not requirethe use of scales with arbitrarily chosen units and with an origin. The numbers can simply be relative in a more general waythan forming the ratio of numbers obtained from a scale with a unit. Such ratios are dimensionless and belong to an absolutescale that is invariant under the identity transformation.There are two major types of orientations needed in ranking alternatives: One is philosophical (also ethical and hence

normative) which says that rank must be preserved for moral, economic or political reasons. It is concerned with what isgood, a subjective concern. The other is technical. It is concerned with what is right or reasonable or true and is descriptivebased more on reason than on value and is objective allowing rank naturally after making comparisons.It turns out that there is no single categorical answer towhen to preserve rank andwhen not to. Rank ismostly preserved

not because of a law of nature or a result of mathematical dicta, but because it causes less trouble with people due to itsapparent fairness. People erroneously believed in the past that rank has to be preservedmostly because of the limitations ofthe techniques they used to create rank order. It is safe to say that to almost any rule onemaywish to put down for preservingrank there is a counter example which violates that rule. The techniques we use to establish rank order are created to serveour needs and values and not the otherway around. There is a good reason for that.We believe that ranking not only dependson our criteria but also on the way our brains and memory work to evaluate alternatives even when we have some naturalmetric to use inmeasurement.Whenwe deal with intangibles, it is difficult to declare that two alternatives are independentof one another in their ranks, because they are both in our memory and when we think of one we are also aware of theother and have difficulty assigning it a value uninfluenced by our knowledge of the other. Alternatives are related in thejudgment process and cannot be arbitrarily assumed to be independent. When alternatives are slightly dependent and theyare compared, their ranks can only be preserved by force as will be shown later.A complex decision needs to be carefully structured so that all the necessary factors are representedwithin the structure.

It also needs judgments in some form to develop priorities. The judgments may be provided by a single expert or by a groupof people whose opinions may differ. How to reconcile such differences is a subject that does not have much bearing on ourpresent considerations about rank preservation and reversal [2].What we believe are the important factors that have bearing on rank and its preservation and reversal we discuss in

the remaining sections of the paper. They have to do with the measurement of tangibles and intangibles in relative orabsolute form, by assigning numerical values to alternatives one at a time or through comparisons, and how assumptions ofdependence and independence affect the outcome.Much of the literature about ranking alternatives assumes independenceamong the alternatives and of the criteria from other criteria and of course also from the alternatives. An exception is theAnalytic Network Process (ANP) with its very broad approach to the importance, preference and likelihood (probability)of influences. It is concerned with the dependence of criteria among themselves and on alternatives and of alternatives onother alternatives.Thus, for our purpose the quest for causes of rank reversal narrow down to two basic factors that influence the ranking

of alternatives: the method of measurement and the assumption of independence. If alternatives depend on each other or ifthe criteria depend on the alternatives then it is known that anything can happen to the rank of the alternatives. It appearsthat treating alternatives as if they are independent is a questionable matter and at best is an assumption that one has hadto live with for convenience and not because it can be perfectly justified in practice.

2. Measurement

To rank a set of alternatives with respect to several attributes or criteria that they have in common, we need to evaluatethe alternatives with respect to each attribute, and then synthesize the resulting information to produce an overall rankfor the alternatives. The ranks are not usually known in advance, nor are they assumed to be hidden in our intuitiveunderstanding. We must reason analytically through our preferences to surface and create a rank.We need to make one thing clear at the start. When we apply conventional measurement to objects or people like

measuring the height of people in inches and their weight in pounds we have two rankings of the people in terms ofthese numbers. To obtain an overall ranking on both criteria we need to tradeoff height against weight in some way and inprinciple use the priorities thus obtained to weight the corresponding numbers and add them to get one set for the overall

1232 T.L. Saaty, M. Sagir / Mathematical and Computer Modelling 49 (2009) 1230–1243

Fig. 1. Five figures drawn with appropriate size of area.

Table 1Judgments, outcomes, and actual relative sizes of the five geometric shapes

Figure Circle Triangle Square Diamond Rectangle Eigen vector Actual relative size

Circle 1 9 2 3 5 .462 .471Triangle 1/9 1 1/5 1/3 1/2 .049 .050Square 1/2 5 1 3/2 3 .245 .234Diamond 1/3 3 2/3 1 3/2 .151 .149Rectangle 1/5 2 1/3 2/3 1 .093 .096

ranks. But the measurement of height and weight is made by scales that have arbitrarily chosen units applied linearly in themeasurement, and that is disturbing because height and weight can have different significance depending on how large orsmall they are. In basket ball for example a slightly taller person may be disproportionately more effective in scoring pointsthan another person who is a little shorter. The opposite may hold for weight since a tall but heavy person may not be ableto move fast and his weight offsets his advantage of height. Thus, the actual numbers on a height scale and on a weight scaleeven when used in reciprocal formmay not convey the advantage or disadvantage of that factor and the effectiveness of thenumbers needs judgment to determine their priority.In any case, we see that inmulti-criteria decisionmaking onemust also obtainmeasurements and ranking for the criteria

and use them to synthesize the ranking of the alternatives under each criterion. But there are no scales of measurement forthe criteria themselves in term of higher level criteria. They need to be compared relative to each other to obtain cardinalnumerical values for them to use in the synthesis of the priorities of the alternatives. Thus, multi-criteria ranking cannotescape making comparison.There are two ways to generate cardinal measurements:(a) With a pre-existing scaleA scale is a triple: A set of objects, a set of numbers and a mapping of the objects to the numbers. A simple example of a

scale is the telephone directory in which each address is assigned a number.As we noted above, measurements on a scale with an arbitrary unit must be interpreted for the significance of the

purpose they are gathered. To interpret their significance or priorities needs making comparisons with respect to an idealor with other similar measurements. Thus, suchmeasurements by themselves are inadequate for use in ranking. In practice,one often abdicates interpreting the significance of measurements particularly when they fall outside the range of one’sexperience to formulate judgments to compare them, and one often without adequate justification uses them directly afternormalization.(b) With numerical comparisons to derive a relative scale of values.There are two ways to make comparisons: The first is to compare numbers from a preexisting scale by taking their

differences or forming their ratios. The second is to pairwise compare the alternatives themselves either by identifying thesmaller (or lesser) one as the unit and estimating the dominance of the larger as a multiple of that unit, or by identifyingan ideal alternative and then rating the alternatives one at a time with respect to the ideal. When alternatives are ratedone at a time their scores are created independently of one another and rank is preserved. The scores are obtained by usingdifferent intensity levels. Intensity levels have numerical values derived from pairwise comparisons first of different classesof homogeneous intensity levels (similar to the orders of magnitude concept) and second by comparing homogeneousintensities in each level and combining the two measurements to get an overall weight for the appropriate intensity.Here is a simple illustration of paired comparisons. Fig. 1 shows five geometric areas to which we can apply to the paired

comparison process in a matrix to test the validity of the procedure. We can approximate the priorities in the matrix byassuming that it is consistent for example by normalizing each column and then taking the average of the correspondingentries in the columns (we computed the principal eigenvector in this case as needed to deal with inconsistency). Table 1gives the outcome.The object is to compare them in pairs to reproduce their relative weights by eyeballing them.


Note the closeness of the last two columns in Table 1, one derived from judgment and the other from actualmeasurementthat were then normalized. By comparing more than two alternatives in a decision problem, one is able to obtain bettervalues for the derived scale because of redundancy in the comparisons, which helps improve the overall accuracy of thejudgments.Unlike the oldway of assigning a number from a scale with an arbitrary unit once and for all to each thing or object, in the

new paradigm of measurement, the measurements depend on each other in the context of the problem and its objectives.While things may or may not depend on each other according to their function, they are always interdependent accordingto their measurements. Thus, it is no surprise that different conclusions can be drawn about their ranks when elements areadded to or deleted from the collection.Whenwe compare alternatives, the number assigned to an alternative depends on how good or poor are the alternatives

withwhich it is compared. It turns out that rank does not changewhenmaking comparisonswith respect to a single criterionif the comparison judgments are consistent. However, it can change when synthesis is made by weighting and adding toobtain an overall rank with respect to multiple criteria. Some people have tried to invent ways not to let that happen, likeraising the alternatives to the power of the criteria and multiplying (a geometric mean approach) but that heroic attemptfailed because raising numbers to powers that are positive but less than one leads to an opposite outcome to that which onecan expect.The question is whether as in ratings, the same process can also be used in comparisons to preserve rank. Rank can be

preserved by using the ideal mode. The ideal mode involves dividing all the priorities of the alternatives with respect to acriterion by the largest priority among them thus obtaining the value one for the alternative with the largest value that isnow called the ideal. The ideal mode is applied to the original set of alternatives and from then on each new alternative iscompared only with the ideal alternative under each criterion and when it is better than the ideal, its priority is allowed toexceed one. We also note that deleting an alternative other than the ideal cannot change rank. In ratings, the ideal mode isused for the intensities and each alternative is assigned its appropriate intensity by comparing it with the ideal. We notethat the ideal is relative to our previous knowledge and experience. As new alternatives arrive the concept of an ideal maychange and one may have to revise the earlier ratings of the alternatives because of the new alternatives. This is a case ofdependence of alternatives on other alternatives in judgment. A good example of the ideal mode and wrong outcome topreserve rank is shown in Figs. 2–4 of Section 5.

Condition for rank preservation in relative measurement (Gang Hao [3])

Let us take the simple case of normalized columns. Let x = (x1, . . . , xn) denote the vector of criteria weights and lety = (y1, . . . , ym), y ∈ R be the vector of composite weights of the alternatives, ordered in such a way that y1 ≥ y2 ≥· · · ≥ ym ≥ 0. Equivalently, we write y1 − y2 ≥ 0, y2 − y3 ≥ 0, . . . , ym−1 − ym ≥ 0, ym ≥ 0, which is obtained by thetransformation SyT ≥ 0 where

S =

1 −1 0 . . . 0 00 1 −1 . . . 0 0...

......

......

0 0 0 . . . 1 −10 0 0 . . . 0 1

.Let us call an alternative irrelevant only when its measurement under every criterion is zero. Assume this is the case

for A. Now let us introduce a relevant alternative, A′m, by changing these zeros to positive values. Let the columns of A begiven vectors a1, . . . , an ∈ Rm for which the last component is new. Thus, prior to introducing A′m we have (e

m)Taj = 0, j =1, . . . , nwhere (em)T = (0, 0, . . . , 0, 1) is anm vector. Prior to A′m we have

yT =n∑j=1

xjaj

eTaj

with eTaj, j = 1, . . . , n designating the column sums. By requiring that SyT ≥ 0 we have

SyT =n∑j=1

xjSaj

eTaj≥ 0.

Now if we change the last component of the vectors aj from zero to uj ≥ 0, we replace aj by aj + ujem, j = 1, . . . , n. Thus,we compute

y =n∑j=1

xjaj + ujem

eT(aj + ujem).


Evidently eT(ujem) = uj and, hence, for rank preservation we need the first (m−2) components on the right of the followingexpression to be nonnegative:

Sy =n∑j=1

xjSaj + ujSem

eTaj(1+ uj

eT aj

) =

y1 − y2...

ym−2 − ym−1ym−1 −

∑j

xjeTaj + uj

uj∑j

xjeTaj + uj

uj

.

It is clear that Sy ≥ 0 does not always hold and hence S is not isotone and therefore does not preserve rank. The followingare sufficient conditions for rank preservations.

Theorem. Sufficent conditions that y1 ≥ y2 ≥ · · · ≥ ym−1 holds are

ak ≥ ak+1, for k = 1, . . . ,m− 2.

Proof. Since y =∑nj=1 xj

aj+ujem

eT(aj+ujem)=∑nj=1 Xj

aj+ujem

eT(aj+uj),we have

yk =n∑j=1

xjajk

eTaj + uj, for k = 1, 2, . . . ,m− 1, and

ym =n∑j=1

xjajk + ujeTaj + uj

.

Then for k = 1, 2, . . . ,m− 2,we have

yk − yk+1 =n∑j=1

xjajk

eTaj + uj−

n∑j=1

xjajk+1eTaj + uj

=

n∑j=1

xjeTaj + uj

(ajk − a

jk+1

).

We can easily see that if (ajk − ajk+1) ≥ 0,we can obtain yk − yk+1 ≥ 0.

More generally, with ρj =µjeTaj, j = 1, . . . , nwe require that

n∑j=1

xj

(Saj

eTaj1

1+ ρj+

ρj

1+ ρjSem

)≥ 0, Sem =

0...−11

.Let bj =

Saj

eTaj

and recall that∑nj=1 xj = 1, xj ≥ 0. The original condition Sy

T≥ 0 means that the point

∑nj=1 xjb

j is in the simplex spannedby b1, . . . , bn in the positive orthant. The introduction of a new alternative changes the boundary edges of the cone, allowingfor the possibility that the new cone contains points other than those in the positive orthant. These points correspond torank reversals. �

3. Independence

A central concern in ranking is whether alternatives are independent or dependent on one another. This is a complexissue for the following reasons. Alternatives may be independent of one another in space and time and function (functionaldependence is illustrated by having one industry depend on another for its input or output) but always depend on each otherin the mind in making comparisons because once identified, they are remembered and categorized and cannot be forgotteneasily and the mind inevitably compares and contrasts themwhether we like it or not. By having many alternatives, we cancreate an ideal and compare each alternative with that ideal or with one another. Thus, we have two kinds of dependence,the first according to function and the second according to structure and relations in themind. This indicates that one should


never treat alternatives as if they are independent with respect to judgment even when they are independent with respectto function.At the risk of repeating, there is no guarantee that themind evaluates anything only in terms of howmuch it has a certain

attribute. Granted there are measurements that are physically intrinsic in objects and these measurements indicate thenumerical standing of the alternatives on a certain property in an irrevocablemanner. But in the end onemust determine thesignificance of suchmeasurement to the purpose one has inmind and how to combine themwith other intangible andmoresubjectively evaluated measurements. Such hard measurements do not have an intrinsic order that has the same priorityin all applications. So the mind may prefer evaluating something as very much or very little or in between depending on itsexperience with similar alternatives and thus the evaluation of that alternative depends on knowledge of other alternatives.In fact the knowledge one has depends on all the alternatives that one is familiar with.In addition society teaches one another the information they have about all known alternatives but some may be

forgotten. Thus, in a sense ranking an alternative depends on all other alternatives known to anyone.

4. Invariance and rationality

One can trace the origin of the axioms of rank preservation among others to the book by Luce and Raiffa [4]. They write:

‘‘Adding new acts to a decision problem under uncertainty, each of which is weakly dominated by or is equivalent to someold act, has no effect on the optimality or non-optimality of an old act.

and elaborate it with

If an act is non-optimal for a decision problem under uncertainty, it cannot be made optimal by adding new acts to theproblem.

and press it further to

The addition of new acts does not transform an old, originally non-optimal act into an optimal one, and it can change anold, originally optimal act into a non-optimal one only if at least one of the new acts is optimal.

and even go to the extreme with:

The addition of newacts to a decision problemunder uncertainty never changes old, originally non-optimal acts into optimalones.

and finally conclude with:

The all-or-none feature of the last form may seem a bit too stringent . . . a severe criticism is that it yields unreasonableresults’’.

Somepeople have taken this last formof invariance as themajor tenet of their so called normative theory. A normative theorysays that you must do it my way or you get wrong decisions. This is essentially what is called the criterion of rationality. Tobe a rational decision maker, one must accept and practice through the axioms of utility theory. The principle of invarianceand the principle of rationality lead to a paradox because rank has been observed to reverse in practice violating invarianceand suggesting that rationality as defined by utility theory is untenable. Practitioners of utility theory have insisted thattheir theory is the best possible; it is the norm. Reexamination of the axiomatic assumptions of utility theory is essential tobe comfortable with that theory.Tyszka [5] has done empirical work to support the hypothesis that there are no irrelevant alternatives in multicriteria

decisions. His experiments deal with ordinal preferences. In a typical experiment, he gave subjects choices among pairs ortriplets of alternatives already ordered with respect to the attributes. They were to choose the most preferred. An irrelevantalternative was introduced in the set preserving the old order, and they were again asked to select the most preferredalternative. The individuals used different decision rules to choose among pairs than among triplets. Two conclusions drawnfrom his detailed experiments are:1 — People selected the best one among the nondominated alternatives and,2 — They avoided choosing dominated alternatives even on a single criterion.Thus, for example, if it occurred that the new alternative dominated some other alternative that was chosen on the first

pass because it was dominant on several but not all the criteria, that alternative was no longer the favored one! The subjectswould have to be forced to comply with rank preservation.In general one expects to preserve rank when a prior decision has already been made or when the new alternatives are

carriers of information that should not affect the ones already ranked (open set of alternatives). To preserve rank one woulduse the procedure for creating an ideal alternative for each criterion and then compares each added alternative directly withthe ideal.


Table 2Style (.4)

A B

A 1 3 .75B 1/3 1 .25

Table 3Rareness (.6)

A B

A 1 1 .5B 1 1 .5

Table 4Style (.4)

A B A1 B1

A 1 3 1 3 .375B 1/3 1 1/3 1 .125A1 1 3 1 3 .375B1 1/3 1 1/3 1 .125

5. Real life examples of rank reversal in violation of classical expectations

There is a diversity of different situations in which rank reversals can occur with ‘‘independent’’ alternatives but withoutchange in criteria or judgments. We will mention some according to our perception of their increasing order of complexity.We first deal with those involving change in judgment that affects the measurement derived from them.If the alternatives form a closed set of mutually exhaustive aspects of a decision and if after they are ranked a new

alternative, needed to round out the possibilities is discovered and added, one might expect the new rank to differ from theold one. The two rankings are often considered independent from each other. If on the other hand the alternatives form anopen set and new ones are added that are independent of the original set, as in sequential student admission to a school,then it may be desired that the rank of the old set should be preserved. So this shows us the situations where one would notwish to undo decisions based on an earlier ranking.

5.1. Copies (number of the same alternatives)

A condition assumed in many individual choice models is regularity. Simply defined it says that the choice probability ofany alternative cannot be altered by increasing or decreasing the number of alternatives of a choice set. Regularity has todo with rank preservation. Corbin and Marley [6] provide an example that, ‘‘concerns a lady in a small town, who wishes tobuy a hat. She enters the only hat store in town, and finds two hats, a and b1, that she likes equally well, and so might beconsidered equally likely to buy. However, now suppose that the sales clerk discovers a third hat, b2, identical to b1. Then thelady may well choose hat a for sure (rather than risk the possibility of seeing someone wearing a hat just like hers), a resultthat contradicts regularity’’. Note that manyness cannot be introduced as a criterion because it implies the dependence ofany alternative on all the alternatives.Let us illustrate rank reversal in relative measurement. We start with the hat example. A plausible set of judgments in

the case of the two hats A and B is given, with A preferred to B as in Tables 2 and 3. Adding A1 that is a copy of A, changes thepreference of B over Awith respect to rareness and thusmakes it themore desired choice overall. However, adding a secondhat B1 restores the preference for A, violating regularity again. Tables 4 and 5 give the judgments related to new alternatives.

WA = .75x.4+ .5x.6 = .6 WA = .42x.4+ .125x.6 = .238WB = .25x.4+ .5x.6 = .4 WB = .16x.4+ .75x.6 = .514,WA1 = .42x.4+ .125x.6 = .238.

Adding B1, a copy of B, restores the original order. The upshot of this is that copies give rise to rank reversal and that thisphenomenon cannot always be accounted for in absolute measurement by introducing criteria or changing criteria weights.Neither of these changed when B1 was introduced. We could go on adding hats without being able to change the weight ofrareness. Thus, we have:

WA = .375x.4+ .25x.6 = .30WB = .125x.4+ .25x.6 = .20

and A is preferred to B.


Table 5Rareness (.6)

A B A1 B1

A 1 1 1 1 .25B 1 1 1 1 .25A1 1 1 1 1 .25B1 1 1 1 1 .25

Table 6Three criteria-three alternatives

C1 C2 C3

A 1/11 9/11 8/18B 9/11 1/11 9/18C 1/11 1/11 1/18

Table 7Adding an alternative

C1 C2 C3

A 1/19 9/12 8/26B 9/19 1/12 9/26C 1/19 1/12 1/26D 8/19 1/12 8/26

Table 8Quality

A B

A 1 5B 1/5 1

5.2. Near copies

We have seen that a copy of an alternative can cause rank reversal. We now show with an example that a near copywhich is an alternative with priority(s) close to an original also can cause rank reversal. Let us look at an example due toBelton and Gear [7]. That example begins with three criteria of equal importance and three alternatives ranked with respectto the criteria as shown in Table 6.

WA = .33x1/11+ .33x9/11+ .33x8/18 = .4467WB = .33x9/11+ .33x1/11+ .33x9/18 = .4650WC = .33x1/11+ .33x1/11+ .33x1/18 = .0790

and B is preferred to A that is preferred to C .If we introduce a fourth alternative D that is a perturbation of Bwith respect to C1, identical to Bwith respect to C2, and

also a perturbation of B with respect to C3 we have:WA = 0.3WB = 0.3WC = .06WD = 0.27 and now A is preferred toB preferred to D preferred to C . Thus, a perturbation or near copy of B has caused rank reversal. It is not difficult to modifythe example so that no matter how small a perturbation one allows, rank reversal can occur. Thus, both copies and smallperturbations of copies can cause rank reversals (Table 7).In addition, if one accepts the fact that a small perturbation can cause rank reversal, it should not be difficult to see that

continued perturbations of these would have similar effects by referring to the perturbed copies and not the originals.

5.3. A Non-copy Alternative

A simple example of rank reversal is the presidential elections of 1991 when the entry of Ross Perot into the electionstook votes away from Bush. The prediction as to who would win the race prior to Perot’s entry is shown in Fig. 2. After Perotentered the race, Bush lost to Clinton as shown in Fig. 3, because Perot took away votes from Bush.If instead of comparing the candidates pairwise, we were to rate them one at a time, adding Perot would not change

the rank order of Bush and Clinton and Bush would be predicted to be the winner, contrary to what happened. Rating oneat a time forces rank preservation, Perot would have no effect, and Bush would wrongly be predicted to win as shown inFig. 4. To force rank preservation by using ratings, start with the situation shown in Fig. 2 and idealize by dividing by thelarger priority of the two candidates, Bush and Clinton, under each criterion. Bush would receive a larger overall prioritythan Clinton. We then assign Perot his proportionate value with respect to the ideal in Fig. 4. This has no effect on the


Fig. 2. Presidential Elections with Standings for Bush and Clinton before Perot.

Fig. 3. Presidential race with three candidates; prediction close to actual result.

Fig. 4. Forcing rank preservation by idealizing gives wrong results.

ranking of Bush and Clinton, so the outcome in Fig. 4 shows that Bush should be the winner. In effect, comparisons take intoconsideration the relative number of people voting for the candidates by considering each criterion separately, and thenweighting and combining the relative numbers.

5.4. Quality of alternatives

Forman [8] gives the following example that illustrates the effect of the number and quality of alternatives introduced.At a company, Susan excels over Jack in every quality except computer skills, where he ismuch needed and thus he is overallpreferred to Susan. When the company hires John, who also has computer skills but is not as good as Jack, Susan becomes


Table 9Price

A B

A 1 1/3B 3 1

Table 10Quality

A B A1

A 1 5 1B 1/5 1 1/5A1 1 5 1

Table 11Price

A B A1

A 1 1/3 2B 3 1 6A1 1/2 1/6 1

the preferred employee and there is reversal in Jack and Susan’s rank. One can see that if one additional computer literateemployee is not sufficient to reverse Jack and Susan’s rank, surely when enough people are hired with computer skills, atsome point such rank reversal would happen.

5.5. Phantoms

Farquhar, Freeman, and Pratkanis [9] in their experiments on phantom alternatives observe that their findings, ‘‘violatethe basic axioms of classical choice theory’’.Marketing literature has many examples of consumer preference rank reversals. An example is the case of a product that

sells for example at $300, but is considered too expensive by customers and is less preferred than one that sells for $150. Astrategy that is often implemented is to introduce a product similar to the $300 one, but pricedmuch higher, e.g., $1000. It iswell documented that the formerly most expensive $300 product then looks like a much better value and is now preferred.Another example deals with the introduction of a phantom alternative that might be priced slightly lower but is not actuallyavailable for purchase because it is ‘‘out of stock’’. The consumer will perceive the existing alternative as an overpriced onedespite its better quality and will select a cheaper one that is also available. These experts believe that normative utilitytheory (and we might add normative absolute measurement) fail to account for such behavior. However, we will showbelow that it can with relative measurement.Again let us account for the effect of phantom products. Two products A and B are evaluated according to quality (Q ) and

price (P). A is 3 times more expensive than B (or simply B has a more desirable price) but it is of considerably better qualitythan B.We assume that Q and P are equally important, and obtain the following priorities (Tables 8 and 9)

WA = .542 WB = .458

Let A1 be a product similar to A in quality and on price dominates B six time more. The addition of A1 makes A lookoverpriced and a consumer may decide that B is the best choice (Tables 10 and 11). and the overall priorities are given by:WA = .338WB = .379WC = .283.

5.6. Decoys

Huber, Payne, and Puto [10] and Huber and Puto [11] provide empirical evidence on the systematic violation of regularityin choice sets with three alternatives. The choice alternatives are described on two attributes for which ‘‘more is better’’on each attribute. They define a decoy as an asymmetrically dominated alternative in the choice set. They show that thepresence of a decoy in the choice set increases the probability of choosing the target alternative close to it. One is familiarwith the example of a sale pamphlet that has an attractive and well priced item that one is told is out of stock. One can ofteninfer that it is placed in the pamphlet as a decoy to induce one to buy another similar item that is available in abundance.This surprising result violates the regularity principle in a predictable direction without imposing any apparent constraintson the choice process. This enhancement of the target’s choice probability has been called the attraction effect, although noconclusive explanation for this phenomenon has appeared in the literature.


Table 12Alternatives with probabilities

p1 p2 = (1− p1)

A1 4 2A2 6 4A3 4 3A4 6 0A5 3 1/3 3 1/3

Table 13Three criteria-three alternatives

Brand Price Quality

X 60 HighY 55 MediumZ 50 Low

5.7. Optimality and rank reversal

An example contradicting the Luce and Raiffa axiom in Section 4 ‘‘if an alternative is nonoptimal, it cannot be madeoptimal by adding a new alternative to the problem’’ is due to Zeleny [12]. Table 12 exhibits lotteries with two possibleoutcomes and gives the payoffs with probabilities p1 and p2, corresponding to five alternatives.Consider the alternatives A1 and A2 only. A2 dominates A1 for all possible combinations of outcomes because

4p1 + 2(1− p1) < 6p1 + 4(1− p1).

A3 and A4 dominate each other for different values of p1 and are equal at p1 = .6. A3 dominates A4 for 0 # p1 < .6 andA4 dominates A3 for .6 < p1#1. If p1 is chosen at random, A3 is the winner 60% of the time and would be chosen over A4.Consider now A3, A4 and A5. A5 is independent of p1 because it gets the same return from p2. However, A5 and A3 dominateeach other for different values of p1 between 0 and .6. Thus, A3 is no long dominant 60% of the time which makes A4 whichis dominant 40% of the time and is independent of A5 the optimal alternative.

5.8. Intransitivity (how a low priority alternative can become dominant)

Robin Hogarth [13] gives the example of Table 13 to illustrate the effect of intransitivity on rank and how a low priorityalternative can become dominant. Assume that of many possibilities the higher quality brand is chosen at the supermarketif its price is not more than 5 cents higher, otherwise the cheaper brand is chosen.In a comparison between X and Y , one would prefer X; between Y and Z , one would choose Y ; however, a comparison

between X and Z leads to the choice of Z . In other words, the apparently sensible heuristic that considers both price andquality implies an inconsistent ordering of one’s preferences over the three brands. The author suggests that the exampleillustrates three important issues in the use of choice rules. First, despite the fact that the rule seems sensible, it can leadto contradictory choices. Second, in adopting what seem to be reasonable heuristics, we often lack insight into how suchrules interact with characteristics of the choice task (in this case the characteristics of the particular alternatives). Moreover,unless the inherent contradictions in such rules are brought to our attention, not only may we never be aware of them, wemight persist in using them to our disadvantage. And third, the example raises the issue of what the desirable properties ofheuristic choice rules should be. Actually if paired comparisons are used one sees the problem in a different light, related tothe degree of inconsistency and strength of relative preference.

6. The analytic network process deals with rank correctly

We now briefly discuss the various forms of performing multicriteria ranking when there is dependence not only of thealternatives on the criteria but also among themselves and of the criteria on the alternatives and among themselves too. Thegeneral formof the AHP, known as the Analytic Network Process (ANP)with its supermatrix approach, dealswith these kindsof dependence. In the AHP, adding alternatives never gives rise to rank reversalwith respect to a single criterion. The AHP canalso preserve rank under multiple criteria by dividing by the weight of the highest ranked alternative after normalization inits ideal mode. Before the ideal mode, rank preservation was assured by using absolute measurement, known as the ratingsprocedure. Ratings involves developing scales of intensities for each of the criteria through paired comparisons, establishinga priority rating for each alternative on these intensities, weighting by the priority of the corresponding criterion, andsumming over the criteria. However, examples and discussion in the literature highlighted the fact that one should be able topreserve rank when necessary even with relative measurement. We were certain that normalization, so critical for derivingrelative measurement from absolute scales particularly when there are mutual dependencies, could not be ignored withoutloss of a fundamental concept. The Distributive mode of the AHP was retained to include the effect of number and rating


Table 14Efficiency (.5)

A B C C C . . . C Norm Ideal

A 1 3 4 4 4 . . . 4 .16 1B 1/3 1 4/3 4/3 4/3 . . . 4/3 .05 .33C 1/4 3/4 1 1 1 . . . 1 .04 .25C 1/4 3/4 1 1 1 . . . 1 .04 .25. . . . . . . . . .C 1/4 3/4 1 1 1 . . . 1 .04 .25

Table 15Cost (.5)

A B C C C . . . C Norm Ideal

A 1 1/2 4 4 4 . . . 4 .13 .50B 2 1 8 8 8 . . . 8 .25 1C 1/4 1/8 1 1 1 . . . 1 .03 .13C 1/4 3/4 1 1 1 . . . 1 .04 .25C 1/4 1/8 1 1 1 . . . 1 .03 .13C 1/4 1/8 1 1 1 . . . 1 .03 .13. . . . . . . . . . . ..C 1/4 1/8 1 1 1 . . . 1 .03 .13


A B Norm. Ideal.

A 1 3 .75 1B 1/3 1 .25 .33

Table 17Costs (.5)

A B Norm. Ideal.

A 1 2 .67 1B 1/2 1 .33 .5

of alternatives when needed. However, the AHP was also extended to include the aforementioned rank preservation modecalled the Ideal mode which preserves rank with respect to dominated (and hence irrelevant) alternatives. Both reversaland preservation are essential in decision making and need to be considered in practice. The ANP is implemented throughthe software SuperDecisions which incorporates the Distributive and Ideal modes in addition to the Absolute Measurementor Ratings mode. We note with emphasis that assigning numbers to alternatives one at a time by any conventional methodor by using the Ratings mode cannot and should not allow for rank reversals and fails to account well for the rank reversalphenomenon that plagues natural and economic occurrences because it cannot deal with complexity of dependence in bothfunction and structure. Here are two examples of rank preservation and reversal in relative measurement.

6.1. Rank preservation with the ideal mode: Introducing irrelevant alternatives

If we add many copies of an ‘‘irrelevant’’ or in better words, ‘‘dominated’’ alternative C (total Cs are 20) in a probleminvolving two equally weighted criteria, Efficiency and Cost, we have for example for paired comparisons using the AHP 1-9scale, rounded off to two decimal places (Tables 14 and 15).For synthesis from Tables 14 and 15 the Distributive mode gives: A = .145, B = .150, C = .040, or, normalized,

A = .43, B = .45, C = .12, and adding copies of the irrelevant (dominated) alternative C will eventually, like the proverbial‘‘straw that broke the camel’s back’’, cause rank reversal as we have just seen.The Ideal mode gives: A = .75, B = .68, C = .19, and there is no rank reversal regardless of how many copies of C are

added.

6.2. Rank reversal with the ideal mode: Introducing a relevant (phantom-like) alternative

Again we begin with A and B as above. We have Tables 16 and 17 and the distributive mode gives A = .71 and B = .29while the Ideal mode gives A = 1.00 and B = .42 (or normalized again A = .71, B = .29).Now, if we add C that is a relevant alternative under efficiency, because it dominates both A and B we obtain Tables 18

and 19.



A B C Norm. Ideal.

A 1 3 1/2 .3 .50B 1/3 1 1/6 .1 .17C 2 6 1 .6 1

Table 19Costs (.5)

A B C Norm. Ideal.

A 1 2 4 .31 .5B 1/2 1 8 .62 1C 1/4 1/8 1/8 .08 .13

Table 20Distributive vs. ideal

Number of criteria Number of alternatives2 3 4 5 6 7 8 9

2 988 965 954 949 958 951 954 9403 967 952 934 944 926 927 938 9324 973 948 901 904 916 912 027 9015 974 924 920 911 902 880 901 9106 951 921 906 914 897 906 889 9117 953 919 893 901 900 903 903 9008 957 927 913 865 908 879 892 8979 972 910 916 881 882 875 889 916

Table 21Distributive vs. utilities


2 859 877 882 918 903 923 913 9173 830 849 855 871 873 875 897 8934 841 832 795 823 860 854 878 8635 801 817 832 831 835 831 849 8806 791 796 798 829 826 837 840 8577 790 793 794 806 824 848 839 8478 803 813 806 781 833 803 840 8539 808 785 812 795 813 801 845 868

Table 22Utilities vs. ideal


2 871 906 920 947 949 959 955 9673 855 891 906 916 931 936 949 9494 862 877 881 902 931 915 939 9465 821 874 890 899 907 926 923 9416 830 850 869 892 907 914 925 9257 825 855 869 886 906 924 916 9238 836 867 875 879 910 890 928 9329 826 859 870 886 902 895 924 928

Here with the Distributive mode A = .30, B = .36 and C = .34 and with the Ideal mode A = .50, B = .59, and C = .57which when normalized becomes: A = .30, B = .36, and C = .34. There is rank reversal with both the Distributive and theIdeal modes because C is dominant with respect to efficiency.An experiment involving 64,000 hierarchies with random assignment of priorities to criteria and to alternatives was

conducted to test the number of times the best choice obtained by normalizing, or by dividing by the maximum (the Idealcase) or by converting to utilities coincided with each other (Tables 20–22). It turns out that all three methods yield thesame answer nearly all the time. We have included three tables to show the comparisons. Each number indicates the totalcoincidences of the best choice out of 1000 experiments with the indicated number of criteria and alternatives. For examplethe first entry 988 in Table 20 gives the number of times out of 1000 trials that the best alternative chosenwas the samewhen


the distributive and the ideal modes were used. We note that if one excludes the distinctions on uniqueness introduced bycopies, for most decisions it does not matter which mode one uses to rank the alternatives. The same kind of experimentwas performed to note the coincidence of the 1st and 2nd ranked alternatives. A similar type of conclusion was obtained forthe three methods when no copies were involved [14].

7. Conclusion

It appears that rank preservation is often a normative or desired concern to make life more orderly and tractable.Rank reversals are a more serious concern because they can happen naturally and affect outcomes in both desirable andundesirable ways. In this paper we have shown that measurement and dependencies are important parameters in dealingwith the question of rank. The AHP has been extended so that with relativemeasurement rank is preservedwhen desired byusing the ideal mode, and allowed to reverse as necessary by using the distributivemode.We have seen abundant examplesfrom the literature that indicate the need for both. Our experience shows that if one wishes to obtain the correct ranking foralternatives with both independence of the criteria from the alternatives and with their dependence on them, one shoulduse the approach to measurement offered by the analytic network process and its supermatrix. In the supermatrix, whenthe criteria are independent of the alternatives the ideal mode is used for the alternatives. However, when the criteria aredependent on the alternatives, the distributive mode must be used. We believe that way the rank preservation and reversalissue is resolved.Our final conclusion is that in order to control rank reversals so they can only occur in an accountable way in any decision

problem we need to use the multicriteria method for the measurement of intangibles with dependence and feedback, theAnalytic Network Process (ANP) [15].

Acknowledgments

The second author thanks the Scientific Research Projects Committee of the Eskisehir Osmangazi University for its partialsupport of this study.

References

[1] A.L. Blumenthal, The Process of Cognition, Prentice Hall, 1977.[2] T.L. Saaty, K. Peniwati, Group DecisionMaking: Drawing out and Reconciling Differences, RWS Publications, 4922 Ellsworth Ave., Pittsburgh, PA, 2008,p. 15213.

[3] T.L. Saaty, The Analytic Hierarchy Process, RWS Publications, 4922 Ellsworth Avenue, Pittsburgh, Pennsylvania, 1990, p. 15213.[4] R.D. Luce, H. Raiffa, Games and Decisions, Wiley, New York, 1957.[5] Tadeusz Tyszka, Contextual Multiattribute Decision Rules, in: Lennart Sjoberg, Tadeusz Tyszka, James A. Wise (Eds.), Human Decision Making Doxa,Bodafors, Sweden, 1983.

[6] R. Corbin, A.A.J. Marley, Random utility models with equality: An apparent, but not actual, generalization of random utility models, Journal ofMathematical Psychology 11 (1974) 274–293.

[7] V. Belton, A.E. Gear, On a short-coming of Saaty’s method of analytic hierarchies, Omega 11 (3) (1983) 228–230.[8] E.H. Forman, Multicriteria prioritization in open and closed systems, George Washington University, 1992.[9] K.M. Freeman, A.R. Pratkanis, P.H. Farquhar, Phantoms as psychological motivation: Evidence for compliance and reactance processes, University ofCalifornia, Santa Cruz, and Carnegie Mellon University, Pittsburgh, 1990.

[10] J. Huber, J.W. Payne, C. Puto, Adding asymmetrically dominated alternatives: Violations of regularity and the similarity hypotheses, Journal ofConsumer Research 9 (1) (1982) 90–98.

[11] J. Huber, C. Puto, Market boundaries and product choice: Illustrating attraction and substitution effects, Journal of Consumer Research 10 (1) (1983)31–44.

[12] M. Zeleny, Multiple Criteria Decision Making, McGraw Hill, New York, 1982.[13] R. Hogarth, Judgment and Choice, John Wiley and Sons, New York, 1987.[14] T.L. Saaty, L.G. Vargas, Experiments of rank preservation and reversal in relativemeasurement, Mathematical and ComputerModelling 17/4 (5) (1993)

13–18.[15] T.L. Saaty, Theory and Applications of the Analytic Network Process: DecisionMakingwith Benefits, Opportunities, Costs, and Risks, RWS Publications,

4922 Ellsworth Ave., Pittsburgh, PA, 2005, p. 15213.

an essay on rank preservation and reversal

Documents