iris: a dss for multiple criteria sorting problems€¦ · the main feature of iris is that it does...

JOURNAL OF MULTI-CRITERIA DECISION ANALYSIS

J. Multi-Crit. Decis. Anal. 12: 285–298 (2003)

Published online in Wiley InterScience (www.interscience.wiley.com) DOI: 10.1002/mcda.364

IRIS: ADSS forMultiple Criteria Sorting Problems

LUIŁSC. DIASa,b* and VINCENTMOUSSEAUc

aFaculdade de Economia, Universidade de Coimbra, Av. Dias da Silva,165, 3004-512 Coimbra, Portugalb INESCCoimbra, Rua Antero de Quental,199, 3000-033 Coimbra, PortugalcLAMSADE, Universit!ee Paris-Dauphine, Place du Mar!eechal De Lattre de Tassigny, 75775 Paris Cedex 16,France

ABSTRACT

This paper presents Interactive Robustness analysis and parameters’ Inference for multicriteria Sorting problems(IRIS), a Decision Support System (DSS) designed to sort actions (projects, candidates, alternatives, clients, etc.)described by their performances on multiple criteria into an ordered set of categories defined a priori. It is based onthe ELECTRE TRI sorting method, but does not require the decision maker (DM) to indicate precise values for allof the method’s parameters. More realistically, the software expects the DM to indicate some constraints that theseparameters should respect, including sorting examples that the program should reproduce. If the constraintsindicated by the DM do not contradict each other (i.e. form a consistent system), then IRIS infers a combination ofparameter values that reproduces all the sorting examples, indicating also the range of possible assignments ofactions to categories that would be possible without violating any of the stated constraints. If the constraints arecontradictory (i.e. form an inconsistent system), then IRIS suggests a combination of parameter values thatminimizes an error function and identifies alternative ways to restore the system’s consistency by removing someconstraints. Copyright # 2005 John Wiley & Sons, Ltd.

KEY WORDS: decision support systems; sorting; classification; ELECTRE TRI; software

1. INTRODUCTION

This paper presents the Interactive Robustnessanalysis and parameters’ Inference for multicriter-ia Sorting problems (IRIS) software, a DecisionSupport System (DSS) for multicriteria ordinalclassification (sorting) problems. In classificationproblems, a set of ‘objects’ (which we call actions)is to be classified into different categories. Theseactions (projects, candidates, alternatives, clients,students, etc.) are described by a vector evaluatingtheir performance on multiple criteria. For in-stance, criteria such as ‘publications’, ‘advancedformation’, or ‘links with industry’ can be used toclassify R&D institutions into categories such as‘poor, ‘fair’, ‘good’ and ‘excellent’. Other examples

can be the classification of loan requests intocategories ‘reject’, ‘accept with high interest rate’,‘accept with low interest rate’, or to sort acompany’s employees into categories associatedwith incentive packages, or to sort automobilesinto categories related with environmental friend-liness. In all these examples, categories can beordered by increasing preference for the decisionmaker (DM) and thus are often called sortingproblems (e.g. Greco et al., 2001; Zopounidis andDoumpos, 2002). Other important aspects are thatcategories are defined a priori, and each action iscompared to the definitions of the categoriesindependently from the other actions. Hence,absolute evaluations are at stake, rather than therelative evaluations that occur in choice or rankingproblem statements, where actions are comparedone to the other (Roy, 1996).

Multicriteria classification methods can be dis-tinguished from others (e.g. methods from statis-tics) in that classification does not automaticallyresult from the vectors describing the actions, butdepends on the judgment of a DM. The DMdefines the ‘boundaries’ of the categories, theimportance of each criterion, etc. The DM’sjudgment is represented in the method through

Received 25 July 2003Copyright # 2005 John Wiley & Sons, Ltd. Accepted 13 December 2004

*Correspondence to: Faculdade de Economia, Univer-sidade de Coimbra, Av. Dias da Silva, 165, 3004-512Coimbra, Portugal. E-mail: [email protected]

Contract/grant sponsor: ICCTIContract/grant sponsor: French Embassy in Portugal;contract/grant number: 328J4 Contract/grant sponsor:French Embassy in Portugal; contract/grant number:500B4

values assigned to preference parameters. How-ever, it is usually difficult for a DM to assignprecise quantitative values to these parameters.Moreover, the parameters reflect preferences thatare often vague and that may change with time. Insome situations, there will not exist a singleisolated DM, but a set of DMs conjointlyresponsible for the decision whose preferences donot perfectly match. For all these reasons, in manycases, it is wise to avoid questioning the DM(s)directly in a quest for precise parameter values. Asan alternative, the DM can be aided by proceduresthat infer parameter values from sorting examples(Mousseau and Slowinski, 1998) and/or by proce-dures that work with imprecise information andperform robustness analysis (Dias and Cl!ıımaco,2000).

The IRIS software intends to become a refer-ence implementation of the ELECTRE TRImethod, succeeding the implementations ofYu (1992) and Mousseau et al. (1999). It supportsthe methodology proposed by Dias et al. (2002),Mousseau and Dias (2004) and Mousseau et al.(2003), based on the method ELECTRE TRI(Yu, 1992; Roy and Bouyssou, 1993), combiningparameter inference with robustness analysis.The main feature of IRIS is that it does notrequire precise values for the criteria importanceparameters nor the method’s outranking cuttinglevel. The DM may instead indicate constraints(intervals or relations) that the parameters shouldrespect. In particular, IRIS allows entering con-straints in the form of sorting examples (for someparticular actions chosen by the DM, he/sheindicates a category or an interval of categorieswhere the actions should be classified accordingto his/her judgment). When the constraintsdefine a consistent system, IRIS computes a‘central’ combination of parameter values thatrespects all the constraints and the correspond-ing sorting, additionally indicating the range ofcategories where each action could have beenclassified without violating any constraint. Whenthe constraints contradict each other (i.e. form aninconsistent system), IRIS suggests alternativeways of removing subsets of constraints ina way that restores consistency (Mousseau et al.,2003).

The next section briefly overviews the ELEC-TRE TRI method. Section 3 summarizes themethodology implemented in IRIS on how toconstruct an ELECTRE TRI sorting model in aninteractive way. Section 4 presents the structure of

the IRIS user interface, which is illustrated by a‘guided tour’ in Section 5. Section 6 brieflypresents two real-world applications of IRIS.Section 7 ends the paper.

2. BRIEF OVERVIEW OF ELECTRE TRI

The ELECTRE family of methods for multi-criteria decision aiding has been developed byBernard Roy and his collaborators for the lastthree decades (Roy, 1991; Roy and Bouyssou,1993). Among these methods, ELECTRE TRI(Yu, 1992; Roy and Bouyssou, 1993) has beenspecifically designed for multicriteria sorting pro-blems.

Let A ¼ fa1; . . . ; amg denote a set of m actionsevaluated according to n criteria (functions)g1; . . . ; gn. We denote C ¼ fC1; . . . ;Ckg, a set ofk categories by preference order, C1 being the leastpreferred (worst category) and Ck the mostpreferred (best category). Each category Ch ðh ¼1; . . . ; kÞ is defined by two profiles: bh is its upper-bound profile, whereas bh�1 is its lower-boundprofile. Thus, it is necessary to define k+1 profilesb0; . . . ; bk such that, except the first one and thelast one, each profile is simultaneously the upperbound of a category and the lower bound of thecategory above it (Figure 1). Yu (1992) hasestablished the conditions that these profilesshould respect, namely: each profile bh must bepreferred to profile bh�1 according to all thecriteria, and the profile b0 (bk) must be worse(better, respectively) on all criteria than the actionsto sort.

The assignment of actions to categories is basedon the concept of outranking relation (a binaryrelation meaning ‘not worse than’). An action ai issaid to outrank a profile bh (denoted by ai S bh)

Figure 1. Definition of categories using profiles.

L. C. DIAS AND V. MOUSSEAU286

Copyright # 2005 John Wiley & Sons, Ltd. J. Multi-Crit. Decis. Anal. 12: 285–298 (2003)

when there are sufficient arguments to state that aiis at least as good as bh. Formally, a credibilitydegree s(ai,b

h) is computed first (a valued out-ranking relation), considering the ‘weight’ of thecriteria that agree with the statement ai S bh andthe discordance opposed by the remaining criteria(see Section 2.1). Then, this index is comparedwith a cutting level l:

ai S bh iff sðai; bhÞ � l ð1Þ

The category of each ai 2 A is found by comparingit with the successive profiles. According to thepessimistic version (the most used one), eachaction ai 2 A is assigned to the highest categoryCh such that ai outranks its lower-bound profileðbh�1Þ. This is equivalent to

ai ! Ch iff sðai; bh�1Þ � l and sðai; bhÞ5l

ð2Þ

since in ELECTRE TRI, the profiles verify thecondition that

sðai; bhÞ5l ) sðai; bhþ1Þ5l

8ai 2 A; k ¼ 0; . . . ; k� 1

2.1. Computation of credibility indices sða; bÞ

2.1.1. Computation of single-criterion concordan-ce. Let us use Djab to denote the advantage of anaction a over another action b on criterion gj:

Djab ¼

gjðaÞ � gjðbÞ if criterion gj is to be

maximized ðthe more

the betterÞ

�gjðaÞ þ gjðbÞ if criterion gj is to be

minimized

8>>>>>><>>>>>>:

The concordance index of criterion gj regardingthe assertion a S b is computed as follows:

cjða; bÞ

¼

0 if Djab5� pj

ðDj þ pjÞ=ðpj � qjÞ if � pj � Djab5� qj

1 if Djab � �qj

8><>:

The criterion fully agrees with a S b wheneverthe advantage Djab is positive or, if negative, whenthe disadvantage (i.e. �Djab) does not exceed thecriterion’s indifference threshold qj. Concordance isnull if the disadvantage reaches or exceeds the

preference threshold pj. This single-criterion con-cordance index varies linearly in between these twothresholds.

2.1.2. Computation of global discordance. Thesingle-criterion concordance indices are aggre-gated into an overall (multicriteria) concordanceindex by the operation

cða; bÞ ¼Xnj¼1

kj :cjða; bÞ

where kj (a non-negative value) represents theimportance coefficient (‘weight’) of the criterion gjðj ¼ 1; . . . ; nÞ. For normalization, and without lossof generality, the sum of the weights k1; . . . ; knshould be 1.

2.1.3. Computation of single-criterion discordan-ce. The discordance index of criterion gj regardingthe assertion a S b is computed as follows:

djða; bÞ

¼

0 if � Djab � uj

ð�Dj � ujÞ=ðvj � ujÞ if uj5� Djab � vj

1 if � Djab > vj

8><>:

Discordance is null whenever the advantage Djab ispositive or, in case it is negative, when thedisadvantage (i.e. �Djab) does not exceed thecriterion’s discordance threshold uj. The criterionfully disagrees with a S b (it ‘vetoes’ thatconclusion) whenever the disadvantage exceedsits veto threshold vj. This single-criterion discor-dance index varies linearly in between these twothresholds. This variant described here was pro-posed by Mousseau and Dias (2004), who suggestto consider uj=0.25pj+0.75vj in the cases wherethe DM does not want to deal explicitly withparameter uj.

2.1.4. Computation of credibility. The credibilityindex for the assertion a S b is computed by thefollowing expression (Mousseau and Dias, 2004):

sða; bÞ ¼ cða; bÞ � ½1� dmaxða; bÞ� with

dmaxða; bÞ ¼ maxj2f1;...;ng

djða; bÞ

If there exists any criterion such that Djab��vj(i.e. if there exists a veto), s(a,b) becomes null.

IRIS: A DSS FOR MULTIPLE CRITERIA SORTING PROBLEMS 287


3. INTERACTIVE CONSTRUCTION OF ASORTING MODEL

Dias et al. (2002) have proposed an interactiveprocess to help a DM in the progressive construc-tion of an ELECTRE TRI sorting model. It aimsat supporting the DM in setting values for theweights k1; . . . ; kn and the cutting level l, assumingthe DM has already fixed the value of theremaining parameters. Indeed, the weights andcutting level are often considered the most difficultto fix, since they cannot be set independently ofeach other. The process combined the ideas ofparameter inference (Mousseau and Slowinski,1998) and robustness analysis (Dias and Cl!ıımaco,2000), considering there existed no discordance(vj=+1). Later, Mousseau and Dias (2004) haveproposed a variant of the outranking relation thatallowed incorporating the concepts of discordanceand veto, as presented in Section 2.

At a given iteration of the interactive process, letR denote the set of constraints on ðl; k1; . . . ; knÞ asindicated by the DM, defining a set T ofcombinations of parameter values deemed accep-table in that iteration. These constraints can beany linear equalities or inequalities, including forinstance:

* an interval of values for the cutting level l;* an interval of values for each weight kj;* comparisons involving the weight of criteria

coalitions, e.g. k1� k3, k1� k2+k4;* limits to the assignment of actions to categories

(sorting examples), e.g. stating that a1 belongsto category C3 originates (from (2)) theconstraints sða1; b2Þ � l � 0 and l� sða1; b3Þ �e (e is an arbitrary very small positive value thatis necessary because the inequality in (2) isstrict); stating that a2 belongs to categories C3

or C4 originates the constraints sða2; b2Þ � l �0 and l� sða2; b4Þ � e; stating that a3 belongsto category C1 originates only one constraintl� sða3; b1Þ � e; etc. Each of these constraintsis linear, since only the weights and the cuttinglevel are variables (Mousseau and Dias, 2004).

The interactive process may start with an emptyset of constraints, adding one or two constraints atthe time as the DM increases his/her insight on theproblem and becomes increasingly confident abouthis/her judgment. By proceeding in this manner,the constraints will usually define a consistent

system admitting at least one solution. However,the possibility of reaching an inconsistent systemof constraints is also foreseen. The results pre-sented and the analysis allowed will vary accordingto the (in)consistency of the constraints.

3.1. When the constraints define a consistent systemWhenever the set of constraints R defines aconsistent system, there will exist at least onecombination of values for ðl; k1; . . . ; knÞ thatrespects all of R’s constraints. Let T denote theset of such combinations. IRIS is then able tocompute (infer) a combination from T anddetermine the sorting of the actions correspondingto it. Furthermore, it tells the DM how differentthe sorting could be without violating anyconstraint. Hence, the DM may either feelconfident to accept the sorting suggested by IRISor may choose to proceed aiming at reducing set Tby adding new constraints. In the latter case, IRISis also able to help. The complete set of outputsfrom IRIS when R is a consistent system is listednext:

* Using linear programming (for details seeMousseau and Dias, 2004), IRIS computes acombination of parameter values from T thatmaximizes the minimum slack concerning R’sconstraints.

* For each action, IRIS indicates the categorywhere it is sorted according to the inferredparameter values.

* For each action, IRIS shows the range ofcategories where the action could have beensorted without violating any constraint. Thisallows observing which of the proposed sortingresults are more affected by the existingimprecision. It also allows to help the DMchoose sorting examples that do not contradictthe constraints previously stated by him/her.Finally, it also allows drawing robust conclu-sions (i.e. conclusions that hold true for all theacceptable combinations of parameter values),e.g. ‘a1 belongs to category C3 or higher’, or ‘a1reaches, atmost, category C4’.

* For each action! category assignment, IRISmay compute (infer) a combination of para-meter values, if it exists, that leads to thatassignment. The chosen combination maximizesthe minimum slack associated with the assign-ment-related constraints. The DM may then



provide new explicit constraints on the para-meter values, namely when he/she wishes toexclude values leading to extreme assignments(first or last from a range) that he/she considersinadequate.

* Using Monte-Carlo simulation, IRIS estimatesthe relative volume of polyhedron T, which canbe seen as an indicator of the input’s precision(it indicates the proportion of combinationsthat is considered acceptable given the con-straints). IRIS also computes the geometricaverage of the number of categories where eachaction may be assigned to (given the con-straints), which can be seen as an indicator ofthe output’s precision. The main interest ofthese ‘proxy’ indicators (others might as wellhave been included) is to observe how theychange from one iteration to another, ratherthan their absolute value.

3.2. When the constraints define an inconsistentsystemWhenever the set of constraints R defines aninconsistent system, there will exist no combina-tion of values for ðl; k1; . . . ; knÞ that respects all ofR’s constraints (i.e. T=1). IRIS will still infer acombination of parameter values and will showthe sorting corresponding to it, although some ofthe constraints are violated. The analysis shouldfocus on how to remove the inconsistency amongthe set of constraints. The complete set of outputsfrom IRIS when R is an inconsistent system islisted next:

* Using linear programming, IRIS computes acombination of parameter values from T thatminimizes the maximum deviation concerningR’s violated constraints.

* For each action, IRIS indicates the categorywhere it is sorted according to the inferredparameter values, highlighting the sorting ex-amples that were not reproduced.

* For each constraint, IRIS indicates whether it isrespected or violated and, in the latter case,computes the deviation.

* IRIS includes an inconsistency analysis moduleto find subsets of constraints from R that, ifremoved, would render the system consistent.These suggestions are presented by cardinalityorder: first subsets containing only one con-

straint, then subsets containing two constraints,etc. This involves solving a series of 0–1programs (see Mousseau et al., 2003), but thesedetails are hidden from the user. Among thesubsets of constraints suggested by IRIS, theDM should abdicate from one, knowing thatafter removing the constraints in this subsetfrom R, the system will become consistentagain. By not limiting itself to present a singleproposal that would minimize the number ofconstraints to remove, IRIS takes into accountthe possibility that the DM attaches differentpriorities to the constraints.

3.3. Interactive processIRIS proposes a process to construct an ELEC-TRE TRI sorting model, which is interactive to theextent that the results from a given iteration can beused to guide the DM in revising the inputs for thefollowing iteration. This process may start withscarce information, i.e. wide intervals for all thevariables ðl; k1; . . . ; knÞ, and none or few additionalconstraints (including sorting examples). At eachiteration, the DM ought to change the informationprovided minimally, by adding, modifying, ordeleting one or a few constraints at a time. Thefast response times by IRIS allow the DM toobserve immediately the effects of the changes,which will be better understood by making suchsmall incremental steps. This will allow the DM tolearn continuously about the problem at hand andhow the method works, hence allowing him/her toprogress from iteration to iteration.

The purpose of the interactive process is toprogressively reduce the set T of acceptablecombinations for the parameters ðl; k1; . . . ; knÞ,although one cannot expect any kind of conver-gence. Indeed, the DM may proceed by trial anderror, placing constraints that will later beremoved. The DM may stop the process whenhe/she feels that the precision of the outputs (asperceived from the sorting ranges) is satisfactoryto his/her purpose, and he/she feels confident andcomfortable with the constraints imposed on T.

The tangible products of this interactiveprocess are

* a set of sorting examples and other constraintsdefining the acceptable values for the para-meters ðl; k1; . . . ; knÞ;



* a precise combination of values ðl�; k�1; . . . ; k�nÞ,

resulting from the inference program, thatdefines an ELECTRE TRI sorting model;

* a precise category or a range of categories foreach action to sort, which is robust given theinformation provided (i.e. given the acceptableparameter values, assignments outside theseranges are not possible).

However, the most important result from theinteractive process may well be an increasedinsight of the DM on the decision problem facedby him/her, and an increased knowledge about his/her preferences, which may even have changedduring the process.

4. IRIS USER INTERFACE

The IRIS 2.0 software (see Dias and Mousseau,2003) runs under the Microsoft Windows operat-

ing system (version 95 or later). A demonstra-tion version is downloadable at http://www4.fe.uc.pt/lmcdias/iris.htm. The left part of thescreen is associated with inputs, whereas the rightpart of the screen is associated with outputs(Figure 2), and the user may drag the line dividingthese two areas. Each of these areas is organizedaccording to a ‘notebook with multiple tabs’metaphor.

The area on the left is used to edit the inputs,namely,

* the performances of the actions to sort and theindication of sorting examples (Actions page);

* the values for the fixed parameters, which arecategories’ profiles and the indifference, pre-ference, discordance and veto criteria thresh-olds (Fixed Par. page);

* the upper and lower bounds for the variablesðl; k1; . . . ; knÞ (Bounds page) and

Figure 2. Fixed parameters and initial results sorted by variability.



* additional constraints to the value of thesevariables (Constraints page).

The results will reflect changes in the inputs afterthe user instructs IRIS to compute them. The rightarea shows the multiple results, namely,

* sorting ranges, inferred sorting, and inferredparameter values (Results page);

* list of constraints for the inference program andoptimal deviations (Infer. Prog. page) and

* geometric mean of the number of possiblecategories for each action (Indices page).

The menu options (pull-down menu at the top andcontext-sensitive ‘pop-up’ menus) include theusual commands for data editing, file managementand user help. Of particular interest are thecommands to split a category in two, to mergeconsecutive categories, to enable the explicit use ofdiscordance thresholds uj (if this option is off, IRISconsiders uj=0.25pj+0.75vj) and to compute thevolume of the polytope formed by the combina-tions of parameter values that respect all theconstraints. Data can be stored and retrieved astext files, allowing simple processes for thetransference of data tables from word processorsor spreadsheets.

5. AN ILLUSTRATIVE EXAMPLE

In order to illustrate the use of IRIS, we will followin this section the steps of a hypothetical DM,considering data from a problem presented in Diaset al. (2000), which was adapted from an applica-tion of ELECTRE TRI to the banking sectorpresented in Dimitras et al. (1995). The data differfrom those presented in Dias et al. (2000) only inthe use of a veto threshold in one of the criteria(left part of Figure 2), whereas the cited paperconsidered no veto.

The decision problem consists in sorting 40actions (each one representing a company) intocategories of bankruptcy risk: very high (C1), high(C2), medium (C3), low (C4) or very low (C5).These five categories are ordered from worst (C1)to best (C5). The left part of Figure 2 presents theprofiles that divide the categories (their perfor-mances and fixed thresholds on seven criteria).Each of the actions to be sorted was evaluated inseven criteria (left part of Figure 3).

5.1. First iterationThe variables in this problem are the parametersthe DM needs to set: the cutting level and weightsðl; k1; . . . ; k7Þ. Initially, the DM placed wideintervals in the Bounds page: she placed kj 2

Figure 3. Results after introducing one sorting example (iteration 2).



½0:01; 0:49� ðj ¼ 1; . . . ; 7Þ (according to theory,these weights must be positive and no criterionshould weigh more than 0.5), and she placed l 2½0:6; 0:99� (according to theory, the cutting levelmay vary between 0.5 and 1.0). Additionally, theDM informed that the second criterion was theone with the highest weight. Therefore, theconstraints k2� k1, k2� k3, k2� k4, k2� k5,k2� k6 and k2� k7 were inserted in page Con-straints. No sorting example was considered at thisstage.

The right part of Figure 2 illustrates the resultscorresponding to this (relatively scarce) informa-tion. Since there are many combinations of valuesfor ðl; k1; . . . ; k7Þ that satisfy the few constraintsthat were introduced, IRIS shows for each action arange of categories where it may be sorted withoutviolating any constraint. This example (whichshows the ranges sorted by variability) presentsthe curiosity of showing that action a28 cannot beassigned to category C2, although it could beassigned to categories C1 or C3 (for a character-ization of such situations see Dias et al., 2000).Among the categories in each range, a darkercolour is used to identify the sorting proposed byIRIS based on the inferred values forðl; k1; . . . ; knÞ, which are shown in the last line ofthe Results page. In the line immediately above it,IRIS presents the combination of values for theseparameters that corresponds to the selected cell(Figure 2 shows a situation where the selected cellcorresponds to the assignment of a28 into C4). Thelatter combination of parameter values changesimmediately when the user selects a different cell.

Given the current information, the Indices pagewould state that the (geometric) average numberof possible categories per action was 2.249, whichis an indicator of the precision from the perspec-tive of outputs. The proportion of combinationsrespecting all the bounds that also respect theadditional constraints (given by the menu com-mand Volume computation) is 14.3%, which is anindicator of the precision from the perspective ofinputs. The DM may monitor how these figureschange from one iteration to the next one.

5.2. Second and third iterationsLet us now imagine that the DM felt confidentthat action a28 was not worse than category C4.Hence, she inserts in page Actions this assignmentexample, which is an interval-type one since itadmits two categories: C4 and C5. The results aredepicted in Figure 3. The Indices would state that

the (geometric) average number of possible cate-gories per action is now 1.518, whereas the resultof the Volume computation command drops to0.9%.

Next, in a third iteration, the DM decides toindicate that action a31 is a good representative forthe worst category (C1), and that action a3 shouldonly be allowed into categories C3 or C4. Theresults are depicted in Figure 4. The result ofthe Volume computation command decreased to0.6%, indicating a further reduction of the setof combinations of parameter values respectingall the constraints. A visible consequence ofthis reduction is, for instance, the fact thatactions a10 and a11 can no longer be assigned tocategory C5.

5.3. Fourth and fifth iterationSo far, the constraints introduced by the DM didnot contradict the constraints previously intro-duced: the system of constraints remained con-sistent. However, sometimes inconsistency isintroduced inadvertently (e.g. when placing severalconstraints at a time) or to question previouslyintroduced constraints. For instance, the DM maywonder why action a6 cannot be sorted intocategory C3 or lower. By indicating this sortingexample, the system becomes inconsistent, sincethere is no combination of values for ðl; k1; . . . ; knÞcapable of satisfying all the constraints. In thissituation, IRIS shows an inferred combination ofparameter values that violates one or moreconstraints by the least possible deviation, andindicates the sorting corresponding to it, high-lighting the sorting examples that were notreproduced (Figure 5).

In these circumstances, the DM must relax orremove constraints. The inconsistency analysismodule tells her which possibilities exist to renderthe system consistent (Figure 6): either theconstraint number 3 is removed (this is theconstraint just inserted, which caused the incon-sistency) or the constraints number 1, 4 and 13 areremoved. Constraints 1 and 4 correspond tosorting examples introduced by the DM, butconstraint 13 corresponds to the cutting level’supper bound, currently set to 0.99. Since thisbound is already rather high, the DM mustconform to remove the constraint of sorting a6 ina category lower than C4, returning to thesituation depicted in Figure 4.

In a subsequent iteration (the fifth), the DMwonders why action a9 cannot be sorted into



category C3 or lower. By indicating this sortingexample, the system becomes inconsistent again,and the inconsistency analysis module helps theDM decide as to how to restore its consistency. Inthis case, there are again two possibilities: eitherremove the constraint concerning a9 or remove theconstraint stating a28 belongs to C4 or higher. TheDM feels stronger about the constraint concerninga9, and hence decides to drop the other constraint.As a matter of fact, she relaxes the constraintstating that a28 belongs to C3 or higher, to see ifthis adjustment suffices. The system actuallybecomes consistent and the results correspondingto it are depicted in Figure 7.

We will stop here our illustration of the use ofIRIS. At this stage, 27 actions are sorted into asingle category, whereas each of the remaining 13actions can be assigned at most into two cate-gories. The DM may be satisfied with these results,or she might want to continue either addinginformation or revising her previous judgment.

6. APPLICATIONS OF IRIS IN REAL-WORLD DECISION PROBLEMS

In this section, we provide two illustrations of real-world decision problems in which the IRIS soft-ware has been playing a significant role in themodelling process. These are ongoing applicationswhose detailed description lies outside the scope ofthis paper. Both applications involve the definitionof a multiple criteria sorting model using theELECTRE TRI method.

6.1. Deciding whether to answer to a call for tender(CfT) at EADSEADS Launch Vehicles (EADS-LV) developedand since 1996 has used an ELECTRE TRI modelto support the decision concerning whether toanswer to a CfT (see Collette and Siarry, 2002,Chapter 12). For EADS-LV, answering to a CfTimplies spending some time conceiving a commer-cial offer; this generates a cost that will be

Figure 4. Results after introducing two additional sorting examples (iteration 3).



integrated in the contract in case of success, or beconsidered as a loss in unsuccessful cases. Choos-ing not to answer to a CfT will neither generategains nor losses.

The model developed aims at sorting CfT intofour ordered categories (‘Yes’, ‘Undecided yes’,‘Undecided no’, ‘No’). The ‘Yes’ category (the‘No’ category, respectively) groups CfT for whichEADS-LV is (is not, respectively) in position ofsuccess. The ‘Undecided yes’ and ‘Undecided no’categories correspond to weaker statements. It isused weekly by a committee in order to make adecision for the current CfT. Since 1996, a largedatabase of more than 400CfT has been collectedincluding, for each CfT, the evaluation vector, theassignment provided by ELECTRE TRI, thedecision made by the committee (either answeror not to the CfT) and the actual output of the CfT(success or loss).

An important issue for EADS is to checkwhether the ELECTRE TRI model has a goodpredictive ability concerning the result of the CfT(success or not). Therefore, the model is to beregularly updated in order that the output of theELECTRE TRI model becomes as close aspossible to the actual output of the CfT on the

past data. More precisely, CfT sorted in thecategories ‘Yes’ or ‘Undecided yes’ should leadto a success while CfT sorted in the categories ‘No’or ‘Undecided no’ should lead to a loss.

In order to update the ELECTRE TRI modelconcerning the weights of criteria, IRIS is aefficient tool to account for past data. Each pastCfT can induce an assignment example in such away that successful CfT should be assigned at leastto the category ‘Undecided yes’ and unsuccessfulCfT should be assigned at most to the category‘Undecided no’. The following fictitious exampleillustrates the way these assignment examples aredesigned. The assignment examples a1, a4, a5 donot contradict the original assignment model, buta2 and a3 provide a situation that should berestored by the inferred model (Table I).

6.2. Integrating multiple expert opinions into asingle sorting modelThis application concerns the public transportpricing in the Paris region (Ile de France). STIF isthe organizing authority in charge of operatingpublic transport networks and deciding on ticketprices and price structures.

Figure 5. A new constraint makes the system inconsistent (fourth iteration).



STIF is willing to modify the pricing of publictransport in Ile de France. With the support ofLAMSADE, STIF has been developing a metho-dology (see Mousseau et al., 2001) aiming atdefining alternative pricing strategies, in coopera-tion with the involved stakeholders. Strategies aregrounded on the partition of Ile de France intozones, the price being defined from a zone toanother on the basis of the distance and the qualityof the public transport in the origin and destina-tion zones. Therefore, a model specifying thequality of the public transport offer is required.

A multiple criteria sorting model based on theELECTRE TRI method has been built to assign aqualitative level for the quality of the public

transport offer (four qualitative levels C1: ‘verylow offer’, C2: ‘fairly low offer’, C3: ‘fairly highoffer’ and C4: ‘very high offer’). The 11 criteriaconsidered correspond to the different aspects(number of stations, frequency, accessibility, etc.)of the public transport network. Experts couldeasily define the limits of categories (and corre-sponding thresholds), but weighting directly thecriteria has been considered as a difficult task, andtherefore an indirect procedure has been proposed.

Based on their knowledge, seven experts haverated the level of quality of 58 zones according totheir evaluations on the criteria. Obviously, therating (C1, C2, C3 or C4) provided by an expert fora specific zone corresponds to an assignment

Figure 6. Inconsistency analysis module (fourth iteration).

Table I. Assignment examples induced by 5 past CfT

Name Original model assignment Actual result Minimum category Maximum category

a1 ‘Undecided yes’ Successful ‘Undecided yes’ ‘Yes’

a2 ‘Undecided yes’ Unsuccessful ‘No’ ‘Undecided no’

a3 ‘Undecided no’ Successful ‘Undecided yes’ ‘Yes’

a4 ‘Yes’ Successful ‘Undecided yes’ ‘Yes’

a5 ‘No’ Unsuccessful ‘No’ ‘Undecided no’



example. Within this context, IRIS has proved tobe a very efficient tool analysing the informationcontained in the 406 (58� 7) assignment examplesprovided by the experts.

First, IRIS made it possible to check forinconsistencies in the rating of the 58 zones foreach expert individually. Most experts did notprovide ratings perfectly compatible with ELEC-TRE TRI; hence, it was possible to determine theminimal subset of assignment examples that whenremoved lead to an information compatible withELECTRE TRI. This analysis led each expert todefine ‘reasonable’ values for his/her criteriaweights.

Second, an important issue to integrate multipleexpert judgments into a single sorting model is theanalysis of the compatibility among judgmentsprovided by coalitions of experts. To proceed tosuch analysis, IRIS was used considering theassignment examples provided by subsets of nexperts (i.e. n*58 examples), and computing theminimal subset of assignment examples that whenremoved lead to a consistent information. Thesmaller this subset is, the more compatible the

expert judgments are. Moreover, for each coalitionof experts, a weight vector was computed (usingthe inference procedure available in IRIS).

Third, a Delphi-like iterative procedure wasused to facilitate ‘convergence’ of assignmentsexamples provided by the seven experts. Threeiterations were performed in which experts wereasked to reconsider their opinion on the zones forwhich no unanimity arose at the precedingiteration (a feedback on the preceding iterationwas provided). Within this iterative context, IRISproved to be very powerful in analysing dynami-cally the evolution of coalitions of experts and howa consensus arises among experts.

7. CONCLUSIONS

IRIS was built to support DMs facing sortingproblems. It does not apply to situations where theDM wishes to automatically sort the actions basedon their characteristics exclusively. Rather, itapplies to situations where the preferences of theDM}as well as the characteristics of the ac-

Figure 7. Results after relaxing the sorting example concerning a28.



tions}will yield a partition of the set of actionsinto a set of ordered categories defined before-hand. From another perspective, it applies tosituations where the DM wishes to attribute gradesto actions, defining the grades beforehand andgrading the actions independently of each other.The important issue to stress is that the subjectivevalues of the DM will influence the outcome of thesorting decisions.

The main advantage of IRIS is the support itmay offer to DMs that do not entirely know theirpreferences, or that do not know how to quantifythese preferences taking into account the meaningof ELECTRE TRI’s parameters and using precisenumbers. By accepting imprecise information (i.e.intervals or other constraints, including sortingexamples), IRIS integrates the inference of para-meter values with the search for conclusions thatare valid despite the imprecision (the robustconclusions). It facilitates an interactive processthat fosters self-learning and the progressivedelimitation of the input and output variability.

The main shortcoming of IRIS is not to considerall the parameters as variables, leaving the DMunsupported to set the value of the categoryprofiles and criteria thresholds (indifference, pre-ference, discordance and veto). However, theparameters it considers as variable (weights andthe cutting level) are arguably the most difficult toset, given their interdependence across the criteria(whereas the remaining parameters require onlysingle-criterion judgments).

One of the paths to pursue in future research isto study precisely how one can solve efficientlyinference problems where more parameters areallowed to vary. Another ongoing path is toimprove the inconsistency analysis module so thatit may propose to the DM different forms ofrelaxing (rather than removing) constraints, pos-sibly taking into account different priority levelsattached to constraints. Finally, real world appli-cations such as the ones described in the previoussection will surely cause new challenges to beaddressed in future versions of IRIS.

As a final remark, we wish to emphasize theimportance of interactivity for DSS. Interactivity,as noted by Courbon (Courbon et al., 1994),should not be seen as a last minute concern, but asa conception tool for DSSs. In a decision processsupported by IRIS, the model is successivelyshaped in an interactive manner. In a way, assuggested by Courbon in the cited paper, a DSSmay help a DM (since, in our case, it explores the

model’s imprecision and extracts conclusions), butit may also be helped by the DM (to the extentthat, in our case, he/she continuously monitorsand delimits the model’s imprecision).

ACKNOWLEDGEMENTS

The methodology IRIS is based on was devel-oped in the framework of two Portuguese–Frenchcooperation projects financed by ICCTI and theFrench Embassy in Portugal (ref. 328J4 and500B4). The software specification was performedby Lu!ııs Dias and Vincent Mousseau, and thesoftware implementation was conducted by LuisDias, Carlos Gomes da Silva and Rui Louren-cco.

The authors thank the referees for their sugges-tions in amending the paper.

REFERENCES

Collette Y, Siarry P. 2002. Optimisation multiobjectif.Eyrolles: Paris.

Courbon JC, Dubois D, Roy B, Pomerol JC. 1994.Autour de l’aide a la d!eecision et de l’intelligenceartificielle. Cahiers du LAFORIA 94/01, Institut BlaisePascal: Paris.

Dias L, Mousseau V. 2003. IRIS}interactive robustnessanalysis and parameter’s inference for multiplecriteria sorting problems (version 2.0).}User Manual,Document of INESC Coimbra, 1/2003, available athttp://www4.fe.uc.pt/lmcdias/IrisMan2.pdf

Dias L, Mousseau V, Figueira J, Cl!ıımaco J. 2000. Anaggregation/disaggregation approach to obtain robustconclusions with ELECTRE TRI. Cahier du LAM-SADE, No. 174, Universit!ee Paris-Dauphine.

Dias L, Mousseau V, Figueira J, Cl!ıımaco J. 2002. Anaggregation/disaggregation approach to obtain robustconclusions with ELECTRE TRI. European Journalof Operational Research 138: 332–348.

Dias LC, Cl!ıımaco JN. 2000. ELECTRE TRI for groupswith imprecise information on parameter values.Group Decision and Negotiation 9: 355–377.

Dimitras A, Zopounidis C, Hurson C. 1995. Multi-criteria decision aid method for the assessment ofbusiness failure risk. Foundations of Computing andDecision Sciences 20(2): 99–112.

Greco S, Matarazzo B, Slowinski R. 2001. Rough setstheory for multicriteria decision analysis. EuropeanJournal of Operational Research 129: 1–47.

Mousseau V, Dias L. 2004. Valued outranking relationsin ELECTRE providing manageable disaggregationprocedures. European Journal of Operational Research156(2): 467–482.



Mousseau V, Figueira J, Dias L, Cl!ıımaco J, Gomes daSilva C. 2003. Resolving inconsistencies amongconstraints on the parameters of an MCDAmodel. European Journal of Operational Research143: 332–348.

Mousseau V, Roy B, Sommerlatt I. 2001. Developmentof a decision aiding tool for the evolution of publictransport ticket pricing in the Paris region. In A-MCD-A Aide Multicrit "eere "aa la D !eecision}MultipleCriteria Decision Aiding, Paruccini M, Colorni A, RoyB (eds). Joint Research Center, European Commis-sion: Luxembourg; 213–230.

Mousseau V, Slowinski R. 1998. Inferring an ELEC-TRE TRI model from assignment examples. Journalof Global Optimization 12: 157–174.

Mousseau V, Slowinski R, Zielniewicz P. 1999. Auser-oriented implementation of the ELECTRE-TRI

method integrating preference elicitationsupport. Computers & Operations Research 27:757–777.

Roy B. 1991. The outranking approach and thefoundations of ELECTRE methods. Theory andDecision 31: 49–73.

Roy B. 1996. Multicriteria Methodology for DecisionAnalysis. Kluwer: Dordrecht.

Roy B, Bouyssou D. 1993. Aide multicrit "eere "aa lad !eecision: m !eethodes et cas. Economica: Paris.

Yu W. 1992. ELECTRE TRI. Aspects m!eethodologiqueset guide d’utilisation. Document du LAMSADE, No.74, Universit!ee Paris-Dauphine.

Zopounidis C, Doumpos D. 2002. Multicriteriaclassification and sorting problems: a literature re-view. European Journal of Operational Research 138:229–246.



iris: a dss for multiple criteria sorting problems€¦ · the main feature of iris is that it does...

Documents