a performance appraisal and promotion ranking system based on fuzzy logic

8/12/2019 A Performance Appraisal and Promotion Ranking System Based on Fuzzy Logic

http://slidepdf.com/reader/full/a-performance-appraisal-and-promotion-ranking-system-based-on-fuzzy-logic 1/8

A performance appraisal and promotion ranking system based on fuzzy logic:

An implementation case in military organizations

Chiung Moon a,*, Joosung Lee b,*, Siyeong Lim c

a Gueulri Advanced Technology, Ansan, Republic of Koreab Department of Information and Industrial Engineering, Yonsei University, Seoul, Republic of Koreac National Infrastructure & GIS Research Division, KRIHS, Gyeonggi-do, Republic of Korea

1. Introduction

The shift to knowledge-based capitalism makes it critical for all

organizations to maintain talented knowledge workers. It is

important for them to find and promote the most qualified

candidates because superior human talent becomes the prime

source of an organization’s competitive advantage [1,2].

When managing the human resources of an organization,

appraising the performance of applicants for a particular position

isacentraltask [3].However,itisoftendifficulttoassignanaggregate

scoreforacandidate’sperformancewhenpreviousassessmentswere

qualitative and originated from other organizations that have

different performance evaluation criteria [4,5]. For example, college

admissions offices review applications that come from diverse

schools, while corporate headquarters review applicants that come

from different work environments. Although, the applicants’ records

may include quantitative measures such as school grades, standard

entrance exam scores, and foreign language proficiency levels, they

also often contain very qualitative descriptions of applicants such as

‘‘social’’, ‘‘hard-working’’ or ‘‘creative’’.The difficulty is to objectively

combine quantitative and qualitative evaluations of applicants to

determine their acceptability to the organization.

In military organizations, transparent and fair appraisal of

personnel is essential for decisions pertaining to promotions and

operations. For an appraisal system to be effective, organizational

members must believe that their opinions are reflected in the

appraisal process [6]. Such appraisal involves a number of

evaluators (or decision makers) with equal authority to assess

each candidate based on both qualitative and quantitative multi-

performance criteria. The impacts and the relationships among

the characteristics used to assign a score can sometimes be

described by linguistic terms, e.g. ‘‘very high’’, ‘‘poor’’, ‘‘medium’’,

etc. The appraisal results are then aggregated to rank order the

performance of the candidates and select the finalists to be

promoted. Note that the candidates come from various military

organizations and have expertise in their specialized fields. Their

performance scores have been recorded at their respective past

organizations.

This paper introduces a methodology that uses ‘fuzzy set

theory’ and ‘electronic nominal group technique’ for multi-criteria

evaluation in the group decision making of military promotion

screening. The methodology makes it possible to rank order the

performance of candidates evaluated by multiple criteria. The

nominal group technique is a structured group decision-making

process for generating ideas, identifying problems, and providing a

prioritized list of ideas through voting by group members. This

ensures equal participation from all group members and allows

managers to use analytical procedures in thefinal decision making.

Applied Soft Computing 10 (2010) 512–519

A R T I C L E I N F O

Article history:Received 13 February 2007

Received in revised form 25 July 2009

Accepted 23 August 2009

Available online 3 September 2009

Keywords:

Performance appraisal

Fuzzy theory

Military promotion screening

Ranking system

Group decision making

A B S T R A C T

Systematic performance appraisal and ranking of candidates applying for promotion is important instrategic human resource management. This paper discussesan approach for the promotion screeningof

candidates applying fora particularcommission in a military organization. The approachusesa fuzzy set

theory and electronic nominal group technique for ranking decisions fairly through the multi-criteria

performance appraisal process. A new ranking procedure considering the metric distance and fuzzy

mean value is proposed, which makes it possible to rank order the performance of the candidates by

aggregating the scores from each evaluator. A new system for performance appraisal and promotion

ranking is also developed. The system has a monitoring function which utilizes performance evaluation

data without abnormal evaluation data, which could occur when a particular evaluator produces an

incorrect result. The system was applied to a military organization in Korea. Theresultsof example show

that thesystematic approach of thefuzzyprocedure is an effectivemethod fortransparent andimpartial

multi-criteria performance evaluation.

Crown Copyright 2009 Published by Elsevier B.V. All rights reserved.

* Corresponding authors.

E-mail addresses: [email protected] (C. Moon), [email protected] (J. Lee).

Contents lists available at ScienceDirect

Applied Soft Computing

j o u r n a l h o m e p a g e : w w w . e l s e v i e r . c o m / l o c a t e / a s o c

1568-4946/$ – see front matter. Crown Copyright 2009 Published by Elsevier B.V. All rights reserved.

doi:10.1016/j.asoc.2009.08.035

mailto:[email protected]


http://www.sciencedirect.com/science/journal/15684946

http://dx.doi.org/10.1016/j.asoc.2009.08.035

http://dx.doi.org/10.1016/j.asoc.2009.08.035

http://www.sciencedirect.com/science/journal/15684946





The steps in nominal group technique are composed of (1)

generating ideas regarding the appraisal problem, (2) recording

ideas from group members, (3) discussing each idea for evaluation,

(4) rating and ranking the ideas, and (5) priority ordering of the

alternatives based on voting and analytical methods. This

technique has been successfully applied to a number of facilitating

group or group decision-making problems [7–9] since it was

suggested by Delbecq and Van de Ven [10].

To assign ranks, we also propose a new procedure that uses ‘the

metric distance’ and‘the fuzzy mean value’ concurrently. There are

various ranking methods available from a number of previous

research articles [11–19].

Lee and Li [20] introduced a ranking method that utilizes the

mean and variance of fuzzy numbers. Candidates, who attain

scores with both highermean values andlower spread, rank higher

using their procedure. In decision-making process, human intui-

tion favors fuzzy numbers with a higher mean value and lower

spread. However, when scores with higher mean values and higher

spreads, or with lower mean value and lower spreads exist, it is

difficult to compare the orders of the scores directly. In order to

resolve this difficulty, Cheng [21] proposed a ranking method that

uses coefficient of variation (CV). However, this method has a

limitation: the difference between CVs could be negligible when

evaluation-scores are normalized.In this paper, we borrow the concept of TOPSIS (the Technique

for Order Preference by Similarity to Ideal Solution [22]) for

assignment the rank. The point is that the best solution should be

closest to the positive-ideal solution and farthest from the

negative-ideal solution. The positive (negative)-ideal solution is

the collection of the best (worst) scores among all candidates’

scores from each criterion in the original meaning. However, in

promotion screening systems, it is not important to achieve higher

scores than other candidates. It is important only to garner enough

points for promotion. Thus, a collection of the maximum

(minimum) scores, the defined limits for the candidate in each

criterion, are chosen as the positive (negative)-ideal solution.

Theremainder of this paper is organizedas follows. In Section 2,

we describethe process of promotion screening. A newfuzzybasedapproach is proposed in Section 3, followed by the performance

appraisal andranking systemin Section 4. Numerical examples are

given in Section 5, and conclusions are presented in Section 6.

2. Promotion screening process

Performance appraisal for promotion in the military is typically

conducted for number of officers (candidates) from independent

organizations. These organizations are characterized by the need

for specialized personnel in policy making, planning, acquisition

andthe like. There aredifferences among theevaluation systems of

the army, air force and navy. Since the metrics and processes of

measuring performance differ among the three organizations,

military headquarters typically combine quantitative and quali-tative performance scores from the past and present records of the

personnel.

In order to combine such mixed performance scores, human

resource department first develop evaluation criteria and establish

relative weighting among them. For example, performance indices

such as past position/education, awards, and organization con-

tribution are assigned to experience, job expertise, and miscella-

neous categories in Table 1. Based on the metrics and relative

weighting, the headquarters selects a group of evaluators to

conduct promotion appraisal.

When the evaluation indices are determined, brainstorming,

nominal group technique or Delphi methods are typically

implemented. Nominal group technique provides a useful way

to generate ideas, prioritize them, and come to a group consensus

among organizational members. In order to use these techniques

for sensitive decision-making processes, as in performance

appraisals of military personnel, it is necessary to collect various

performance evaluation criteria at different organizational levels,

and analyze the relative importance of each. This is to prevent anyparticular organization’s self-interests from dominating the

selection and weighting of the performance evaluation criteria.

Finally, the finalists to be promoted are selected.

The selection of evaluators and aggregation of individual

evaluators’ appraisal results influence the final scores of perfor-

mance evaluation. Problems could occur when there is an overly

influential member in the evaluators group, or when a particular

evaluator assigns evaluation results that are too high or low

compared to the average scores. To avoid such problems, U.S.

military organizations often exclude the maximum and minimum

scores from the final performance evaluation results [23].

However, this method requires a large number of evaluators.

Also, it is not clear how to remove data points when multiple

evaluators ascribe identical maximum or minimum scores.Therefore, an improved approach is necessary, one that uses all

performance evaluation data without removal.

3. Fuzzy methods

In this section, definitions of fuzzy set theory and linguistic

variables as described by Zimmermann are reviewed (2001). Then,

a ranking method that simultaneously considers the metric

distance and fuzzy mean value is proposed. The distance from

the ideal solution and the fuzzy mean value are usual criteria for

ranking fuzzy numbers. Importantly, however, a shorter distance

does not always mean a larger mean value. If the score of one

candidate has a shorter distance from the positive-ideal solution,

but a lower mean value than the other candidate, which candidateis better? This illustrates the importance off a ranking method that

can consider both criteria simultaneously.

3.1. Fuzzy number and linguistic variable

Definition 1. If X is a collection of objects denoted generically by x,

then a fuzzy set ˜ A in X is a set of ordered pairs:

˜ A ¼ fð x;m ˜ Að xÞÞj x 2 X g

m ˜ Að xÞ is the membership function or grade of membership of x in ˜ A

that maps X to themembership space M (when M contains only the

two points 0 and 1, ˜ A is nonfuzzy and m ˜ Að xÞ is identical to the

characteristic function of a nonfuzzy set). The range of the

Table 1

Key performance criteria for promotion screening.

Key perfor man ce criter ia Per formance indices

Service rating 1. Completeness of job objectives

2. Service rating on job assignment

Multi-area aptitude 1. Creativity

2. Organizational contribution

3. Management capability

4. Achievement

5. Job expertise6. Teamwork

Growth potential 1. Planning

2. Communication

3. Group discussion

4. Foreign language

5. Information systems usage

Innovativeness 1. Innovation score

C. Moon et al./ Applied Soft Computing 10 (2010) 512–519 513



membership function is a subset of the nonnegative real numbers

whose supremum is finite. Elements with a membership of zero

degrees are normally not listed.

Definition 2. The (crisp) set of elements that belong to the fuzzy

set ˜ A at least to the degree a is called the a-level set :

Aa ¼ f x 2 X jm ˜ Að xÞ ag

A0

a ¼ f x 2 X jm

˜ Að xÞ >ag is called ‘‘strong a-level set’’ or ‘‘strong a-

cut’’.

Definition 3. A fuzzy number ˜ M is a convex normalized fuzzy set ˜ M

of the real line R such that

1. It exists exactly one xo 2 R with m ˜ M ð x0Þ ¼ 1.

2. m ˜ M ð x0Þ is piecewise continuous.

In this paper, trapezoidal fuzzy numbers are used. A fuzzy

number ˜ M can bedefined as (a, b, c , d) asshownin Fig.1. In addition

its membership function is defined as in Eq. (1).

m ˜ M ¼

0; x<a x a

b a

; a x b

1; b x c x d

c d; c x d

0; x>d

8>>>>>><>>>>>>:

(1)

For example, a fuzzy number, which could be ‘‘approximately

5’’, would normally be defined as the quadruple (3, 4, 6, 7). If b = C

in a fuzzy number ˜ M ¼ ða; b; c ; dÞ, then ˜ M is calleda triangular fuzzy

number. Additionally, a nonfuzzy number k can be expressed as (k,

k, k).

Thebasic operations of thefuzzynumbers used in this paper are

defined as follows:

˜ M 1 ˜ M 2 ¼ ðm1l þ m2l;m1lm þ m2lm; m1um þ m2um; m1u þ m2uÞ (2)

˜ M 1 ˜ M 2 ¼ ðm1l m2l;m1lm m2lm; m1um m2umm1u m2uÞ (3)

˜ M 1 ¼ ðm1u; m1um;m1lm; m1lÞ (4)

1

˜ M 1¼

1

m1u;

1

m1um;

1

m1lm

;1

m1l

(5)

where ˜ M 1 ¼ ðm1l;m1lm; m1um; m1uÞ and ˜ M 2 ¼ ðm2l;m2lm; m2um;m2uÞ

represent two trapezoidal fuzzy numbers with lower, lowermodal,

upper modal and upper values.

Definition 4. A linguistic variable is characterized by a quintuple

ð x;T ð xÞ; U ;G; ˜ S Þ in which x is the name of the variable; T ( x) (or

simple T ) denotes the term set of x, i.e., the set ofnamesof linguistic

values of x, with each value being a fuzzy variable denoted gen-

erically by X and ranging over a universe of discourse U that is

associated with the base variable u; G is a syntactic rule (which

usually hasthe form of a grammar) forgeneration of thename, X , of

values of x; and ˜ S is a semantic rule for associating with each X its

meaning, ˜ S ð xÞ, which is a fuzzy subset of U .

Typical values of cardinality used in the linguistic models are

odd ones, such as 7 or 9, with an upper limit of granularity of 11 or

no more than 13, where the midterm represents an assessment of

‘‘middle value’’, with the rest of the terms being placed

symmetrically around it [24].

In this paper, the candidates are scored by the linguistic

variables. Accordingly, we propose that a set of seven terms, T ,

could be given as follows:

T ðscoredÞ ¼ fVG; G; MG; M ; MB; B; VBg

The following semantics are proposed for the set of seven terms

like those in Fig. 2.

VG ¼ VeryGood ¼ ð8;9;10;10Þ

G ¼ Good ¼ ð7;8;8;9ÞMG ¼ MediumGood ¼ ð5;6;7;8ÞM ¼ Medium ¼ ð4;5;5;6ÞMB ¼ Medium Bad ¼ ð2;3;4;5ÞB ¼ Bad ¼ ð1;2;3ÞVB ¼ VeryBad ¼ ð0;0;1;2Þ

The membership function of ‘‘VG’’ is

mverygood ¼

0; x<8 x 8; 8 x 91; 9 x 100; x>10

8>><>>: (6)

Each linguistic variable has its membership function like that in

Eq. (6).

3.2. Ranking method

In this situation, candidates are scored by linguistic variables. It

is assumed that there are m candidates, n evaluators and l criteria.

Each of n evaluators classifies into grade m candidates for each of l

criteria, respectively. The scores range between 0 and 10 in fuzzy

concept.

We define the following sets:

m candidates; f Aij1 i mgn evaluators; fDij1 i ngl criteria; fC ij1 i lg

Fig. 1. Trapezoidal fuzzy number ˜ M . Fig. 2. The membership functions for fuzzy numbers.

C. Moon et al./ Applied Soft Computing 10 (2010) 512–519514



Let xijk (1 i m, 1 j n, 1 k l) be the ith candidate’s

grade classified by jth evaluator for k criterion. For example, if the

first candidate receives a grade of ‘G’ from the second evaluator for

the third criterion, then x123 should be (7, 8, 8, 9). There are three

procedures to rank the candidates. Thefirst is to get theweights for

criteria, and the second is to aggregate the evaluators’ scores for

each candidate. The last is to rank the candidates by our proposed

ranking method.

(1) Weighting for criteria

When decision-making problems arise, there exist two

kinds of ‘weighting’, one for the criteria and the other for the

evaluators. In general decision-making situations, opinions of

some evaluators such as chief are more important than those of

the others. But in our systems, the weighting of evaluators is

equal because the effects of evaluators should be fair. Thus,

only weighting for performance criteria is taken into con-

sideration.

Let wkð1 k lÞ be the weights for each criterion. These are

predetermined by the experts using the intranet/internet. The

weights are fuzzy numbers such as the scores for candidates.

‘‘High(Low)’’ is usedin place of‘‘Good/Bad’’.For example, a fuzzy

number, which could be assigned ‘‘VH (Very High)’’ to a

criterion, would normally be defined as the quadruple (8, 9,

10, 10). We can derive a normalized weight such that

¯ wk ¼ wkPl

i¼1 wi¼

wklPl

i¼1 wiu

;wk

lmPli¼1 wi

um

;wk

umPli¼1 wi

lm

;wk

uPli¼1 wi

l

! (7)

where, wi ¼ ðwil; wi

lm;wium;wi

uÞ represents a trapezoidal fuzzy

number with lower, lower modal, upper modal and upper

values.

(2) Aggregating the scores

The fuzzy number yt (1 i m) is defined as the aggregated

score for the ith candidate. Then, yt is obtained as follows.

where xijk = ( xijk,l, xijk,lm, xijk,um, xijk,u) represents a trapezoidalfuzzy numberwith lower,lowermodal, upper modal andupper

values. yi = ( yi,l, yi,lm, yi,um, yi,u) denotes the unified score for the

ith candidate from all evaluators by all criteria.

According to Boender et al. [25], the normalized set of fuzzy

numbers has to satisfy the conditions that the sum of the

middle values is 1 and the sum of the products of the low and

high values is 1. The first term in summation satisfies these

conditions. The second term in the summation signifies the

average of ratings from all evaluators, and corresponds to the

objective, which is a fair screening.

(3) Ranking the candidates

From now on, the candidates will be ranked by the aggregated

scores, yis. Thus, one who scores ‘‘Very Good (Very Bad)’’ in all

criteria is chosen as the positive (negative)-ideal solution, even if

none of the real candidates do so in all criteria. The value of the

positive (negative)-ideal solution is dependent on the weights for

criteria.

In Goetschel and Voxman [26], the method of ranking the fuzzy

numbers is found using their means. The ordering method is

defined as such that if

m ¼Z 1

0 a½aðaÞ þ bðaÞda; v

¼Z 10a½

c ðaÞ þ

dðaÞ

da; (9)

and m v, then ˜ M 1 ˜ M 2, where (a(a), b(a)) and (c (a), d(a))

denote the a-cut of the fuzzy number ˜ M 1, ˜ M 2.

Let y* = (a*, b*, c *, d*), y = (a, b, c , d), and yi = (ai, bi, c i, di) be

the positive-ideal solution, negative-ideal solution, and ith

candidate’s score, respectively. Also, let m*, m, and mi be the

mean values of the positive-ideal solution, negative-ideal solution,

and ith candidate.

We define the following distance measurements:

d

¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiðaÞ2 þ ðb

Þ

2þ ðc Þ2 þ ðd

Þ

2q

(10)

d

¼ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiðaÞ2 þ ðb

Þ2

þ ðc Þ2 þ ðd

Þ2q (11)

di ¼ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

a2i þ b2

i þ c 2

i þ d2

i

q (12)

Using Eqs. (9)–(12), we suggest the following ranking measures.

M i ¼ 1

2

di d

d

d þ

m i m

m m

(13)

where d di d* and m mi m*.

Thevaluesof M i are between 0 and 1. If the value of M i is larger,

the candidate is closer to the ideal solution and farther from the

negative solution. In (13), each candidate’s scores are simulta-

neously compared with the positive (negative)-ideal solution in

terms of the distance and fuzzy mean. The greater the distancefrom the negative-ideal solution and the larger the fuzzy mean

value compared to the negative-ideal solution, the higher the

candidate ranks.

Eq. (13) is the mixture of the ranking methods in Hwang and

Yoon [22] and Goetshel and Voxman [26]. The first term of Eq. (13)

shows the distance from the ideal solution and the second term

shows the difference from the ideal solution’s fuzzy mean. The fact

that one candidate’s score is closer to the ideal solution does not

necessarily mean that it has a greater fuzzy mean than the others.

We consider thedistancefrom theideal solution andfuzzymean at

the same time. We derive Eq. (13) from the concept that the ideal

solution has to be the collection of the best solutions in each

criterion. This is the difference between our approach and TOPSIS

introduced by [22]. In the promotion screening process, it is not

yi ¼Xl

k¼1

¯ wk 1

n

Xn

j¼1

xi jk

0@ 1A8<:

9=;

¼Xl

k¼1

wklPl

h¼1 whu

;wk

lPlh¼1 wh

um

;wk

lPlh¼1 wh

lm

;wk

lPlh¼1 wh

l

!

Pn j¼1 xi jk;l

n ;

Pn j¼1 xi jk;lm

n ;

Pn j¼1 xi jk;um

n ;

Pn j¼1 xi jk;u

n

!( )

¼Xl

k¼1

wklPl

h¼1 whu

Pn j¼1 xi jk;l

n ;

wklPl

h¼1 whum

Pn j¼1 xi jk;um

n ;

wklPl

h¼1 whlm

Pn j¼1 xi jk;lm

n ;

wklPl

h¼1 whl

Pn j¼1 xi jk;u

n

!( )

¼Xl

k¼1

wklPl

h¼1 whu

Pn j¼1 xi jk;l

n

( );

Xl

k¼1

wklPl

h¼1 whum

Pn j¼1 xi jk;um

n

( );

Xl

k¼1

wklPl

h¼1 whlm

Pn j¼1 xi jk;lm

n

( );

Xl

k¼1

wklPl

i¼h whl

Pn j¼1 xi jk;u

n

( ) !

(8)




important that one candidate has better grades than the others. It

is more important that the grades of candidates could exceed the

level necessary for promotion.

4. Performance appraisal and promotion ranking system

Performance appraisal and promotion ranking (PAPR) is the

main primary role of human resource management in military

organization. Therefore, to support the PAPR function, we develop

a system called the PAPR system. The system is effective in

performance appraisal and ranking, and can help organizations

transform the employee evaluation process into a well-defined,

fair, and transparent process. This developed system is composed

of three main functions: (1) determining the weights of evaluation

criteria, (2) rating and data monitoring, and (3) data aggregation

and ranking.

4.1. Determining the weights of evaluation criteria

In order to aggregate decision data, the nominal group

technique plays an important role in generating ideas, prioritizing

them, and coming to a group consensus among organizational

members. An effective electronic nominal group technique is

developed for generating evaluation criteria and providing a

prioritized list of criteria through voting by group members. Theweight of evaluation criteria can be determined by aggregating the

decisions of external and internal experts in the field of human

resource management. The process for determining weights using

the electronic technique is shown in Fig. 3.

In Fig. 3, the process for determining the weights of evaluation

criteria consists of three parts as follows:

(1) The department of human resource management develops a

list of key criteria. In this appraisal and ranking system for

promotion screening, four criteria, such as, service rating,

multi-area aptitude, growth potential and innovativeness, are

considered.

(2) Decision makers evaluate the criteria through the electronic

nominal group technique. The technique provides an advanta-

geous decision-making environment to equalize participation,

encouraging the free flow of ideas in a nonthreatening setting

and enabling participaints to reach final decisions.

(3) The weights of key criteria are determined by aggregating the

decision maker’s results.

This technology can be used to collect and assess the relative

importance of evaluation criteria at various organizational levels.

The linguistic weighting variables, seen in Fig. 2, are used to assess

the weight of each criterion.

4.2. Rating and data monitoring

Thesecond function consists of rating, data monitoringand data

processing. The committee for decision making involves a number

of evaluators with equal authority who assess each candidate and

implement individual analyses. The candidates’ performance

scores recorded at their respective past organizations are provided

to the human resource management department for rating.

Fig. 3. Process of determining the weights of criteria.

Fig. 4. Example of data monitoring for abnormality and incorrectness. Fig. 5. Monitoring procedure.




The monitoring process is employed to examine abnormal

or incorrect data received from the evaluators. The degree

of scattering is used to control abnormal data as shown in Fig. 4.

The procedure for data monitoring is shown in Fig. 5.

This procedure utilizes all performance evaluation data without

excluding the maximum and minimum scores from the final

performance evaluation results. If all input is perceived as normal,

the linguistic assessments received from evaluators are converted

intotrapezoidal fuzzy numbersto construct a fuzzy-decisionmatrix,

andfrom this, thefuzzy weight of eachcriterionis determined. Then,

a weighted-normalized fuzzy-decision matrix is constructed.

4.3. Data aggregation and ranking

The appraisal results of individual evaluators should be

aggregated to compute the final appraisal scores to determine

the ranking of each candidate. The data aggregation and ranking

Fig. 6. Overall schematic flow diagram of the PAPR system.

Fig. 7. Assigning the weights of evaluation criteria.

Table 2

The weights of evaluation criteria.

Criteria Fuzzy number

C 1 0.2056 0.2551 0.2766 0.3415

C 2 0.1963 0.2449 0.2553 0.3293

C 3 0.2056 0.2551 0.2766 0.3415

C 4

0.1589 0.2041 0.2340 0.2927

Table 3

The aggregated scores for each candidate.

Can dida te Fuz zy n umb er

y1 5.5140 7.8571 8.9574 12.0813

y2 5.3330 7.6293 8.7305 11.8780

y3 4.9003 7.6905 8.6028 11.8577

y4 5.5296 7.8741 8.9858 12.0976

y5 5.5885 8.0782 9.2199 12.2927

y6 5.6667 8.0544 9.4397 12.3821

y7 4.8162 6.9830 8.0993 11.3252

y8 5.6760 8.0646 9.2057 12.2805

y9 5.5140 7.8571 8.9574 12.0813

y10 5.2617 7.5408 8.5319 11.7602




function transforms the several individual multi-criteria scored

lists of candidates into one aggregated rank-ordered list. Then,

using the ranking measures to determine the ranking of each

candidate, finalists can be selected for promotion.

The overall schematic flow diagram of the PAPR system is

shown in Fig. 6.

5. Numerical example

An example of ranking promotion candidates is introduced and

demonstrates the effectiveness of the PAPR system. In this

example, there are three evaluators, four criteria, and 10

candidates. A detailed procedure for ranking is shown in Section

3. Eqs. (2)–(5) are used for the fuzzy operations. First, the weights

of evaluation criteria are determined at by aggregating the

decisions of external and internal experts in the field of human

resource management, as shown in Fig. 7. Then, the aggregatedweights of the criteria are summarized in Table 2.

Then, the candidate’s scores are evaluated for each criterion.

Appendix A shows the scores that each candidate received from

each evaluator. The first candidate’s score from the second

evaluator for the third criterion is ‘G’, so x123 = (7, 8, 8, 9). From

Eq. (8), the aggregated scores of each candidate can be expressed

by a fuzzy number, yi. Table 3 displays the fuzzy numbers

representing the aggregated scores of each candidate.

In this case the positive and negative-ideal solutions are

y* = (6.1308, 8.6327, 10.4255, 13.0488), and y = (0, 0, 1.0426,

2.6098) which are derived from Table 2, the weights for criteria,

and Eq. (6), fuzzy membership functions.

Lastly, the candidates are ordered by the proposed ranking

method. The distance and fuzzy means are derived using Eqs. (9)–(12), and the candidates are ranked using Eq. (13). In Table 4, the

value represents the degreeof separation from theideal solution. A

largervalue means that thecandidate is closer to theideal solution,

in this case, the sixth candidate attains the highest ranking. Fig. 8

shows the visual ranking results from the PAPR system.

From the experimental result, we know that the developed

PAPR system can handle applicant records that contain both

qualitative and quantitative information. Finally, the system

provided performance evaluations of all candidates.

6. Conclusions

This paper develops an efficient performance appraisal and

ranking system for the promotion screening of candidates applyingfor a particular commission in a military organization. The system

uses fuzzy theory and electronic nominal group technique to

produce fair ranking decisions through a multi-criteria perfor-

mance appraisal process. The electronic nominal group technique

is adopted to collect and assess the relative importance of various

performance evaluation criteria collected at different organiza-

tional levels. This technology can prevent any particular organiza-

tion’s self-interests from dominating the selection and weighting

of performance evaluation criteria.

A new ranking procedure considering the metric distance and

fuzzy mean value is also proposed, which makes it possible to rank

order the performance of candidates by aggregating the scores of

multiple evaluators. The system also has a monitoring function that

uses all performance evaluation data without any removal. This

function is to prevent abnormal evaluation data which could occur

whenthere is an overly influential member in the evaluators’group,

or when a particular evaluator gives an incorrect evaluation result.

The system developed was applied to a military organization in

Korea. The results of this example have shown that this systematic

approach with a fuzzy procedure is a suitable method to produce

transparent and fair multi-criteria performance evaluations in

military organizations.

Appendix A

We assess 10 candidates’ scores for four criteria by three

evaluators. In the actual system, the evaluators score the candidates

using a computer program and theraw data are not shown. However,

for the purpose of demonstrating the procedure used in the example,

it is necessary to show the raw scores here.

Criteria Candidates Evaluators Criteria Candidates Evaluators

D1 D2 D3 D1 D2 D3

C 1 A1 VG G G C 3 A1 VG G G

A2 VG G VG A2 G VG G

A3 G VG G A3 G G G

A4 G G VG A4 G VG VG A5 G G VG A5 VG G VG

A6 VG VG G A6 G MG G

A7 VG G MG A7 MG G G

A8 G G G A8 VG VG VG

A9 G VG G A9 VG G G

A10 G G G A10 G VG G

C 2 A1 G VG G C 4 A1 G VG MG

A2 G G MG A2 MG G G

A3 G G G A3 MG G VG

A4 VG G G A4 MG G G

A5 G G VG A5 G VG G

A6 VG VG VG A6 G VG VG

A7 G MG G A7 MG G MG

A8 G G G A8 G VG VG

A9 G G VG A9 MG G VG

A10 G MG VG A10 MG G G

Table 4

The final ranking.

Candidate M 1 M 2 M 3 M 4 M 5 M 6 M 7 M 8 M 9 M 10

Value 0.8853 0.8610 0.8502 0.8876 0.9107 0.9197 0.7932 0.9092 0.8853 0.8467

Ranking 5 7 8 4 2 1 10 3 6 9

Fig. 8. . Visual ranking results from PAPR system.




References

[1] P. Allan, Avoiding common pitfalls in performance appraisal, Industrial Manage-ment (1992) 30–32.

[2] H.G. Chambers, M. Foulon, H. Hanfield-Jones, S.M. Hankin, E.G. Michaels III, Thewar for talent, McKinsey Quarterly (1998).

[3] G. Lowenberg, K.A. Conrad, Current Perspectives in Industrial/OrganizationalPsychology, Allyn & Bacon, Boston, MA, 1998.

[4] A.G.Momeyer,Why no one likesyour performance appraisal system? Training 23(1986) 95–98.

[5] G. Dressler, Human Resource Management, 6th ed., Prentice-Hall, Englewood

Cliffs, NJ, 1994.[6] S.W. Gilliland, J.C. Langdon, Creating performance management systems that

promote perceptions of fairness, in: J.W. Smither (Ed.), Performance Appraisal:State of the Art in Practice, Jossey-Bass, San Francisco, CA, 1998.

[7] E.W. Duggan, C.S. Thachenkary, Integrating nominal group technique and jointapplication development for improved systems requirement determination,Information & Management 41 (2004) 399–411.

[8] J. Perry,S. Linsley, Theuse of thenominalgroup technique as an evaluativetoolinthe teaching and summative assessment of the inter-personal skills of studentmental health nurses, Nurse Education Today 26 (2006) 346–353.

[9] P.L. Williams, N. White, R. Klem, S.E. Wilson, P. Bartholomew, Clinical educationand training: using the nominal group technique in research with radiographersto identify factors affecting quality and capacity, Radiography 12 (2006) 215–224.

[10] A.L. Delbecq, A.H. Van de Ven, Nominal versus interacting group processes forcommittee decision making effectiveness, Journal of Academy Management 14(1971) 203–212.

[11] G.Bortolan, R.Degani, A reviewof some methods forranking fuzzysubsets, FuzzySets and Systems 15 (1985) 1–19.

[12] J.J.Buckley,Ranking alternatives using fuzzy numbers, Fuzzy Sets and Systems 15(1985) 21–31.

[13] J.J. Buckley, Fuzzy hierarchical analysis, Fuzzy Sets and Systems 17 (1985) 233–247.

[14] S.H.Chen, Ranking fuzzy numbers withmaximizingset andminimizing set,FuzzySets and Systems 17 (1985) 113–129.

[15] L.S. Chen, C.H. Cheng, Selection IS personnel use fuzzy GDSS based on metricdistance method, European Journal of Operational Research 160 (2005) 803–820.

[16] C.T. Chen, C.T. Lin, S.F. Huang, A fuzzy approach for supplier evaluation andselection in supply chain management, International Journal of ProductionEconomics 102 (2006) 289–901.

[17] D. Dubois, H.Prade, The mean valueof a fuzzynumber,Fuzzy SetsandSystems24(1987) 279–300.

[18] M. Modarres, S. Sadi-Nezhad, Ranking fuzzy numbers by preference ration, FuzzySets and Systems 118 (2001) 429–436.[19] E. Triantaphyllou, C.T. Lin, Development and evaluation of five fuzzy multi-

attribute decision-making methods, International Journal of Approximate Rea-soning 15 (1996) 281–300.

[20] E.S Lee, R.L. Li, Comparison of fuzzy numbers based on the probability measure of fuzzy events, Computers and Mathematics with Applications 15 (1988) 887–896.

[21] C.H. Cheng, A new approach of ranking fuzzy numbers by distance method, FuzzySets and Systems 95 (1998) 307–317.

[22] C.L. Hwang, K. Yoon, Multiple attribute decision making, Lecture Notes in Eco-nomics and Mathematical Systems, vol. 198, Springer, Berlin, 1981.

[23] Korean Military, Guide Book for the Human Resource Management in KoreaMilitary, Korea Military, Seoul, 2005.

[24] F. Herrera, E. Herrera-Viedma, Linguistic decision analysis: steps for solvingdecision problems under linguistic information, Fuzzy Sets and Systems 115(2000) 67–82.

[25] C.G.E. Boender, J.G. de Graan, F.A. Lootsma, Multi-criteria decision analysis withfuzzy pairwise comparisons, Fuzzy Sets and Systems 29 (1989) 133–143.

[26] R. Goetschel Jr., W. Voxman, Elementary fuzzy calculus, Fuzzy Sets and Systems

18 (1986) 31–43.


a performance appraisal and promotion ranking system based on fuzzy logic

Documents