ion in ssc exams

8/2/2019 ion in SSC EXAMS

1/10

NOTICE

Equalisation of scores in SSC Examinations

Staff Selection Commission has been conducting various examinations in

multiple batches because of large number of candidates and difficulties in

getting adequate educational institutions for holding the examinations in a

single batch. For perhaps the first time in its history, the number of applicants

in a single examination exceeded one million when the Combined Higher

Secondary Level Examination, 2010 for the recruitment of Lower Division

Clerks and Data Entry Operators, elicited response from over 16 lakh

candidates, with app. 21% of them applying online. This would require the

Examination, rescheduled on 27& 28.11.2010 (in view of Common Wealth

Games), to be held in at least three batches. The Commission, with the help of

experts, has striven to construct question papers of comparable difficulty level.

While such an exercise is theoretically possible, in practice it is impossible to

have two or more question papers of identical difficulty levels. Even if the

difficulty levels of question papers vary slightly, candidates taking more

difficult papers may be at a disadvantage viz-a-vis others. Therefore, there is a

need for equating of the marks in examinations involving multiple batches and

question papers.

2. The Commission had examined the views of an Expert Group,

constituted by it with the approval of Government of India in 2009, on this

issue. The Commission had placed before the Expert Group that the technique

to be followed for equating should be transparent, easily comprehensible to

the candidates, acceptable to experts and prove itself in Courts of Law if and


2/10

when challenged. This was accepted by the Expert Group which further

advised the Commission to place a paper on the technique on its website for

adequate time, give publicity to such placement through the media, invite

comments, observations and suggestions and decide on adopting the

technique thereafter.

3. Accordingly, the Commission places this paper inviting views, comments

and suggestions from academicians, parents, candidates and also stakeholders

on or before 31.10.2010. Such views, comments and suggestions, if any, may

be addressed to Smt. Vandana Sethi, Deputy Secretary, Staff Selection

Commission, Block-12, CGO Complex, Lodhi Road, New Delhi-110 003 (e-mail:

[email protected]). The Commission proposes to adopt the

Equipercentile Equating Technique for examinations involving multiple batches

from Combined Higher Secondary Level Examination, 2010 onwards.


3/10

PAPER ON EQUATING TECHNIQUES

1. Introduction:

Building Question Papers based on the same test specification either on

blueprint or content template will not ensure that these tests are equivalent,

specifically based on the difficulty level, content and ability tested. To ensure

fairness in the scoring system when candidates are administered different

Question Papers of the test, an appropriate statistical procedure is needed to

equate the test scores based on the different Question Papers. In other words,

Equating is needed.

2. Large-scale testing programs often require multiple Question Papers to

maintain test security over time and to enable equivalent tests to be

administered without repeating identical items. The outcomes after equating are

well get the same scores in tests in which different question papers were

administered.

3. Equating is a statistical process that is used to adjust scores on

multiple question papers so that scores on the forms can be used

interchangeably. It adjusts for differences in difficulty among Question

Papers that are built to be similar in difficulty and content.

4. Example:

Here is an example of Test of English language in 2 Question Papers (VA1 and

VA2) which consists of 50 items each that would assess the usage of articles

amongst other things.

VA1

1. ___ book that I bought yesterday is interesting.

2. I saw ___ man wearing a red suit.


4/10

VA2

1. ___ goat was crossing ___ river.

2. I have been waiting for ___ auto.

After evaluating the Answer Sheets the overall performance of the students in

two different Question Papers from similar background students is found to be

varying with the following mean scores.

Mean scores are:

VA1 14.5

VA2 12.5

The number of right answers in VA1 is found to be 2 units more than those

of VA2. Similar variation in mean scores may be found in the case of all

the questions included in the test. Therefore, there could be a need for

adjustment of scores of each test so that scores from different question

papers can be used interchangeably

5. Process followed in Equating

The purpose of equating should be clear.

The Examination should have multiple question papers on the same

subject.

Alternate question papers are set in accordance with items of the samecontent, difficulty level and ability tested.

Data of responses to items/questions should be collected.

The responses should be in the form of (A, B, .., X) or (Yes/No)

etc. for objective type tests.

Equating requires that a choice be made about what type of relationships

between different question papers is to be estimated. For example, this


5/10

may involve decision on whether to use linear/non linear methods of

equating.

Descriptive statistics to obtain mean, standard deviation and/or the

percentile rank of scores may need to be generated after evaluation of the

papers.

A statistical equating method should be carefully selected for adoption.

The results of equating should be evaluated carefully. .

6. Different Methods of Equating

a) Median/Mean Equating

b) Linear Equating (Based on mean and S.D.)

c) Equipercentile Equating

d) Equating using Item Response Theory

a) Median/Mean Equating

Procedure:

In the given example, two question papers of Verbal Ability Test are

administered on two similar groups of student(VA1 and VA2)

Mean score of VA1 = 14.5 & Mean score of VA2 = 12.5

Difference between mean scores = 14.512.5 = 2

Mean equating involves the addition of a constant (2 = difference in the

mean scores) to all the raw scores on VA2 to find equated scores on VA1

b) Linear equating (Based on Mean and S.D.)

This involves equating the two tests using the mean and standard deviation

of each question paper. If x represents a score on the VA2 (in the given

example) and y represents a score on the reference form (VA1), then x and y

are equivalent in a group of examinees if:


6/10

Procedure:For adjusting the new scores according to the reference form the formula is:

X(VA2) Y(VA1)

Mean 12.5 14.5

S.D. 4.5 4

Y adjusted to X:

c) Equipercentile Equating

The method of equi-percentile equating involves fixing up a common percentile

for all forms (it is observed that same test score in each of the question papers

gives rise to different percentiles) and then finding the different scores in these

forms in terms of number right scores. Thus a score of 17.2 in series -1 score is

equivalent to 14.5 in series-2.This in essence is equipercentile equating.

Procedure: Given below are two sets of scores, series 1 and series 2, which are

multiple question papers of the same test. As far as possible, the no. of

examiners in either series should be equal. Their percentile ranks are calculated

from the grouped data using SPSS or Excel.

y mean( y) x mean(x)

SD(y) SD(x)

(x mean(x))y SD(y) mean(y)

SD(x)

10 12.54 14.5 12.28

4.5


7/10

series 2

score percentile rank

1 0.00%

2 5.50%

3 11.10%

4 16.60%

5 22.20%

6 27.70%

7 33.30%

8 38.80%

9 44.40%

10 50.00%

11 55.50%

12 61.10%

13 66.60%

14 72.20%

15 77.70%

16 83.30%

17 88.80%

18 94.40%

19 100.00%

series 1

score percentile rank

3 0.00%

4 5.20%

5 10.50%

6 15.70%

7 21.00%

8 26.30%

9 31.50%

10 36.80%

11 42.10%

12 47.30%

13 52.60%

14 57.80%

15 63.10%

16 68.40%

17 73.60%

18 78.90%

19 84.20%

20 89.40%

21 94.70%

22 100.00%


8/10

The above graph shows both percentile ranks of series 1 and 2 are put together

in order to enable the percentile score of one raw mark of series-1 to be slightly

different from the raw mark score of series-2. The percentile rank of 17 in

series -1 is 88.8% and in series -2 the same score of 17 has a percentile of

93.6%.

In the method of equi-percentile equating, a cut-off of a percentile rank to

qualify for selection is to be fixed, say 75 percentile and this common 75

percentile has a raw score of 14.5 in series-1 and 15.26 in series-2. Therefore,

anyone who gets more than 14.5 in series -1 and 15.26 in series 2 will qualify

and the equi-percentile is 75.

SSC PROPOSES TO USE THE EQUIPERCENTILE METHOD IN VIEW OF ITS

SIMPLICITY


9/10

d) Equating using Item Response Theory (IRT)

Basic Concepts

Ability (): Ability is measured on the scale (-3 to +3)

Item difficulty (b): The item difficulty of an item and the ability of the

test taker are on the same scale.it is invariably taken as the middle pointof the item characteristic curve where the curve shows a tendency of

contra flexure that is bending in the opposite directions (-3 to +3) Both of these are on the same scale and along the x-axis

Probability of getting the correct answer (0 to 1) on the y-axis

This is a fraction like 0.75which means if a person of particular ability

say ( any ability between -3 and +3) attempts 100 times the same item

75 times he will get it right and 25 times he will get it wrong.

Fred Lords Model

- +

-3 + 3

- 4 + 4

tan = a Item

discriminationItem Difficulty b

Guessing c

Pi ()(probability ofgettinganswerright onany item I

0.5

+1.01-c


10/10

Types of equating in IRT

i) Horizontal equating

ii) Vertical equating

Horizontal equating is equating scales of about the same abilityVertical equating is equating scales across completely different levels of ability

THE COMMISSION DOES NOT USE IRT IN ITS EXAMINATIONS.

ion in ssc exams

Documents