irt models for polytomous response data · pdf filenominal response data • nominal...

IRT Models for Polytomous Response Data

University of KansasItem Response Theory

Stats Camp ‘07

Overview• General Theory

• Polytomous Data Types:– Graded Response– Partial Credit– Nominal Response

• Polytomous IRT Models:– Graded Response– Partial Credit– Nominal Response

General Theory of Polytomous Response Models

• Item response models for polytomous data extend the general underlying IRT premise…– Modeling item response behavior as a function of

(possibly multiple) latent traits.

• …to data types that are not only limited to binary responses.– The type of model is dependent on the type of data

that have been collected.

A Polytomous Starter Example

• Imagine you have the following data from a set of “examinees”:– The number of baskets

made by each examinee out of 10 shots from various locations on a basketball court.

Basketball Data

• The idea of the number of successfully made baskets out of a set number of attempts could possibly follow a well known distribution.– The Binomial Distribution:

( ) xnx ppxn

xXP −−⎟⎟⎠

⎞⎜⎜⎝

⎛== 1)(

Binomial, Continued…

Binomial Distribution Parameters• n is the number of trials (10 for each location in our

example).• x is the number of made baskets for each location.• p is the probability of a made basket for each trial

(assumed constant).

( ) xnx ppxn

xXP −−⎟⎟⎠

⎞⎜⎜⎝

⎛== 1)(

Linking IRT and the Data

• If we could adapt IRT for these data, here is what IRT could provide for the basketball example:– A measure of the tendency for an “examinee”

to hit shots from around the court (the examinee’s latent variable).

• Could be thought of as an examinee’s shooting ability.

– A measure of the relative difficulty of each shot location (the model item parameters).

Linking, Continued• The idea behind polytomous IRT models is to

adapt the IRT portion of the model so that the data are modeled appropriately.

• In this case, the binomial model can be adapted by using the binary IRT model response function in place of the binomial model p parameter.

• See: – Safrit, Costa, and Costa (1989)– Spray (1990)

Binomial IRT Model

• A Two-Parameter Logistic Binomial IRT model would then be:

( )

( )jij

jij

bDa

bDa

ij eeP −

−

+= θ

θ

θ1

)(

• Where:

( ) ijijij xnij

xij

ij

ijiijij PP

xn

xXP −−⎟⎟⎠

⎞⎜⎜⎝

⎛== )(1)()|( θθθ

Binomial IRT Response Function

Binomial IRF

0

0.2

0.4

0.6

0.8

1

-4-3

.4-2

.9-2

.4-1

.9-1

.4-0

.8-0

.3 0.2

0.72

1.24

1.76

2.28 2.

83.

323.

84

Theta

P(X=

x|Th

eta) P(X=0|Theta)

P(X=1|Theta)P(X=2|Theta)P(X=3|Theta)P(X=4|Theta)

Open “binomial response demo.xls” for file demo

Graded Response Models

(AKA Ordered Categorical Responses)

Graded Responses

• One of the more straight-forward polytomous data types is that of the graded-response format.

• Graded response data consists of a score that is an ordinal number, typically ranging from 0 to M.– 0, 1, 2,…, M+1

• Higher scores represent better performance on the item.

Example Graded Response ItemFrom the 2006 Illinois Standards Achievement Test (ISAT):

www.isbe.state.il.us/assessment/pdfs/Grade_5_ISAT_2006_Samples.pdf

ISAT Scoring Rubric

Additional Example Item• Cognitive items are not the only ones where graded

response data occurs.

• Likert-type questionnaires are commonly scored using ordered categorical values.– Typically, these ordered categories are treated as continuous

data.• Methods of analysis typically include Factor Analysis.

– More on this during Friday’s discussion.

• Consider the following item from the Satisfaction With Life Scale (e.g. SWLS, Diener, Emmons, Larsen, & Griffin, 1985)…

SWLS Item #1

I am satisfied with my life.1. Strongly disagree.2. Disagree.3. Slightly disagree.4. Neither agree nor disagree.5. Slightly agree.6. Agree.7. Strongly agree.

Graded Response Model

• An extension of the 2PL model for graded response data is Samejima’s Graded Response Model (1969).

• The graded response model specifies the likelihood that an examinee of a given ability will respond provide a response that receives a grade of xij (xij = 0,…,M).

Graded Response Model

• Samejima’s Graded Response Model:

( ) ( )( )( )ijxij

ijxij

ij bDa

bDa

iijijixe

exXPP −

−

+=≥= θ

θ

θθ1

|*

• Where:

( ) ( ) ( )ixixiijij ijijPPxXP θθθ *

1*| +−==

Graded Response Model Parameters

( ) ( )( )( )ijxij

ijxij

ij bDa

bDa

iijijixe

exXPP −

−

+=≥= θ

θ

θθ1

|*

• For each item j: one discrimination parameter aj.• For M-1 difficulty parameters bm.

– b1 < … < bm

• P(Xij ≥ 0) = 1.0• P(Xij ≥ M) = 0.0

Graded Response Function

Graded Response IRF

0

0.2

0.4

0.6

0.8

1

-4-3

.4-2

.9-2

.4-1

.9-1

.4-0

.8-0

.3 0.2

0.72

1.24

1.76

2.28 2.

83.

323.

84

Theta

P(X=

x|Th

eta) P(X=0|Theta)

P(X=1|Theta)P(X=2|Theta)P(X=3|Theta)P(X=4|Theta)

Open “graded response demo.xls” for file demo

Nominal Response Models

Nominal Response Data

• Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric values.– Rather, item responses are in the form of nominal

categories.

• Information gained from the use of such models can be useful for detecting which distracter options are better than others in multiple choice tests.

Nominal Response Model Features

• The nominal response model is a model for the categorical responses possible within an item of a test.

• The model specifies the probability that an examinee with a given value of the latent trait selects response option m.

Example Nominal Response Item

Additional Item Types

• Non-cognitive tests can also contain differing item types that could be modeled using a Nominal Response Model.

• For example, consider an item from a questionnaire about political attitudes…

Example Nominal Response Item

Which political party would you identify yourself with?

1. Democrat2. Republican3. Independent4. Green5. Unaffiliated

Nominal Response Model

• An extension of the 2PL model for nominal response data is Bock’s Nominal Response Model (1972).

• The nominal response model specifies the likelihood that an examinee of a given ability will selection option kj of item j.

Nominal Response Model

• Bock’s Nominal Response Model:

( )( )

( )∑=

−

−

== m

h

ba

ba

iijjhijh

jkijk

e

ekXP

1

|θ

θ

θ

Nominal Response Model Parameters

( )( )

( )∑=

−

−

== m

h

ba

ba

iijjhijh

jkijk

e

ekXP

1

|θ

θ

θ

• For each level k of item j: one discrimination parameter ajk and one difficulty parameter bjk.

• No additional constraints on the parameter values.

Nominal Response FunctionOpen “nominal response demo.xls” for file demo

Nominal Response IRF

0

0.2

0.4

0.6

0.8

1

1.2

-4

-3.3

-2.7 -2

-1.4

-0.8

-0.1

0.52

1.16 1.8

2.44

3.08

3.72

Theta

P(X=

m|T

heta

)

P(X=a|Theta)P(X=b|Theta)P(X=c|Theta)P(X=d|Theta)

Conclusion

• Numerous polytomous IRT models exist.– Each extends the basic philosophy of IRT to a

a data type that is not binary.• Some differ with respect to how each model is

parameterized for the same data.• Some differ with respect to the differing types of

data to be modeled.

• Each polytomous IRT model specifies the behavior of an examinee as a function of a latent trait (often representing ability).

Next…

• Estimation of Parameter for IRT Models–Estimate person parameters

when item parameters are known

–Joint estimation of person and item parameters