OECD Programme for International Student Assessment

(PISA) Mathematics

Margaret Wu

ACER & University of Melbourne

OECD PISA Project - led by ACER, with ETS, Westat, NIER, Cito

• 15 year-old students– end of compulsory education– not intact class sample

• About 40 countries

• 3-year cycle:– 2000 Reading– 2003 Mathematics– 2006 Science

• Student and School questionnaires

Mathematics Framework - 1

• Experts driven, not curricula driven– TIMSS based on common curricula– PISA based on “definition of Mathematics”

deefined by a group of “expert” mathematics educators.

• Expert Group Members– Jan de Lange, Werner Blum, Mary Lindquist,

Vladmir Burjan, Sean Close, John Dossey, Zbigniew Marciniak, Mogens Niss, Kyungmee Park, Luis Rico, Yoshinori Shimizu

Mathematics Framework - 2

• Definition of PISA Mathematics Literacy:– Mathematics literacy is an individual’s capacity

to identify and understand the role that mathematics plays in the world, to make well-founded judgements and to engage in mathematics, in ways that meet the needs of that individual’s life as a constructive, concerned, and reflective citizen.

Mathematics Framework - 3• Organisation of content

– TIMSS by topics and subtopics of mathematics • Number, Algebra, Measurement, Geometry, Data

– PISA by overarching ideas (phenomenological approach)

• Quantity, Space and Shape, Change and Relationships, Uncertainty

• Conception of assessment items is different; making a distinction between teaching and assessment

Mathematics Framework - 4

• Processes– Emphasis on Mathematisation - the processes

involved from encountering a real-world problem to generating a solution

– Three competency classes• Reproduction - practised routine procedures

• Connection - making judgements, reasoning

• Reflection - making generalisations

Item Context Emphasis on authenticity - 1

• No “naked” drills items, e.g.,– Solve a linear or quadratic equation– Construct an angle– Simplify a fraction

• Few intra-mathematics items, e.g.,– pattern observation in sum of odd numbers– properties of numbers, e.g., perfect numbers.

Item Context Emphasis on authenticity - 2

• Problem context is not just for the sake of adding context, but for real-world application.

How far is the foot of a 2 m ladder from the wall when the top of the ladder is 1.92 m above the ground?

X

Not a good PISA item!

Another example of better context

– Farmer Dave keeps chickens and rabbits. Dave counted altogether 65 heads and 180 feet. How many chickens does Dave have?

Tickets to the school concert costs $4 for an adult and $2 for a child. 65 tickets were sold for a total of $180. How many children’s tickets were sold?

Item Context Emphasis on authenticity - 3

• Problems with contexts that influence the solution and its interpretation are preferred for assessing mathematical literacy.

You must be over 21 to drink alcoholic drink. Which people should you check?drinking beer 22 yrs old drinking coke 16 yrs old

If a card has a vowel on one side, it must have an even number of the other side. Which cards should you turn over to check?

A 6 J 7 (Wason. Griggs&Cox)

Item Format

• MC and Open-ended; about half of each kind.

• Raw responses captured as much as possible

• Double-digit coding to keep track of different approaches

Example Mathematics Item - 1

The picture shows a spinner used in playing games. For a game, the spinner is used to choose a person at random to start the game. Explain how you will use this spinner to choose a person at random if there are (1) three players, and (2) nine players.

5

4

3

2

1

6

The picture shows a spinner used in playing games. (1) What is the probability of spinning a “3”? (2) What is the probability of spinning an even number?

Fits PISA framework

better

Example Mathematics Item - 2Break-in

On the radio, an advertisement for an insurance company ran as follows: “Every 10 minutes, a car is stolen in Zedland. Every 21 minutes, a house is broken into. Take up an insurance policy today.”

Using only the information given in the advertisement, can you conclude

(1) anything about the chance a car will be stolen in Zedland?

(2) that it is more likely to have a car theft than a house break-in?

Give reasons to support your answer.

Example Mathematics Item - 3

LaneReaction time (secs)

Final time (secs)

1 0.147 10.092 0.136 9.993 0.197 9.874 0.18 Did not finish race5 0.21 10.176 0.216 10.047 0.174 10.088 0.193 10.13

To date, no humans can react in less than 0.110 of a second.

Challenges for Test Developers

• Real-world mathematics are not easy to find (for 15 year-olds)

• Predominantly about price, cost, discounts in everyday life (too domestic?).

• PISA is somewhat “middle-class” - telephone, internet, cars, computers

Hong Kong Performed Well

370

420

470

520

570

620

Hon

g K

ong

Japa

n

Kor

ea

New

Zea

land

Fin

land

Aus

tral

ia

Can

ada

Sw

itzer

land

Uni

ted

Kin

gdom

Bel

gium

Fra

nce

Aus

tria

Den

mar

k

Icel

and

Liec

hten

stei

n

Sw

eden

Irel

and

Nor

way

Cze

ch R

epub

lic

Uni

ted

Sta

tes

Ger

man

y

Hun

gary

Rus

sia

Spa

in

Pol

and

Latv

ia

Italy

Por

tuga

l

Gre

ece

Luxe

mbo

urg

Mex

ico

Bra

zil

Ma

the

ma

tics

Me

an

Sco

re

Compare TIMSS and

PISA

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

sta

nd

ard

ised

sco

res

jpn

jpn

kor

kor

nzl fin

fin

can

can

aus

aus

uk

uk

cze

cze

usausahun

hun

rus

rus

lav

lav

ita

PISA TIMSS

ita

nzl

hkg

hkg

Compare TIMSS and PISA

-3

-2

-1

0

1

2

3

-3 -2 -1 0 1 2 3

Standardised TIMSS score

sta

nd

ard

ised

PIS

A s

core

korjpn

ita

can

hun

rus

aus

cze

finuk

nzl

usa

lav

hkg

Cluster Analysis

cae nsl eng sco aus irl usa bfl nld

rus hkg kor jpn

Item Hong Kong found easier

Apples M136Q02

• Number of apple trees = n2

• Number of conifer trees = 8n– where n is the number of rows of apple trees.

• There is a value of n for which the number of apple trees equals the number of conifer trees. Find the value of n and show your method of calculating this.

Item Hong Kong found more difficult

Speed of Racing Car M159Q03Speed(km/h)

180

160140

120

100

80

60

40

20

00 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0

0.5 1.5 2.5

Starting line Distance along the track (km)

Speed of a racing car along a 3 km track(second lap)

What can you say about the speed of the car between the 2.6 km and 2.8 km marks?

A. The speed of the car remains constant.

B. The speed of the car is increasing.

C. The speed of the car is decreasing.

D. The speed of the car cannot be determined from the graph.

Conclusions

• PISA mathematics relates to real-world

• PISA tests something a little different from TIMSS

• For Hong Kong,– Mathematics education must be made more

relevant to everyday life and to real-world applications.