use of pisa in quality improvement polices – richard yelland, oecd head of policy advice and...

23
OECD EMPLOYER BRAND Playbook 1 Programme for International Student Assessment (PISA) Richard Yelland

Upload: unicefmne

Post on 13-Jun-2015

353 views

Category:

Education


0 download

DESCRIPTION

Presentation at the conference "Quality Education for Better Schools, Results and Future" organized by UNICEF and the Ministry of Education in Podgorica, July 8-10, 2014

TRANSCRIPT

Page 1: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

OECD EMPLOYER

BRAND

Playbook

1

Programme for

International

Student

Assessment (PISA)

Richard Yelland

Page 2: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

2 PISA in brief

• Over half a million students…– representing 28 million 15-year-olds in 65 countries/economies

… took an internationally agreed 2-hour test…– Goes beyond testing whether students can

reproduce what they were taught…

… to assess students’ capacity to extrapolate from what they know and creatively apply their knowledge in novel situations

– Mathematics, reading, science, problem-solving, financial literacy

– Total of 390 minutes of assessment material

… and responded to questions on…– their personal background, their schools

and their engagement with learning and school

• Parents, principals and system leaders provided data on…– school policies, practices, resources and institutional factors that

help explain performance differences .

Page 3: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

3 PISA in brief

• A shared learning tool for all involved– ‘Crowd sourcing’ and collaboration

• PISA draws together leading expertise and institutions from participating countries to develop instruments and methodologies…

… guided by governments on the basis of shared policy interests

– Cross-national relevance and transferability of policy experiences

• Emphasis on validity across cultures, languages and systems

• Frameworks built on well-structured conceptual understandingof academic disciplines and contextual factors

– Triangulation across different stakeholder perspectives

• Systematic integration of insights from students, parents, school principals and system-leaders

– Advanced methods with different grain sizes

• A range of methods to adequately measure constructs with different grain sizes to serve different decision-making needs

• Productive feedback, at appropriate levels of detail, to fuel improvement at every level of the system .

Page 4: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

4 The structure of the PISA assessment

2000 2003 2006 2009 2012 2015

Reading Reading Reading Reading Reading Reading

Mathematics

Mathematics

Mathematics

Mathematics

Mathematics Mathematics

Science Science Science Science Science Science

Problem Solving

Digital Reading

Problem Solving, Financial

literacy, Digital Math, Digital

reading

Collaborative Problem Solving,

Financial literacy,

Page 5: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

• PISA 2012:

–Student and school questionnaires

• Options:

– ICT questionnaire

–Educational career questionnaire

–Parent questionnaire

5 Questionnaires

Page 6: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

6

Climbing Mount Fuji

Mount Fuji is a famous dormant volcano

in Japan.

Mount Fuji is only open to the public for

climbing from 1 July to 27 August each

year. About 200 000 people climb

Mount Fuji during this time.

On average, about how many people

climb Mount Fuji each day?

A. 340

B. 710

C. 3400

D. 7100

E. 7400

PISA 2012 Sample Question 1

Page 7: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

7

Percent of 15-year-olds who scored Level 2 or AboveS

hang

hai-C

hina

Sin

gapo

reH

ong

Kon

g-C

hina

Kor

eaE

ston

iaM

acao

-Chi

naJa

pan

Fin

land

Sw

itzer

land

Chi

nese

Tai

pei

Can

ada

Liec

hten

stei

nV

ietn

amP

olan

dN

ethe

rland

sD

enm

ark

Irel

and

Ger

man

yA

ustr

iaB

elgi

umA

ustr

alia

Latv

iaS

love

nia

Cze

ch R

epub

licIc

elan

dU

nite

d K

ingd

omN

orw

ayF

ranc

eN

ew Z

eala

ndO

EC

D a

vera

ge

Spa

inR

ussi

an F

eder

atio

nLu

xem

bour

gIta

lyP

ortu

gal

Uni

ted

Sta

tes

Lith

uani

aS

wed

enS

lova

k R

epub

licH

unga

ryC

roat

iaIs

rael

Gre

ece

Ser

bia

Rom

ania

Tur

key

Cyp

rus*

Bul

garia

Kaz

akhs

tan

Uni

ted

Ara

b E

mira

tes

Tha

iland

Chi

leM

alay

sia

Mex

ico

Uru

guay

Mon

tene

gro

Cos

ta R

ica

Alb

ania

Arg

entin

aB

razi

lT

unis

iaJo

rdan

Qat

arC

olom

bia

Per

uIn

done

sia

0

10

20

30

40

50

60

70

80

90

100

PISA 2012 Sample Question 1

Page 8: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

8

Revolving DoorA revolving door includes three wings which rotate within a circular-shaped space. The inside diameter of

this space is 2 metres (200 centimetres). The three door wings divide the space into three equal sectors.

The plan below shows the door wings in three different positions viewed from the top.

The two door openings (the dotted arcs in the diagram) are the same size.

If these openings are too wide the revolving wings cannot provide a sealed

space and air could then flow freely between the entrance and the exit,

causing unwanted heat loss or gain. This is shown in the diagram opposite.

What is the maximum arc length in centimetres (cm) that each door

opening can have, so that air never flows freely between the entrance and

the exit?

Maximum arc length: ____________ cm

PISA 2012 Sample Question 4

Page 9: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

9

Percent of 15-year-olds who scored Level 6 or AboveS

hang

hai-C

hina

Sin

gapo

re

Chi

nese

Tai

pei

Hon

g K

ong-

Chi

na

Kor

ea

Japa

n

Mac

ao-C

hina

Liec

hten

stei

n

Sw

itzer

land

Bel

gium

Pol

and

Ger

man

y

New

Zea

land

Net

herla

nds

Can

ada

Aus

tral

ia

Est

onia

Fin

land

Vie

tnam

Slo

veni

a

OE

CD

ave

rag

e

Aus

tria

Cze

ch R

epub

lic

Fra

nce

Slo

vak

Rep

ublic

Uni

ted

Kin

gdom

Luxe

mbo

urg

Icel

and

Uni

ted

Sta

tes

Isra

el

Irel

and

Italy

Hun

gary

Por

tuga

l

Nor

way

Den

mar

k

Cro

atia

Sw

eden

Latv

ia

Rus

sian

Fed

erat

ion

Lith

uani

a

Spa

in

Tur

key

Ser

bia

Bul

garia

Gre

ece

Rom

ania

Uni

ted

Ara

b E

mira

tes

Tha

iland

0

5

10

15

20

25

30

PISA 2012 Sample Question 4

Page 10: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

Singapore

Hong Kong-ChinaChinese Taipei

Korea

Macao-ChinaJapan LiechtensteinSwitzerland

NetherlandsEstonia FinlandCanada

PolandBelgiumGermany Viet Nam

Austria AustraliaIrelandSlovenia

DenmarkNew ZealandCzech Republic France

United KingdomIceland

LatviaLuxembourg NorwayPortugal ItalySpain

Russian Fed.Slovak Republic United StatesLithuaniaSwedenHungary

CroatiaIsrael

GreeceSerbiaTurkey

Romania

BulgariaU.A.E.KazakhstanThailand

ChileMalaysia

Mexico410

420

430

440

450

460

470

480

490

500

510

520

530

540

550

560

570

580

Mean score

High mathematics performance

Low mathematics performance

… Shanghai-China performs above this line (613)

Montenegro, with 11 countries performing below

Average performance

of 15-year-olds in

MathematicsFig I.2.13

US

Page 11: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

Change in performance between PISA 2003 and 2012

Indonesia

Thailand

Russian Fed.

United States

Latvia

Spain

NorwayLuxembourg

Ireland

Austria

SwitzerlandJapan

Liechtenstein

Korea

Brazil

Tunisia

Mexico

Uruguay

Turkey

Greece

Italy

Portugal

Hungary

Poland

Slovak Republic

OECD average

Germany

Sweden

France

Denmark

Iceland

Czech Republic

New ZealandAustralia

Macao-China

Belgium

Canada

Netherlands

Finland

Hong Kong-China

-4

-3

-2

-1

0

1

2

3

4

5

350 400 450 500 550 600

Ave

rag

e a

nn

ua

l m

ath

em

ati

cs

sc

ore

ch

an

ge

Average mathematics performance in PISA 2003

Montenegro

Imp

rovin

g p

erfo

rma

nc

eD

ete

riora

ting

pe

rform

an

ce

PISA 2003 performance below the OECD averagePISA 2003 performance

above the OECD average

Fig I.2.1811

Page 12: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

Mathematics, reading and science Israel, Poland, Portugal, Turkey, Brazil, Dubai

(UAE), Hong Kong-China,

Macao-China, Qatar, Singapore, Tunisia

Mathematics and readingChile, Germany, Mexico, Albania, Montenegro,

Serbia, Shanghai-China

Mathematics and scienceItaly, Kazakhstan, Romania

Reading and scienceJapan, Korea, Latvia, Thailand

Mathematics onlyGreece, Bulgaria, Malaysia,

United Arab Emirates (ex. Dubai)

Reading only Estonia, Hungary, Luxembourg, Switzerland,

Colombia, Indonesia, Liechtenstein, Peru,

Russian Federation, Chinese Taipei

Science onlyIreland

Of the 65 countries 45 improved at least in one subject12

Page 13: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

AustraliaAustria

BelgiumCanada

Chile

Czech Rep.

Denmark

Estonia

Finland

France

Germany

Greece

Hungary

IcelandIreland

Israel

Italy

Japan

Korea

Luxembourg

Mexico

Netherlands

New Zealand

Norway

Poland

Portugal

Slovak Rep.

Slovenia

Spain Sweden

Switzerland

Turkey

UK

US

Singapore

Hong Kong-ChinaChinese Taipei

Macao-China

Liechtenstein

Viet Nam

Latvia

Russian Fed.Lithuania

Croatia

SerbiaRomania

Bulgaria United Arab Emirates

Kazakhstan

Thailand

Malaysia

Montenegro

02468101214161820222426

2012Shanghai-China

Socially equitable

distribution of learning

opportunities

Strong socio-economic

impact on student

performance

Performance and equity:

a tradeoff ?

Page 14: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

0

10

20

30

40

50

60

70

80

Sh

an

gh

ai-

Chin

aH

on

g K

on

g-C

hin

aM

acao

-Chin

aV

iet

Nam

Sin

ga

pore

Ko

rea

Chin

ese

Ta

ipe

iJap

an

Lie

chte

nste

inS

witze

rlan

dE

sto

nia

Ne

the

rlan

ds

Po

lan

dC

an

ad

aF

inla

nd

Be

lgiu

mP

ort

ug

al

Ge

rma

ny

Tu

rke

yO

EC

D a

vera

ge

Italy

Sp

ain

Latv

iaIr

ela

nd

Au

str

alia

Th

aila

nd

Au

str

iaL

uxe

mb

ourg

Czech

Rep

ublic

Slo

ve

nia

Un

ite

d K

ing

do

mL

ith

uan

iaF

rance

Norw

ay

Ice

lan

dN

ew

Zea

land

Ru

ssia

n F

ed

.U

nite

d S

tate

sC

roa

tia

Den

ma

rkS

wed

en

Hun

ga

ryS

lova

k R

ep

ub

licM

exic

oS

erb

iaG

ree

ce

Isra

el

Tu

nis

iaR

om

an

iaM

ala

ysia

Indo

nesia

Bu

lga

ria

Ka

za

kh

sta

nU

rug

ua

yB

razil

Costa

Ric

aC

hile

Colo

mbia

Mo

nte

ne

gro

U.A

.E.

Arg

en

tina

Jord

an

Pe

ruQ

ata

r

%

Percentage of resilient students

More than 40

% resilient Between 20%-40% of resilient students Less than 20%

Fig II.2.414

Socio-economically disadvantaged students not only score lower in mathematics, they also report lower levels of engagement, drive, motivation and self-beliefs. Resilient students break this link and share many characteristics of advantaged high-achievers.

A resilient student is situated in the bottom quarter of

the PISA index of economic, social and cultural

status (ESCS) in the country of assessment and

performs in the top quarter of students among all

countries, after accounting for socio-economic status.

Page 15: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

-50

-40

-30

-20

-10

0

10

20

30

Jo

rdan

Qata

rT

haila

nd

Ma

laysia

Icela

nd

U.A

.E.

Latv

iaS

inga

po

reF

inla

nd

Sw

ede

nB

ulg

aria

Ru

ssia

n F

ed.

Alb

an

iaM

on

ten

eg

roL

ithu

ania

Kaza

kh

sta

nN

orw

ay

Ma

ca

o-C

hin

aS

loven

iaR

om

ania

Pola

nd

Ind

one

sia

Un

ited

Sta

tes

Esto

nia

Ch

inese

Taip

ei

Sha

ngh

ai-

Ch

ina

Belg

ium

Tu

rke

yG

reece

Fra

nce

Hung

ary

Serb

iaS

lovak R

epu

blic

Vie

tna

mC

ana

da

Ne

the

rla

nds

OE

CD

ave

rag

eP

ort

ug

al

Uru

gua

yC

roa

tia

Isra

el

Cze

ch R

ep

ub

licA

ustr

alia

Un

ited

Kin

gd

om

Sw

itze

rla

nd

Germ

any

Arg

en

tin

aD

enm

ark

Me

xic

oN

ew

Ze

ala

nd

Tu

nis

iaIr

ela

nd

Ho

ng

Kon

g-C

hin

aS

pa

inB

razil

Ja

pa

nK

ore

aIt

aly

Peru

Austr

iaL

iech

tenste

inC

osta

Ric

aC

hile

Luxe

mb

ou

rgC

olo

mb

ia

Sc

ore

-po

int

dif

fere

nc

e (

bo

ys

-gir

ls)

Gender differences in mathematics performance Fig I.2.25

Boys perform better than girls

Girls perform better than boys

15

Page 16: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

Resources make a difference…

…but only up to a point

16

Page 17: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

Spending per student from the age of 6 to 15 and

mathematics performance in PISA 2012

Slovak Republic

Czech RepublicEstonia

Israel

Poland

Korea

Portugal

New Zealand

CanadaGermany

Spain

France

Italy

Singapore

Finland

Japan

Slovenia Ireland

Iceland

Netherlands

Sweden

Belgium

UK

AustraliaDenmark

United States

Austria

Norway

Switzerland

Luxembourg

Viet Nam

Jordan

Peru

Thailand

Malaysia

Uruguay

Turkey

Colombia

Tunisia

MexicoMontenegro

Brazil

Bulgaria

Chile

CroatiaLithuania

Latvia

Hungary

Shanghai-China

R² = 0.01

R² = 0.37

300

350

400

450

500

550

600

650

0 20 000 40 000 60 000 80 000 100 000 120 000 140 000 160 000 180 000 200 000

Ma

the

ma

tic

s p

erf

orm

an

ce

(sc

ore

po

ints

)

Average spending per student from the age of 6 to 15 (USD, PPPs)

Cumulative expenditure per student less than USD 50 000

Cumulative expenditure per student USD 50 000 or more

Fig IV.1.817

Page 18: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

Hong Kong-China

Brazil

Uruguay

Croatia

Latvia

Chinese Taipei

Thailand

Bulgaria

Jordan

Macao-China

UAE

Argentina

Indonesia

Kazakhstan

Peru

Costa RicaMontenegro

Tunisia

Qatar

Singapore

Colombia

MalaysiaSerbia

Romania

Viet Nam

Shanghai-China

USA

Poland

New Zealand

Greece

UK

Estonia

Finland

Slovak Rep.

Luxembourg

Germany

AustriaFrance

Japan

TurkeySweden Hungary

AustraliaIsrael

Canada

Ireland

Chile

Belgium

SpainDenmark

Switzerland

Iceland

Slovenia

PortugalNorway

Mexico

Korea

Italy

R² = 0.19

300

350

400

450

500

550

600

650

700

-0.500.511.5

Ma

the

ma

tic

s p

erf

orm

an

ce

(sc

ore

po

ints

)

Equity in resource allocation(index points)

Countries with better performance in mathematics tend

to allocate educational resources more equitably

Greater

equityLess

equity

Adjusted by per capita GDP

Fig IV.1.11

30% of the variation in math performance across OECD countries is explained by the degree of similarity of

educational resources between advantaged and disadvantaged schools

OECD countries tend to allocate at least an equal, if not a larger, number of teachers per student to disadvantaged schools; but disadvantaged schools tend to have great difficulty in attracting qualified teachers.

Page 19: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

Video series on

Strong Performers and

Successful Reformers in

Education

http://www.pearsonfoundation.org/oecd

Page 20: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

What’s next?

PISA 2015

20

Page 21: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

• Main subject: Science

• Number of participants : 72

• Field trials in 2014

• Main survey 2015

• Results released in December 2016

PISA 201521

Page 22: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

• Engagement of all is important:

– Policy-makers

– Teachers and Schools

– Students and Parents

– Media

– Research community

22

Page 23: Use of PISA in quality improvement polices – Richard Yelland, OECD Head of Policy Advice and Implementation Division

Thank you !

Find out more about PISA at www.pisa.oecd.org

• National and international publications

• The complete micro-level database

With acknowledgements to the PISA team

Email: [email protected]

[email protected]