Running a School on Data: the role of Value-Added Data

Course: Using CEM data in Practice

Day 2 Session 4

Wed 30th May 2012

Peter Hendry: CEM Consultant

Peter.Hendry@cem.dur.ac.uk

CASE STUDY : MidYIS Year 7 to GCSE Raw Residuals

Comments?

-0.7

0.1

0.6

0.8

0.2

-0.3

0.6

-0.2

-0.6

0.6

0.3

0.4

0.3

0.6

-0.1

-0.2

1.5

0.8

0.8

-0.1

0.0

-3.0 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0

Short Course Welsh

Short Course Physical Education

Welsh

Sociology

Science

Religious Studies

Physical Education

Music

Mathematics

Home Economics

History

German

Geography

French

English Literature

English

Drama

Design & Technology

Business Studies

Art & Design

Additional Science

Average Raw Residual

MidYIS Year 7 2003/2004 to GCSE 2008 Value Added Analysis

Average Raw Residuals by Subject

School: 6734060

CASE STUDY : MidYIS Year 7 to GCSE Standardised Residuals

0.0

-0.1

0.5 0.5

1.1

-0.1 -0.1

0.40.2 0.3

0.20.4

-0.5

-0.1

0.4

-0.2

0.2

0.5 0.4

0.1

-0.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

Ad

ditio

na

l Sc

ien

ce

Art &

De

sig

n

Bu

sin

es

s Stu

die

s

De

sig

n &

Te

ch

no

log

y

Dra

ma

En

glis

h

En

glis

h L

itera

ture

Fre

nc

h

Ge

og

rap

hy

Ge

rma

n

His

tory

Ho

me

Ec

on

om

ics

Ma

the

ma

tics

Mu

sic

Ph

ysic

al E

du

ca

tion

Re

ligio

us

Stu

die

s

Sc

ien

ce

So

cio

log

y

We

lsh

Sh

ort C

ou

rse

Ph

ysic

al …

Sh

ort C

ou

rse

We

lsh

Ave

rag

e S

tan

dar

dis

ed R

esid

ual

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

2005 2006 2007 2008

Av

era

ge

Sta

nd

ard

ise

d R

es

idu

al

Exam Year

Statistical Process Control ChartNationally Represesentative Standardisation

GCSE Overall

Examination Year

CASE STUDY : MidYIS Year 7 to GCSE Standardised Residuals

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

0 50 100 150 200 250 300 350 400 450

Ave

rag

e S

tan

dar

dis

ed R

esid

ual

Rank

Distribution of Schools' Average Standardised Residuals (National Schools' Standardisation) with 95% Confidence Intervals: GCSE School Level

Year 7 To GCSE

CASE STUDY : MidYIS Year 7 to GCSE Standardised Residuals

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

2005 2006 2007 2008

Av

era

ge

Sta

nd

ard

ise

d R

es

idu

al

Exam Year

Statistical Process Control ChartNationally Represesentative Standardisation

GCSE Mathematics

Examination Year

CASE STUDY : MidYIS Year 7 to GCSE Standardised Residuals

-2

-1

0

1

2

3

4

0 40 80 120 160 200 240 280 320 360 400

Ave

rag

e S

tan

dar

dis

ed R

esid

ual

Rank

Distribution of Schools' Average Standardised Residuals (National Schools' Standardisation) with 95% Confidence Intervals: GCSE Mathematics

Year 7 to GCSE

CASE STUDY : MidYIS Year 7 to GCSE Standardised Residuals

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

8.0

50 60 70 80 90 100 110 120 130 140 150 160 170

GC

SE P

oint

s Sc

ore

MidYIS Test Score (Standardised on a Nationally Representative Sample)

Year 7 MidYIS Score 2003/2004 Against GCSE Mathematics 2008Yr 7 MidYIS to GCSE: all pupils

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

A* A B C D E F G

Mea

n S

tan

dar

dis

ed R

esid

ual

Grade

By grade

Grouped by GCSE Grade

Result

Mean Raw

Resid

Mean Stand

Residual CountA* 8 0A 7 1.17 0.94 8B 6 -0.03 -0.02 24C 5 0.04 0.03 36D 4 -0.61 -0.49 27E 3 -1.28 -1.03 18F 2 -1.88 -1.51 13G 1 -2.74 -2.21 13

total 139

Case Study:Maths All pupils

Case Study:Maths All pupils

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

D C B AM

ean

Sta

nd

ard

ised

Res

idu

al

Band

By quartile band

Grouped by Quartile Band

Bandlower bound

upper bound

Mean Raw

Residual

Mean Stand

Residual CountD 0 89.9 -0.98 -0.79 13C 89.9 100 -0.80 -0.65 32B 100 110.1 -0.39 -0.32 46A 110.1 200 -0.69 -0.55 48

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

A* A B C D E F G

Mea

n S

tan

dar

dis

ed R

esid

ual

Grade

By grade

GIRLS

Grouped by GCSE Grade

Result

Mean Raw

Resid

Mean Stand

Residual CountA* 8 0A 7 1.14 0.92 4B 6 -0.08 -0.06 11C 5 -0.16 -0.13 16D 4 -0.50 -0.41 14E 3 -1.28 -1.03 12F 2 -1.66 -1.34 6G 1 -2.64 -2.13 8

total 71

BOYS

-3

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

A* A B C D E F G

Mea

n S

tan

dar

dis

ed R

esid

ual

Grade

By grade

Grouped by GCSE Grade

Result

Mean Raw

Resid

Mean Stand

Residual CountA* 8 0A 7 1.19 0.96 4B 6 0.02 0.01 13C 5 0.19 0.16 20D 4 -0.72 -0.58 13E 3 -1.29 -1.04 6F 2 -2.06 -1.66 7G 1 -2.88 -2.32 5

total 68

-1.5

-1

-0.5

0

0.5

1

1.5

D C B A

Mea

n St

anda

rdis

ed R

esid

ual

Band

By quartile band

-1.5

-1

-0.5

0

0.5

1

1.5

D C B A

Mea

n St

anda

rdis

ed R

esid

ual

Band

By quartile bandGIRLS

Grouped by Quartile Band

Bandlower bound

upper bound

Mean Raw

Residual

Mean Stand

Residual CountD 0 89.9 -1.39 -1.12 6C 89.9 100 -1.13 -0.91 18B 100 110.1 -0.57 -0.46 26A 110.1 200 -0.42 -0.34 21

BOYS

Grouped by Quartile Band

Bandlower bound

upper bound

Mean Raw

Residual

Mean Stand

Residual CountD 0 89.9 -0.63 -0.51 7C 89.9 100 -0.38 -0.31 14B 100 110.1 -0.16 -0.13 20A 110.1 200 -0.89 -0.72 27

MidYIS Test

Score

Predicted GCSE Points

Actual GCSE Points

Raw Residual

Standardised

Residual

overall 105.8 4.8 4.2 -0.6 -0.5girls 105.2 4.8 4.0 -0.7 -0.6boys 106.5 4.9 4.3 -0.5 -0.4

AVERAGE SCORES

Case study: Alis value-added dataFour sets of VAD are available!

From average GCSE baseline:• all Alis cohort • type of Institution• Syllabus

From the baseline test• all Alis cohort

SPC Chart with confidence limits: WHOLE SCHOOL

All Alis Cohort: Institution

Syllabus

-0.3

-0.2

-0.1

-0.0

0.1

0.2

0.3

0.420

04

2005

2006

2007

2008

2009

2010

Ave

rage

Sta

ndar

dise

d R

esid

ual

Exam Year

-0.3

-0.2

-0.1

-0.0

0.1

0.2

0.3

0.4Final_Result

Using PARIS software: Baseline Test

Whole School

From your perspective, which set of VAD would you use for the different user groups? (Governors, HoDs, Media, Parents, SMT/SLT...)