29th 30thOF SEPTEMBER 2016

INTERNATIONAL CONFERENCE

USE OF THE EDUCATION QUALITY RESEARCH RESULTS: PROBLEMS AND PROSPECTS

IMPROVING SCHOOL PERFORMANCE: A CAUTIONARY TALE FROM CANADA WITH A HAPPY ENDING

T. SCOTT MURRAY

SCHOOLS GENERATE THREE TYPES OF OUTCOMES:

THEY IMPART:

Technical skills and knowledge

Cognitive skills that are needed to apply technical

skills and knowledge

Attitudes and beliefs that encourage the application

of these skills in work

… BUT THE GAME HAS CHANGED:

Schools main job used to be select the top performing students who would

go on to post-secondary education, the rest got basic skills

Now education systems need to impart higher levels of skills to a much higher proportion of

the population to prepare them for the demands of post-secondary education

To achieve this goal schools need to improve the literacy

and numeracy skills of students

IRONICALLY THE SOVIET SYSTEM REALIZED

THE CENTRAL IMPORTANCE OF LITERACY IN 1948

THE DEFINITION OF LITERACY HAS

EVOLVED CONSIDERABLY OVER THE PAST 30 YEARS

Advances in theory have have provided a means to understand the determinants of task difficulty

The improved understanding of task difficulty has supported the development of much more reliable

assessments

Literacy is now defined as a continuum than moves from learning

to read to reading to learn

LEARNING TO READ TO READING TO LEARN: WE NEED MORE

PEOPLE ABLE TO FUNCTION AT LEVEL 3 IN THE PRE-FRONTAL CORTEX

Students and workers need the cognitive skills to support efficient acquisition of technical skills and knowledge and then to apply these skills in work to globally competitive levels

0

Level 1

225 275 325 375 500

Level 2 Level 3 Level 4 Level 5

Learning to read Reading to learnProficiency dominated by mechanics of reading Proficiency dominated by cognitive strategies

THE THRESHOLD BETWEEN LEARNING TO READ AND READING

TO LEARN FALLS AT A CRITICAL THRESHOLD IN BLOOMS REVISED TAXONOMY

Creating

Evaluating

Analyzing

Applying

Understanding

Remembering

The critical boundary between Level 2 and 3

…AND IN THE SOLO TAXONOMY

Prestructural Unistructural Multistructural Relational Extended abstract

QUANTITATIVE PHASE QUANTITATIVE PHASE

misses point

identifydo simpleprocedure

enumerate describelist combinedo algorithms

compare/contrastexplain causesanalyserelateapply

hypothesizereflect

CANADA HAS INVESTED HEAVILY IN USING

ASSESSMENT TO IMPROVE LEARNING OUTCOMES:

CANADA:

Participated in PIRLS, TIMSS and PISA to provide assessments

of literacy, math and science outcomes

Established a national assessment of literacy,

math and science outcomes

Established provincial assessments of literacy, math

and science outcomes at various grades

THE RESULTS OF THESE ASSESSMENTS REVEAL THAT ROUGHLY HALF

OF SECONDARY GRADUATES DO NOT REACH LEVEL 3. THESE RESULTS HAVE

PRECIPITATED LARGE INVESTMENTS BUT PISA SCORES CONTINUE TO FALL

540

535

530

525

520

515

510

505

500

2000 2003 2006 2009 2012

534

528 527

524 523

534

529

525

Reading

Science

OUR ANALYSIS SUGGESTS A DIFFERENT APPROACH

TO ASSESSMENT IS NEEDED…KEY WEAKNESSES INCLUDE:

All the current assessment resultstell teachers is that a significant number of them are doing a bad job

Assessment results only arrive when the assessed students have moved to the next grade

The assessment results are seldom adjusted to account for differences in the characteristics of the students being served or for factors over which teachers and schools have control

Assessments results are not reliable for individual students so do not identify the students are struggling

Assessments provide relatively little information about students in the lower ranges of the scales, the very students who need help

ACCOUNTABILITY AND PERVERSE EFFECTS:

SOMETIMES THE RESULT IS NOT AS EXPECTED:

THERE IS NO MECHANISM

TO TRANSFER INNOVATIONS ONCE IDENTIFIED

SCHOOL PROFILE

FOR ONTARIO

ONTARIO UNDERPERFORMS ACROSS THE BOARD…

The analysis of socioeconomic gradients

(Figure 3) indicated that Ontario students scored well below

their counterparts in Quebec and Alberta, across the full range

of SES. The school profile above shows that the SES intake of

most schools in Ontario is above the OECD mean. However,

the majority of Ontario’s schools scored below the regression

line, indicating that they were not performing as well as other

Canadian schools with comparable student intake. Thus,

Ontario’s relatively low overall performance is not attributable

to a few low SES schools with low performance. Rather, it is

associated with a more general pattern of slightly lower than

expected performance among the majority of its schools.

Source: J. D. Willms, UNB

700

600

500

400

V

IV

III

II

I

Level

Socioeconomic Status

-1 0 1

Rea

ding

Sco

re

A NEW GENERATION OF ASSESSMENTS

IS YIELDING THE LONG AWAITED SCHOOL IMPROVEMENT:

ASSESSMENTS THAT GIVE INDIVIDUAL RESULTS IN REAL TIME AND IDENTIFY STRUGGLING STUDENTS I.E.:

The Early Years Evaluation that assesses physical, social and

cognitive skills in Grades 1 – 4 in real time

The Bow Valley College web-based adaptive real time assessment of the reading

mechanics that underlie learning to read

EARLY YEARS EVALUATION

COGNITIVE SCORES BY LETTER NAMING FLUENCY

Lett

er-N

amin

g Fl

uenc

y (D

IBEL

S)

Kindergarten Year

EYE (above 2)

EYE (1 – 2)

EYE (below 1)

Sep Oct Nov Dec Jan Feb Mar Apr May Jun

50

40

30

20

10

0

BOW VALLEY COLLEGE FOUNDATION ASSESSMENT

1.0

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0

Vocabulary Real Word recognition

Pseudo-word recognition

Spelling Digit Span

SUMMARY OF DECODING AND VOCABULARY LEVEL BY LATENT CLASS

Latent Class

Very limited

Limited

Adequate

A

PRINT SKILLS

COMPREHENSION SKILLS

A B C D

A NEW GENERATION OF ASSESSMENTS

IS YIELDING THE LONG AWAITED SCHOOL IMPROVEMENT:

The SkillPlan teacher in-service literacy instruction training that provides teachers with a deep knowledge of how to help struggling students

Professional learning communities that serve to transfer promising practices and awards that acknowledge innovation

Extra resources for schools serving disadvantaged populations

Provides them with information they need in real time to help struggling students

Provides the skills needed to help struggling students

Acknowledges innovators

Provides measues that can be aggregated to higher levels

THIS NEW GENERATION OF ASSESSMENTS:Values teachers

Transfers innovations rapidly

Rewards teachers serving disadvantaged populations

2016

THANK YOU FOR YOUR ATTENTION!