practice and research in education: how can we make both better, and better aligned?

31
Practice and research in education: How can we make both better, and better aligned? Robert Coe @ProfCoe ResearchED 2013, Dulwich College, 7 Sept 2013

Upload: gabi

Post on 07-Jan-2016

24 views

Category:

Documents


0 download

DESCRIPTION

Practice and research in education: How can we make both better, and better aligned?. Robert Coe @ ProfCoe ResearchED 2013, Dulwich College, 7 Sept 2013. Improving practice and research. Problems with research and evidence C an research tell us what works? - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Practice and research in education: How can we make both better,  and better aligned?

Practice and research in education:How can we make both better, and better aligned?

Robert Coe @ProfCoe

ResearchED 2013, Dulwich College, 7 Sept 2013

Page 2: Practice and research in education: How can we make both better,  and better aligned?

Improving practice and research

Problems with research and evidence Can research tell us what works? How can practice be improved?

– Think hard about learning– Invest in good CPD– Evaluate teaching quality– Evaluate impact of changes

2

Improving Education: A triumph of hope over experiencehttp://www.cem.org/attachments/publications/ImprovingEducation2013.pdf

Page 3: Practice and research in education: How can we make both better,  and better aligned?

Problems with research and evidence

3

Page 4: Practice and research in education: How can we make both better,  and better aligned?

Problems with evidence

Evidence can be found to support any position in education

Ofsted asks schools to produce evidence to demonstrate that PP spending has narrowed the gap (even though some of it may not have)

DfE misunderstands/misuses evidence (relative gaps as percentage difference; small changes with small samples)

4

Page 5: Practice and research in education: How can we make both better,  and better aligned?

Problems with research

Quality varies, but a lot is not very good Quality really matters How do you know who or what to trust? Academic papers are inaccessible Academic debates are (mostly) pointless Peer review doesn’t work

5

Page 6: Practice and research in education: How can we make both better,  and better aligned?

Small positives

Impact agenda requires ‘public benefit’ EEF: funding for high quality evaluation Recurrent policy interest in Evidence-Based

Education (see http://www.cem.org/evidence-based-education/introduction)

Social media & internet gives instant critique, debate, interaction

6

Page 7: Practice and research in education: How can we make both better,  and better aligned?

Evidence about the effectiveness of different strategies

7

Page 8: Practice and research in education: How can we make both better,  and better aligned?

Toolkit of Strategies to Improve Learning

The Sutton Trust-EEF Teaching and Learning Toolkit http://www.educationendowmentfoundation.org.uk/toolkit/

Page 9: Practice and research in education: How can we make both better,  and better aligned?

Impact vs cost

Cost per pupil

Eff

ect

Siz

e (

mon

ths

gain

)

£00

8

£1000

Meta-cognitive

Peer tutoringEarly Years

1-1 tuitionHomework (Secondary)

Mentoring

Summer schools After

school

AspirationsPerformance pay

Teaching assistants

Smaller classes

Ability grouping

Promising May be

worth it

Notworth

it

Feedback

Phonics

Homework (Primary)

CollaborativeSmall gp

tuition Parental involvement

Individualised learning

ICT

Behaviour

Social

www.educationendowmentfoundation.org.uk/toolkit

Page 10: Practice and research in education: How can we make both better,  and better aligned?

Some things that are popular or widely thought to be effective probably can’t improve learning– Ability grouping (setting); After-school clubs;

Teaching assistants; Smaller classes; Performance pay; Raising aspirations

Some things look ‘promising’– Effective feedback; Meta- cognitive and self

regulation strategies; Peer tutoring/peer‐assisted learning strategies; Homework

Key messages

Page 11: Practice and research in education: How can we make both better,  and better aligned?

Clear, simple advice:

Choose from the top left Go back to school and do it

11

For every complex problem there is an answer that is clear, simple, and wrong

H.L. Mencken

Page 12: Practice and research in education: How can we make both better,  and better aligned?

12

Why not? We have been doing some of these things for a

long time, but have generally not seen improvement

Research evidence is problematic– Sometimes the existing evidence is thin– Research studies may not reflect real life– Context and ‘support factors’ may matter

Implementation is problematic– We may think we are doing it, but are we doing it right?– We do not know how to get large groups of teachers

and schools to implement these interventions in ways that are faithful, effective and sustainable

Page 13: Practice and research in education: How can we make both better,  and better aligned?

So how might practice be improved?

13

Page 14: Practice and research in education: How can we make both better,  and better aligned?

Four steps to improvement

Think hard about learning Invest in good professional development Evaluate teaching quality Evaluate impact of changes

Page 15: Practice and research in education: How can we make both better,  and better aligned?

1. Think hard about learning

Page 16: Practice and research in education: How can we make both better,  and better aligned?

Impact vs cost

Cost per pupil

Eff

ect

Siz

e (

mon

ths

gain

)

£00

8

£1000

Meta-cognitive

Peer tutoringEarly Years

1-1 tuitionHomework (Secondary)

Mentoring

Summer schools After

school

AspirationsPerformance pay

Teaching assistants

Smaller classes

Ability grouping

Promising May be

worth it

Notworth

it

Feedback

Phonics

Homework (Primary)

CollaborativeSmall gp

tuition Parental involvement

Individualised learning

ICT

Behaviour

Social

www.educationendowmentfoundation.org.uk/toolkit

Page 17: Practice and research in education: How can we make both better,  and better aligned?

Poor Proxies for Learning Students are busy: lots of work is done

(especially written work) Students are engaged, interested, motivated Students are getting attention: feedback,

explanations Classroom is ordered, calm, under control Curriculum has been ‘covered’ (ie presented to

students in some form) (At least some) students have supplied correct

answers (whether or not they really understood them or could reproduce them independently)

17

Page 18: Practice and research in education: How can we make both better,  and better aligned?

Learning happens when people have

to think hard

A simple theory of learning

Page 19: Practice and research in education: How can we make both better,  and better aligned?

2. Invest in effective CPD

Page 20: Practice and research in education: How can we make both better,  and better aligned?

How do we get students to learn hard things?

Eg Place value Persuasive

writing Music

composition Balancing

chemical equations

• Explain what they should do• Demonstrate it• Get them to do it (with

gradually reducing support)• Provide feedback • Get them to practise until it is

secure• Assess their skill/

understanding

Page 21: Practice and research in education: How can we make both better,  and better aligned?

How do we get teachers to learn hard things?

Eg Using formative

assessment Assertive

discipline How to teach

algebra

• Explain what they should do

Page 22: Practice and research in education: How can we make both better,  and better aligned?

Intense: at least 15 contact hours, preferably 50 Sustained: over at least two terms Content focused: on teachers’ knowledge of

subject content & how students learn it Active: opportunities to try it out & discuss Supported: external feedback and networks to

improve and sustain Evidence based: promotes strategies

supported by robust evaluation evidence

What CPD helps learners?

Page 23: Practice and research in education: How can we make both better,  and better aligned?

3. Evaluate teaching quality

Page 24: Practice and research in education: How can we make both better,  and better aligned?

Classroom observation: The new Brain Gym?

Validity evidence– Are observation ratings really a reflection of

teaching quality?

Impact evaluation– Does the process of observation and feedback

lead to improvement?– In what, how much and for what cost?

24

Page 25: Practice and research in education: How can we make both better,  and better aligned?

Validity evidence Do observation ratings correspond with other indicators of

teaching quality or effectiveness?– Student learning gains– Student ratings– Peer (teacher) perceptions– Self ratings

Are they consistent? – Across occasions– Across raters

Are ratings influenced by spurious confounds– Charisma– Confidence– Subject matter– Students’ behaviour– Time of day

25

Page 26: Practice and research in education: How can we make both better,  and better aligned?

Does observation improve teaching?

Need studies with– Clearly defined intervention– High quality outcome measures (student learning)– Good control of counterfactual (eg RCT)– Adequate sample– Measures of sustained impact

Just one would be nice …

26

Page 27: Practice and research in education: How can we make both better,  and better aligned?

4. Evaluate impact of changes

Page 28: Practice and research in education: How can we make both better,  and better aligned?

1. Wait for a bad year or choose underperforming schools to start with. Most things self-correct or revert to expectations (you can claim the credit for this).

2. Take on any initiative, and ask everyone who put effort into it whether they feel it worked. No-one wants to feel their effort was wasted.

3. Define ‘improvement’ in terms of perceptions and ratings of teachers. DO NOT conduct any proper assessments – they may disappoint.

4. Only study schools or teachers that recognise a problem and are prepared to take on an initiative. They’ll probably improve whatever you do.

Mistaking School Improvement (1)(Coe, 2009)

Page 29: Practice and research in education: How can we make both better,  and better aligned?

5. Conduct some kind of evaluation, but don’t let the design be too good – poor quality evaluations are much more likely to show positive results.

6. If any improvement occurs in any aspect of performance, focus attention on that rather than on any areas or schools that have not improved or got worse (don’t mention them!).

7. Put some effort into marketing and presentation of the school. Once you start to recruit better students, things will improve.

Mistaking School Improvement (2) (Coe, 2009)

Page 30: Practice and research in education: How can we make both better,  and better aligned?

Clear, well defined, replicable intervention

Good assessment of appropriate outcomes

Well-matched comparison group

EEF DIY

Evaluatio

n Guide

Key elements of good evaluation

Page 31: Practice and research in education: How can we make both better,  and better aligned?

Summary … A lot of educational research is rubbish, but

some is very good: relevant and rigorous Four steps to improve practice:

– Think hard about learning– Invest in good CPD– Evaluate teaching quality (but not with dodgy

observation)– Evaluate impact of changes

[email protected] @ProfCoe

www.cem.org