practice and research in education: how can we make both better, and better aligned?
DESCRIPTION
Practice and research in education: How can we make both better, and better aligned?. Robert Coe @ ProfCoe ResearchED 2013, Dulwich College, 7 Sept 2013. Improving practice and research. Problems with research and evidence C an research tell us what works? - PowerPoint PPT PresentationTRANSCRIPT
Practice and research in education:How can we make both better, and better aligned?
Robert Coe @ProfCoe
ResearchED 2013, Dulwich College, 7 Sept 2013
∂
Improving practice and research
Problems with research and evidence Can research tell us what works? How can practice be improved?
– Think hard about learning– Invest in good CPD– Evaluate teaching quality– Evaluate impact of changes
2
Improving Education: A triumph of hope over experiencehttp://www.cem.org/attachments/publications/ImprovingEducation2013.pdf
Problems with research and evidence
3
∂
Problems with evidence
Evidence can be found to support any position in education
Ofsted asks schools to produce evidence to demonstrate that PP spending has narrowed the gap (even though some of it may not have)
DfE misunderstands/misuses evidence (relative gaps as percentage difference; small changes with small samples)
4
∂
Problems with research
Quality varies, but a lot is not very good Quality really matters How do you know who or what to trust? Academic papers are inaccessible Academic debates are (mostly) pointless Peer review doesn’t work
5
∂
Small positives
Impact agenda requires ‘public benefit’ EEF: funding for high quality evaluation Recurrent policy interest in Evidence-Based
Education (see http://www.cem.org/evidence-based-education/introduction)
Social media & internet gives instant critique, debate, interaction
6
Evidence about the effectiveness of different strategies
7
∂
Toolkit of Strategies to Improve Learning
The Sutton Trust-EEF Teaching and Learning Toolkit http://www.educationendowmentfoundation.org.uk/toolkit/
∂
Impact vs cost
Cost per pupil
Eff
ect
Siz
e (
mon
ths
gain
)
£00
8
£1000
Meta-cognitive
Peer tutoringEarly Years
1-1 tuitionHomework (Secondary)
Mentoring
Summer schools After
school
AspirationsPerformance pay
Teaching assistants
Smaller classes
Ability grouping
Promising May be
worth it
Notworth
it
Feedback
Phonics
Homework (Primary)
CollaborativeSmall gp
tuition Parental involvement
Individualised learning
ICT
Behaviour
Social
www.educationendowmentfoundation.org.uk/toolkit
∂
Some things that are popular or widely thought to be effective probably can’t improve learning– Ability grouping (setting); After-school clubs;
Teaching assistants; Smaller classes; Performance pay; Raising aspirations
Some things look ‘promising’– Effective feedback; Meta- cognitive and self
regulation strategies; Peer tutoring/peer‐assisted learning strategies; Homework
Key messages
∂
Clear, simple advice:
Choose from the top left Go back to school and do it
11
For every complex problem there is an answer that is clear, simple, and wrong
H.L. Mencken
∂
12
Why not? We have been doing some of these things for a
long time, but have generally not seen improvement
Research evidence is problematic– Sometimes the existing evidence is thin– Research studies may not reflect real life– Context and ‘support factors’ may matter
Implementation is problematic– We may think we are doing it, but are we doing it right?– We do not know how to get large groups of teachers
and schools to implement these interventions in ways that are faithful, effective and sustainable
So how might practice be improved?
13
∂
Four steps to improvement
Think hard about learning Invest in good professional development Evaluate teaching quality Evaluate impact of changes
1. Think hard about learning
∂
Impact vs cost
Cost per pupil
Eff
ect
Siz
e (
mon
ths
gain
)
£00
8
£1000
Meta-cognitive
Peer tutoringEarly Years
1-1 tuitionHomework (Secondary)
Mentoring
Summer schools After
school
AspirationsPerformance pay
Teaching assistants
Smaller classes
Ability grouping
Promising May be
worth it
Notworth
it
Feedback
Phonics
Homework (Primary)
CollaborativeSmall gp
tuition Parental involvement
Individualised learning
ICT
Behaviour
Social
www.educationendowmentfoundation.org.uk/toolkit
∂
Poor Proxies for Learning Students are busy: lots of work is done
(especially written work) Students are engaged, interested, motivated Students are getting attention: feedback,
explanations Classroom is ordered, calm, under control Curriculum has been ‘covered’ (ie presented to
students in some form) (At least some) students have supplied correct
answers (whether or not they really understood them or could reproduce them independently)
17
∂
Learning happens when people have
to think hard
A simple theory of learning
2. Invest in effective CPD
∂
How do we get students to learn hard things?
Eg Place value Persuasive
writing Music
composition Balancing
chemical equations
• Explain what they should do• Demonstrate it• Get them to do it (with
gradually reducing support)• Provide feedback • Get them to practise until it is
secure• Assess their skill/
understanding
∂
How do we get teachers to learn hard things?
Eg Using formative
assessment Assertive
discipline How to teach
algebra
• Explain what they should do
∂
Intense: at least 15 contact hours, preferably 50 Sustained: over at least two terms Content focused: on teachers’ knowledge of
subject content & how students learn it Active: opportunities to try it out & discuss Supported: external feedback and networks to
improve and sustain Evidence based: promotes strategies
supported by robust evaluation evidence
What CPD helps learners?
3. Evaluate teaching quality
∂
Classroom observation: The new Brain Gym?
Validity evidence– Are observation ratings really a reflection of
teaching quality?
Impact evaluation– Does the process of observation and feedback
lead to improvement?– In what, how much and for what cost?
24
∂
Validity evidence Do observation ratings correspond with other indicators of
teaching quality or effectiveness?– Student learning gains– Student ratings– Peer (teacher) perceptions– Self ratings
Are they consistent? – Across occasions– Across raters
Are ratings influenced by spurious confounds– Charisma– Confidence– Subject matter– Students’ behaviour– Time of day
25
∂
Does observation improve teaching?
Need studies with– Clearly defined intervention– High quality outcome measures (student learning)– Good control of counterfactual (eg RCT)– Adequate sample– Measures of sustained impact
Just one would be nice …
26
4. Evaluate impact of changes
∂
1. Wait for a bad year or choose underperforming schools to start with. Most things self-correct or revert to expectations (you can claim the credit for this).
2. Take on any initiative, and ask everyone who put effort into it whether they feel it worked. No-one wants to feel their effort was wasted.
3. Define ‘improvement’ in terms of perceptions and ratings of teachers. DO NOT conduct any proper assessments – they may disappoint.
4. Only study schools or teachers that recognise a problem and are prepared to take on an initiative. They’ll probably improve whatever you do.
Mistaking School Improvement (1)(Coe, 2009)
∂
5. Conduct some kind of evaluation, but don’t let the design be too good – poor quality evaluations are much more likely to show positive results.
6. If any improvement occurs in any aspect of performance, focus attention on that rather than on any areas or schools that have not improved or got worse (don’t mention them!).
7. Put some effort into marketing and presentation of the school. Once you start to recruit better students, things will improve.
Mistaking School Improvement (2) (Coe, 2009)
∂
Clear, well defined, replicable intervention
Good assessment of appropriate outcomes
Well-matched comparison group
EEF DIY
Evaluatio
n Guide
Key elements of good evaluation
∂
Summary … A lot of educational research is rubbish, but
some is very good: relevant and rigorous Four steps to improve practice:
– Think hard about learning– Invest in good CPD– Evaluate teaching quality (but not with dodgy
observation)– Evaluate impact of changes
[email protected] @ProfCoe
www.cem.org