![Page 1: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d745503460f94a54696/html5/thumbnails/1.jpg)
A randomised controlled trial to improve writing quality during the
transition between primary and secondary school
Natasha Mitchell, Research FellowHannah Buckley, Statistician
York Trials Unit, University of York
![Page 2: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d745503460f94a54696/html5/thumbnails/2.jpg)
Co-authors: Ainsworth, H., Heaps, C., Hewitt, C., Jefferson, L., Torgerson, C., & Torgerson, D.
Funded by: Education Endowment Foundation
Intervention delivery partner: Calderdale Excellence Partnership
![Page 3: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d745503460f94a54696/html5/thumbnails/3.jpg)
• Protocol available at:http://educationendowmentfoundation.org.uk/uploads/
pdf/Transitions_-_Calderdale.pdf
• Full report available at: http://educationendowmentfoundation.org.uk/uploads/
pdf/EEF_Evaluation_Report_-_Improving_Writing_Quality_-_May_2014_v2.pdf
![Page 4: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d745503460f94a54696/html5/thumbnails/4.jpg)
Overview
• EEF Transitions Round: • Evaluate 3 different writing interventions with
aim to improve children’s writing skills during transition from primary school to secondary school
• Individually Randomised (Discover)
• Split Plot Design (Exeter)
• Cluster Randomised (Calderdale)
![Page 5: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d745503460f94a54696/html5/thumbnails/5.jpg)
Background
• Many children leave primary school without achieving a Level 4 or above in writing (Dept of Education, 2013)
• The intervention was aimed particularly at children:– from disadvantaged backgrounds – not achieving a ‘secure Level 4’ in year 6
• Not reaching Level 4 at the end of KS2 linked to progress in secondary school (Dept of Education, 2014)
![Page 6: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d745503460f94a54696/html5/thumbnails/6.jpg)
• Using memorable experiences and the Self-Regulated Strategy Development (SRSD) programme.
• SRSD strategy developed in America
• Writing process model involves:• Plan• Draft• Edit • Revise
• Key aspects include self monitoring & goal setting to provide pupils with ownership over improving their writing.
Background
![Page 7: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d745503460f94a54696/html5/thumbnails/7.jpg)
What is the effectiveness of the Improving Writing Quality programme compared with “business as usual” on the writing skills of
participating children?
Research Question
![Page 8: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d745503460f94a54696/html5/thumbnails/8.jpg)
• Pragmatic cluster randomised controlled trial.
• Approximately 24 schools (feeding into 3 secondary schools) randomised on a 1:1 ratio to intervention and control groups
Design
![Page 9: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d745503460f94a54696/html5/thumbnails/9.jpg)
Intervention Group •The intervention was offered to all pupils in Year 6 in 2013.•Pupils predicted to achieve Level 3 or an insecure Level 4 in English (based on teacher assessment) continued to receive intervention in Year 7 (Autumn term).•Memorable experiences for pupils in Year 6.•Pupils used SRSD programme to embed these experiences into their writing.•Professional development for primary and secondary school teachers in key elements of the writing intervention SRSD programme (including discussion, modelling and planning).
Control Group•Business as Usual in 2013.•Primary schools trained in SRSD in 2014.
Design
![Page 10: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d745503460f94a54696/html5/thumbnails/10.jpg)
• Hosted a information meeting to explain intervention and evaluation, hosted by delivery partner and evaluators in February 2013.
• Recruited: 23 primary schools; 3 secondary schools.
• Randomised primary schools in March 2013.
• Post test conducted in December 2013 after transition to secondary school.
Design
![Page 11: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d745503460f94a54696/html5/thumbnails/11.jpg)
Outcome measure
•Progress in English (PiE) 11: Second Edition Long Form (LF) Test, GL Assessment
Primary outcome
•Combined score on extended writing tasks
Secondary outcome
•Combined score on reading tasks and combined score on the spelling and grammar tasks
Outcomes
![Page 12: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d745503460f94a54696/html5/thumbnails/12.jpg)
• Cluster trial
• Pupils from same primary school transitioning to different secondary schools
• Cross-classified multilevel model used to account for this
• Adjustment was made for baseline predicted KS2 writing level alongside other predictors of attainment (gender, FSM status, EAL status, month of birth)
• ITT
Analysis Methods
![Page 13: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d745503460f94a54696/html5/thumbnails/13.jpg)
• 209 pupils meeting inclusion criteria randomised to control and 223 to intervention
• Following transition 130 control and 154 intervention pupils
• Primary analysis conducted on 119 control and 142 intervention pupils
Results
![Page 14: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d745503460f94a54696/html5/thumbnails/14.jpg)
• A significant increase of 2.53 marks in writing score (p=0.002, ES=0.74) was found for struggling writers in the intervention group compared with the control group.
• Although this increase was larger when analysis was conducted on the FSM subgroup, there was no significant difference between FSM and non-FSM children.
• There was no evidence of a difference between the randomised groups in reading or spelling and grammar score.
Results
![Page 15: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d745503460f94a54696/html5/thumbnails/15.jpg)
Strengths•Study conducted to CONSORT standards.•Used blind marking of test papers.•Good relationship between CEP & schools therefore had high completion.•Analysis plan produced a priori.•Analysis accounted for clustering.•Followed principles of intention to treat.
Strengths & Limitations
![Page 16: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d745503460f94a54696/html5/thumbnails/16.jpg)
Limitations•During the trial there was ‘intervention development’.•Anglicising the intervention. •Refresher training to secondary schools incorporated ‘lessons learnt’ and adaptations.•Imperfect implementation in secondary schools.•Based in single geographical area.
Strengths & Limitations
![Page 17: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d745503460f94a54696/html5/thumbnails/17.jpg)
• This intervention appears to be highly effective at improving children’s writing skills despite being used for a relatively short period of time.
• No statistically significant difference found between FSM and non-FSM children.
Conclusions
![Page 18: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d745503460f94a54696/html5/thumbnails/18.jpg)
• Larger-scale effectiveness RCT of the intervention to confirm findings of this trial.
• Multiple geographical areas to be included.
• Developing an ‘Anglicised’ version for use in primary schools.
What next?
![Page 19: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d745503460f94a54696/html5/thumbnails/19.jpg)
Questions?
Thank you