a systematic review of interventions to support looked-after children in school

A systematic review of interventions to supportlooked-after children in schoolcfs_850 1..13

Kristin Liabo*, Kerry Gray† and David Mulcahy†*Research Fellow, Social Science Research Unit, Institute of Education, University of London and †Care Leaver,

Children’s Active Involvement Service, London, UK

ABSTRACT

A systematic review of interventions to support looked-after childrenin school included interventions that aimed to improve attainment, orprevent drop-out or exclusions, and those that aimed to reduceabsenteeism in the care population. Studies were critically appraisedand their results were considered. No study was found robust enoughto provide evidence on effectiveness, but promising interventionswere identified. The review highlights the lack of evidence in an areathat has received a lot of policy attention in the past few years. Futureevaluations need to be underpinned by lessons learned from existingevaluations, clearly defined theories and definitions, and by the viewsof professionals, researchers, policy-makers and young people in care.

Correspondence:Kristin Liabo,Social Science Research Unit,Institute of Education,University of London,18 Woburn Square,London WC1H 0NR,UKE-mail: [email protected]

Keywords: children in care system,education/training/supervision,effectiveness studies,evidence-based practice, foster care,looked-after children

Accepted for publication: January2012

INTRODUCTION

The educational outcomes for looked-after childrenare poor compared with the general population. Inthe UK, 12% of looked-after children gained five ormore A–C General Certificate of Secondary Educa-tions compared with 31% of those with free schoolmeals (Department for Education 2010a,b). Educa-tion is important to later health (Lleras-Muney2005), and dropping out of school can mean thatchildren lose out on enjoyable activities and engage-ment with peers.

Reasons for low achievement amongst the carepopulation have been identified in the care system,which has been seen not to prioritize education. Carehome environments may lack books, educationalmaterials or an appropriate study area (Hatton &Marsh 2007). Low expectations have been identified,resulting in looked-after children not receiving thesupport they need (Heath et al. 1994; Jackson &Sachdev 2001). Pre-care experiences of abuse andneglect also impact negatively on school attainment(Heath et al. 1994; Berridge 2007). Educational

achievement for this group tends to be poor interna-tionally, which indicates that much of the problem liesoutside of the system (Weyts 2004).

The effectiveness of social care interventions isunder-researched in general (Stevens et al. 2009) andresearch on looked-after children presents particularissues in terms of access and different gatekeepers(Heptinstall 2000).The objective of this review was toidentify effective evaluations of interventions aimed atsupporting looked-after children to stay in school orimprove their attainment.

This review was conducted in collaboration withlooked-after young people and care leavers.The groupis currently writing a separate paper describing theirinvolvement.

METHODS

The review included interventions targeted at childrenaged 10–15 in mainstream schools who had beenplaced by the authorities to live outside of their familysetting. The age limit was set because it encompassedthe transition from primary to secondary school.

bs_bs_banner

doi:10.1111/j.1365-2206.2012.00850.x

1 Child and Family Social Work 2012 © 2012 Blackwell Publishing Ltd

The intervention had to support the attainment orimprove the attendance of looked-after children andbe delivered to carers, children or professionals, orimplemented at a strategic level such as reorganizationof services or introduction of new procedures. Themain outcomes of interest were final-year exams,exclusion numbers, attendance numbers, literacy andnumeracy. This facilitated a manageable search strat-egy but limited the scope of the review as it excludedimportant outcomes such as mental health, motiva-tion and satisfaction. On the other hand, this puts ourfocus on outcomes currently set as UK governmenttargets, enabling us to identify studies of particularrelevance to these.

To be included, studies had to have made attemptsat measuring outcomes at baseline and follow-up.Thedecision to include studies without a comparisongroup reflects our expectations of available research.Also, uncontrolled studies have a value in identifyingpromising interventions and contain information onimplementation that is useful to the development ofinterventions. No minimum length to follow-up mea-sures was set, nor was there any lower limit on samplesize, because we wanted to identify all attempts atmeasuring the effect in this area. Study design wasfactored into the quality assessment.

Searches were conducted in March–June 2010 in:Educational Resource Information Centre, Disserta-tion Abstracts, International Bibliography of SocialSciences, Social Sciences Citation Index, ConferenceProceedings Citation Index, the Social PsychologicalEducational and Criminological Trials Register of theCampbell Collaboration, the Australian EducationIndex, the British Education Index, Social Policy andPractice, Social Services Abstracts, SociologicalAbstracts, Applied Social Sciences Index andAbstracts, Embase, Medline, PsychInfo, the CochraneCentral Register of Controlled Trials, CumulativeIndex to Nursing and Allied Health Literature andGoogle.

The following string of search terms served as thebasis for all searches: (looked-after child$ or adopt$ orresidential care or in care or foster$ or accommodatedchild$ or group home$1 or care order$1 or specialguardian$ or placement$1 or orphan$ or children$home or public care or custod$ or child$ welfare orunaccompanied asylum seeker$1 or welfare care)AND child$ or young person$1 or young peopleor boy$1 or girl$ or teenage$ or schoolchild$ oryouth$1 or adolescent$ or juvenile$ AND (educa-tion or school) adj4/near4 (attendance$1 or non-attendance$1 or absenteeism$1 or exclusion$ or

expel$ or suspension$1 or dropout$1 or drop out$1 ortruan$ or refus$ or phobia or disengag$ or attainmentor result$1 or exam$1 or complet$ or support$ or stayin school or stay in education or achieve$ or success).

Free text searches were conducted in English titlesand abstracts, and matched to subject headings orMeSH terms. No date or language limits were set.Theweb sites of the following organizations were scanned:Who CaresTrust, Fostering Network, PrincesTrust, ANational Voice, Brooks, NCH Action for Children,Barnardo’s, Voice of the Child in Care, Shaftesburyyoung people and the NSPCC. The web site ‘SocialPrograms that Work’ was searched, as was the bibliog-raphies of relevant reviews and studies. UK researcherswith expertise in looked-after children’s educationwere contacted. The authors of a Campbell review ofdrop-out interventions scanned their bibliography forstudies that focused on looked-after children.

All search hits were imported into EPPI-Reviewer 4(The EPPI-Centre, Social Science Research Unit,London, UK).This is a web-based electronic softwarefor managing systematic reviews informed by experi-ences from more than 200 reviews supported by orcarried out at the EPPI-Centre (http://eppi.ioe.ac.uk/cms). EPPI-Reviewer facilitates electronic importof all search hits and supports screening, criticalappraisal and synthesis of results. A screening tool wastested on selected hits. During this process, the pro-tocol was changed to include studies that had mea-sured attainment.The search had focused on drop-outinterventions, and some interventions to improveattainment may therefore have been missed. Forexample, our search strategy did not include theterm ‘tutor$’. Twenty-four per cent of all electronicabstracts were double screened and included studiescoded in EPPI-Reviewer.

Studies were tabulated on population, setting, inter-vention focus, outcomes, publication year and studyquality criteria (comparison group, sample size andhow outcomes were measured). Interventions variedconsiderably and meta-analysis was deemed inappro-priate because of the wide variety within programmesand study design. A descriptive review of each studywas conducted. The studies were then grouped intocategories based on the content and nature of theinterventions, grouping similar approaches together.The findings were considered under each of these.

RESULTS

The electronic search strategy identified 6514 studyabstracts (Fig. 1). The flow diagram of search hits

Supporting looked-after children in school K Liabo, K Gray and D Mulcahy


shows the key stages in the screening process, whichresulted in the inclusion of 11 studies.

Study characteristics

Of the 11 studies that filled the criteria of this review,6 were before-after evaluations without a comparisongroup, 4 were policy evaluations of implementationprocesses and outcomes, and 1 was a before-afterstudy with a non-equivalent comparison group. Thepolicy evaluations focused mainly on the process ofimplementation and the views of key stakeholders. It

may therefore seem like a misconception to assesstheir reliability in terms of their outcome findings.However, they addressed the impact of the interven-tions and therefore fit with the review’s aim to identifyall attempts at measuring impact in this field. Allincluded studies and their key characteristics areshown in Table 1.

From the content of these interventions, six catego-ries emerged: strategic interventions, pilot interven-tion of spending targeted money, residential school,community project, reading encouragement andtutoring.

Figure 1 Overview of search hits.



Table 1 Study characteristics

Study Sample sizePopulation, intervention,

comparison and outcomesLength tofollow-up

Strategic interventionsBerridge et al. (2009)

Looked after and learning:evaluation of the VirtualSchool Head pilot. London:Department of Children,Schools and Families.

Official statistics in each of the11 participating authorities2005–2008. For Key Stagesresults, the number per yearranged from 100 to 140children. For GCSE results,the number per year rangedfrom 315 to 405.

Survey of 31 children, 25 carers,21 designated teachers and 10social workers.

Population: school-aged looked-after children.Intervention: the Virtual School Head is a personresponsible for monitoring and tracking theeducation of looked-after children, and workingacross agencies to achieve this, within a localauthority.

No comparison group.Outcomes: achievement at Key Stages,

achievement in final year exams, exclusions, daysmissed at school and self-perceived success atschool.

The official statisticsincluded relate to theperiod of the pilot, asdoes the survey.

Harker et al. (2004)Taking care of education:an evaluation of theeducation of looked-afterchildren. London: TheNational Children’sBureau.

Official statistics in each of thethree participating authorities2000–2002. The number peryear ranged from 20 to 56children.

56 children participated inbaseline and follow-upinterviews (includingstandardized scales).

Population: school-aged looked-after children.Intervention: person responsible for workingwithin the local authority to improve looked-afterchildren’s education, and coordinate local effortsto do so.

No comparison group.Outcomes: achievement at Key Stages,

achievement in final year exams, exclusions, daysmissed at school, attitudes to education andsense of belonging at school, strengths anddifficulties questionnaire, and self-esteemquestionnaire (standardized).

The official statisticsincluded relate to theperiod of the pilot.Baseline data by thestandardizedquestionnaires werecollected in the firstyear of the pilot and 18months later (pilot stillongoing).

Zetlin et al. (2004)Improving educationoutcomes for children infoster care: intervention byan education liaison.Journal of Education forStudents Placed at Risk,9(4), 421–9.

60 treatments and 60 controlcases (non-equivalent atbaseline).

Population: children in foster care aged 5–17 years.Intervention: specialist education officer tosupport social workers on education cases.

Comparison: group randomly drawn from adifferent social services office. This office did nothave an education liaison officer.

Outcomes: Maths and reading test results, GradePoint Average, attendance, number of schools,and special educational need status.

One academic year afterthe education liaisonofficer was introduced.

Pilot of spending targeted moneyConnelly et al. (2008)

The educational attainmentof looked-after children –local authority pilotprojects: final researchreport. Edinburgh:Scottish GovernmentSocial Research.

Baseline collected for 722children and follow-upmeasures available for 636, butnot on all measures. Follow-upsample size for attendance andexclusion rates was 551.

Baseline and follow-up samplesizes for attainment were232 for reading, 229 for writingand 235 for Maths.

Interviews with 51 children, 14foster carers and 111professionals.

Population: school-aged looked-after children.Some interventions targeted pre-school children,but outcomes for these were not measured.

Intervention: providing authorities with moneyearmarked ‘looked-after children’s education’,spent in different ways across authorities.

No comparison group.Outcomes: attendance and exclusion rates,

national assessment results and nationalqualification results.

One academic year intothe pilot projectperiod.

Residential schoolJones & Lansdverk (2006)

Residential education:examining a new approachfor improving outcomesfor foster youth. Childrenand Youth Services Review,28, 1152–68.

School completion ratesprovided for 206 students.

At 6-month follow-up n = 42, at12-month follow-up n = 24.

Population: looked-after children with no prospectof returning to their birth families and lacking astable placement option, but without seriousbehaviour problems.

Intervention: residential school.No comparison group.Outcomes: school completion status, further

education, housing, employment, and socialsupport and substance use.

School completion rateswere recorded for thefirst three cohorts thatentered the residentialschool.

Longer-term outcomeswere measured 6 and12 months after theyoung people left theschool.

Community projectLee et al. (1989)

Keep youth in school: acommunity based practicemodel to keep at riskyouth in school.Washington, DC: NationalCatholic School of SocialService and The CatholicUniversity of America

97 children participated across 3years, 87 of these wereactive participants and thedata relate to these.

The sample varied fromquarter to quarter.

Population: looked-after children aged 12–15 yearsand deemed at risk of dropping out of school.

Intervention: community-based project includingmentoring, career development activities andmeetings for carers and professionals.

No comparison group.Outcomes: grade changes, attendance, staying in

school, self-esteem and attitude to school.

The evaluation followedthe project over 3years and data werecollected during theseyears.



Strategic interventions

Strategic interventions were applied at an organiza-tional level to change policy and practice to supportan improvement in looked-after children’s educationaloutcomes. Strategic interventions tried to improve thepartnership working between education and socialcare services and therefore focused on changing prac-tice in these departments. Some also included initia-tives that worked directly with children and youngpeople.

There were three studies of such interventions.Twowere English policy pilots: one pilot implemented inthree local authorities (Harker et al. 2004) and oneevaluation of the Virtual School Heads pilot imple-mented across 11 authorities (Berridge et al. 2009).The third evaluation was a US study looking at theimpact of having an educational specialist to advise

social workers on educational issues (Zetlin et al.2004).

Pilot of spending targeted money

One study evaluated a Scottish pilot of 18 authorities,who were given money to improve the educationalattainment of their looked-after children (Connellyet al. 2008).The report presents process findings fromindividual projects but the outcome findings relate tochildren across the authorities.

The projects provided five main categories ofsupport: direct support (e.g. tutoring or mentoring),personal education planning, transition support(between primary and secondary school), staff devel-opment and provision of technological support (com-puters or Internet access).

Table 1 Continued

Study Sample sizePopulation, intervention,

comparison and outcomesLength tofollow-up

Reading encouragementGriffiths et al. (2009)

The Letterbox Club 2007 to2009: final evaluationreport. London: Booktrustand the Department forChildren, Schools andFamilies.

Original sample size not stated;follow-up test results availablefor 765 children.

Population: looked-after children in years 3/4 and5/6.

Intervention: Monthly parcels of books, Mathsgames and stationery, over a 6-month period.

No comparison group.Outcomes: Maths and reading test results.

Children were tested atthe start and end ofthe intervention whichlasted 6 months.

Finn (2008)Evaluation of Reading Rich.Edinburgh: ScottishGovernment SocialResearch.

74 children participated,follow-up data collectedfor 41.

Population: looked-after children aged 7–16,primarily in residential homes.

Intervention: support to improve the readingenvironment in children’s homes and workshopsto encourage reading.

No comparison group.Outcomes: reading activity, attitudes and ability.

Children wereinterviewed at the startand end of theintervention whichlasted approximately1 year.

TutoringFraser et al. (2008)

Compass/Catch Up literacyintervention pilot for LAC.Thetford: Catch Up.

10 children, follow-up measuresavailable for 5 of them, plus 3children who were notincluded in the originalsample.

Population: children in foster care aged 11–14.Intervention: A tutoring programme called CatchUp, delivered by foster carers (primarily).

No comparison group.Outcomes: reading comprehension.

The intervention ran forone calendar year andtests wereadministered duringthis period (at thestart, mid-point andend).

Worsley & Beverley (2008)Catch Up pilot forlooked-after children2007–2008. Norfolk VirtualSchool.

26 children entered the studyand follow-up results wereavailable for 20.

Population: looked-after children spanning nationalcurriculum year groups 2–10.

Intervention: a tutoring programme called CatchUp, delivered by teaching assistants (primarily).

No comparison group.Outcomes: gains in reading age, using test results.

Children received twoterms of Catch Up andwere tested before andafter the intervention.

Lustig (2008)A silent and significantsubgroup: closing theachievement gap forstudents in foster care(doctoral dissertation).University of California SanDiego, California StateUniversity San Marcos, SanDiego State University

Follow-up measures availablefor 88 children. The size ofthe original sample is notstated.

Population: looked-after children aged 5–18 years.Intervention: one of three tutoring interventions,

all one-to-one but with slightly differentapproaches.

No comparison group.Outcomes: reading, spelling and arithmetic,

measured by the Wide Range Achievement Test.

The children had toreceive 12 weeks oftutoring betweenbaseline and post-tests.

GCSE, General Certificate of Secondary Education.



Residential school

One US study evaluated outcomes for a residentialeducation programme for young people in care (Jones& Lansdverk 2006). The aim was to provide youngpeople with a stable placement that would supportthem through high school and prepare them forfurther education or work, and facilitate permanentrelationships that would last beyond their time in theschool.

Community project

One US study evaluated a community initiative thatcombined mentoring, carer involvement and voca-tional support for young people in foster care (Leeet al. 1989). The intervention was based on the eco-logical systems theory, which argues that the environ-ment and the youth can both be changed.The projectdirected its efforts to ‘improving the transactionsbetween youth and their environment, enhancing theadaptive and coping capabilities of the youth, andimproving their environment’ (p. 6).

Reading encouragement

Two UK studies evaluated interventions aimed atencouraging looked-after children to read, with thehope that this would improve their literacy. The Let-terbox Club posted monthly parcels of books, Mathsgames and stationery to looked-after children (Grif-fiths et al. 2009). The Reading Rich interventionincluded book gifts, work with residential care homesto improve their reading environment, and readingand writing activities (Finn 2008).

Tutoring

Tutoring is often initiated by birth parents to boosttheir children’s exam results and thereby their chancesof getting into particular schools or universities. Inter-ventions in this category aimed to provide looked-afterchildren with the same opportunity. Two UK pilotsevaluated Catch Up, which is a structured tutoringprogramme delivered by foster carers and teachingassistants (Fraser et al. 2008; Worsley & Beverley2008). One US study compared three different tutor-ing approaches delivered by volunteers (Lustig 2008).

Study quality

These evaluations employed a wide range of studymethods and the critical appraisal identified concerns

about the lack of control groups, reporting ofnumbers, small samples sizes and large loss to follow-up. None of these studies would have met the inclu-sion criteria usually required for a Cochrane orCampbell review on the effectiveness of an interven-tion (The Cochrane Collaboration 2008).

In spite of methodological weaknesses, many ofthe studies are examples of complex interventionswhere considerable effort was made to measurestandardized outcomes. Reasons given for loss tofollow-up reflect the difficulties associated withresearching this population: placement moves,changes in legal status, inadequate or incompletelocal authority data management systems and dataaccess problems. One study was affected by a naturalcatastrophe (Lustig 2008). Baseline n for Griffithset al. (2009) is unknown, but their follow-up samplewas impressive at 765.

Some studies compensated for the lack of a controlgroup by comparing results with official statistics orfindings from other studies.The reliability of this relieson the quality of the comparison data. Four UK evalu-ations reported incidences of discrepancy in the localauthority data collected for looked-after children’seducational outcomes (Harker et al. 2004; Connellyet al. 2008; Finn 2008; Berridge et al. 2009). Becausedata monitoring is a specific function of the VirtualSchool, this may have improved in the UK since thetime of the evaluations.

Findings

Strategic interventions (Table 2)

The strategic interventions did not identify any cleartrends resulting from the programmes, but collabora-tion between different departments improved.

The Virtual School Head pilot survey found thatone in three children were more concerned abouttheir placement or school move than their educationalprogress and the authors expressed concern that thisanxiety was not reflected in the adults’ responses. Alsoof concern was the amount of ‘not sure’ responsesamongst adults, which might indicate that theywere unaware of the children’s needs, views andbehaviours.

In the Taking Care of Education evaluation, emo-tional well-being and self-esteem scores improved at18-month follow-up. No other score changesreached statistical significance. Young people placedimportance on encouragement from carers andteachers as a trigger to their achievement in school.



Children valued interventions that made them feelspecial but did not want their looked-after statusto be highlighted in front of peers. Some youngpeople said that they did not value support whenthey had no problems in the first place. Theevaluation questioned why the authorities did notsupport looked-after children to attend mainstreamactivities and concluded that integrating educationalsupport in placements and placement moves iscentral to helping looked-after children succeed inschool.

Pilot of spending targeted money (Table 3)

The researchers struggled to complete follow-up buttheir dataset is impressive considering the mobile

population and number of projects involved. Attain-ment improved in the children for whom results wereavailable but we do not know whether they differedsignificantly from the other children. The impact onschool attendance was small but positive. These find-ings were supported by interviews with young people,carers and professionals.

The authors concluded that individualized and flex-ible approaches were most successful and the projectswere valued by carers and families as well as youngpeople. One of the main problems was finding quali-fied staff. A lot of the pilot projects’ work went intoestablishing relationships with social work and educa-tion departments and schools, and projects were con-cerned about the continuation of these relationshipsafter the end of the pilot period.

Table 2 Strategic interventions results

Study Staying in school Attainment

Virtual SchoolHeads Pilot(Berridge et al.2009)

No clear change found. Small numbersmean that individual children and familiescould have influenced the figures. Theauthors said that improved data collectionprocedures during the course of the pilotcould have influenced the findings.

Figures for attainment were generally better in the pilotareas than overall nationally.

Because of small figures in each authority, the authorsstated that observed changes may be attributed toindividual differences within the population rather thanthe introduction of a virtual school head.

Taking Care ofEducation(Harkeret al. 2004)

Authorities in the pilot reduced theirpermanent exclusions to zero, similarto a downward trend nationally. Schoolabsences fell to below the nationalaverage in one authority, remained stablein another, and increased to three-fold thenational average in the third.

No clear indication of changes in key stage results andGCSE examinations.

The authors said percentage changes were not meaningfulbecause of small numbers. It was difficult to ascertainimpact of services because the population was sodiverse.

Educationliaison (Zetlinet al. 2004)

The treatment group reduced their absencesfrom 15.56 to 13.74, and the control groupalso reduced from 12.39 to 7.59. Thereduction was larger in the control groupthan in the treatment group (t = 2.31,P < 0.03)

The intervention group improved their Maths and readingtest scores at post-test whereas the control groupdeteriorated. The differences from pre- to post-testwere not statistically significant, but the differencesbetween the groups at pre-test were, indicating that theintervention group was catching up with the controlgroup.

There were no statistically significant differences betweenthe groups in terms of Grade Point Average.

GCSE, General Certificate of Secondary Education.

Table 3 Pilot of spending targeted money results


The educationalattainment oflooked-after children(Connelly et al. 2008)

Attendance increasedfrom 78% to 81%.

Baseline not collected.

When comparing National Assessment Levels from 1 year to another, 40%of the children in the pilots advanced by one level (mean improvement0.4–0.5 level), which is better than the average progress for thispopulation, and similar to the advances nationally made by young peoplenot in care. The improvement reached statistical significance.

Average number ofdays excluded fellfrom 0.85 to 0.65days.



Residential school (Table 4)

The authors concluded that a completion rate of 76%is promising, considering that an additional 3% left toa lower level of care and other studies have found highschool completion rates between 55% and 77% in thispopulation. The placement achieved some perma-nence as the average length of stay was 448 dayscompared with the young people’s previous history of338 days per placement.

The interviews at 6-month follow-up found that28% of young people were attending college, whichis comparable with other studies. The flux in housingand employment was less or the same as in otherstudies of care leavers. A cause for concern was therate of substance abuse after discharge, which washigher than would have been expected. The authorsconcluded that the school achieved outcomes com-parable with foster care, which is encouragingbecause foster care was not an option for theseyouths.

Community project (Table 5)

The evaluation found no significant impact from theintervention after the first project year. The mostpopular and well-run element of the project wasmentoring. The vocational component was not sopopular, mainly because the young people felt thatthe jobs offered were too menial. The Saturday tutor-ing was poorly attended, but tutoring was overall apopular initiative. The project struggled to engagecarers.

Reading encouragement (Table 6)

The Reading Rich evaluation set out to assess theimpact from the intervention but changed its focus.Only a very small component collected baseline andfollow-up reading scores. The writers’ residencies inchildren’s homes were very popular and the interven-tions appeared to improve carers’ awareness of literacyas an out-of-school activity.

The Letterbox Club evaluation had a large sampleand found statistically significant effects. Childrenwho scored high on attainment improved the most.The lowest achievers deteriorated between pre- andpost-test. The report does not provide further infor-mation on the characteristics of the children.

Tutoring (Table 7)

Many children dropped out of the studies becausethey moved school and/or placements. Those thatstayed until follow-up appeared to improve their skills.The sample sizes for two studies were very small butthe US study comparing three different forms oftutoring included follow-up measures for 88 youngpeople. One of the UK pilots of strategic interventionsfound that tutoring was very popular (Berridge et al.2009) and it has been found to be an effective inter-vention for improving reading and Maths skills inchildren aged 5–14 (Ritter et al. 2006).

DISCUSSION

These findings are indications of promising interven-tions rather than evidence of effect. The pilot of

Table 4 Residential school results


Residential school(Jones & Lansdverk 2006)

156 out of 206 (76%) completed high school and 7out of the 206 (3%) left to a lower level of care.

Not measured.

Table 5 Community project intervention results


Vocational supportand mentoring(Lee et al. 1989)

Around 10% of the young people in each year were school dropoutsand the project was not able to re-engage any of them.

Grades improved initially after 3and 6 months, but not after 1year. All measures were takenduring the intervention.

Attendance improved significantly in the first year of theintervention but not after that. No numbers are provided to backup the statement.



spending targeted money found encouragingresults, especially because these projects appearedto work with hard to engage children. The Letterboxevaluation had encouraging results from a low-intensive, low-intrusive intervention. The residentialschool appears as a promising alternative to foster careplacement. From an equity point of view, tutoringprovides a service that is popular amongst manyparents.

Nine of the studies were pilot evaluations of a newlydeveloped programme. This indicates that the devel-opment of programmes is still in its early stagesdespite long-standing concern for the education ofthis group (Jackson 1987; Berridge 2007). It mighttherefore be prudent to consider the studies in thisreview alongside Wholey (1987), who argues that suc-cessful evaluations are based on clear definitions of theproblem, intervention and outcomes, a clear logic of

testable assumptions linking resources, implementa-tion, outcomes and impact, and an agreement onevaluation priorities (Wholey 1987).

Definitions of problems, interventions and outcomes

All of the studies’ rationales were derived explicitly orimplicitly from data on looked-after children’s lowachievement in school (Jackson 1987; Department ofHealth 2002). The interventions appeared to havebeen developed in response to the system’s failure toprovide adequate education to children in care. Withthe 2007 Care Matters white paper, UK looked-afterchildren are now more monitored in the UK schoolsthan ever. Future interventions need to considerattainment support as well as other aspects such as theeffect of emotional trauma resulting from pre-careexperiences (Berridge 2007).

Table 6 Reading encouragement interventions results


Letterbox Club(Griffiths et al. 2009)

Not measured Reading was recorded as standardized scores. If all children made anaverage gain score of 0 from pre- to post-test (8 months’ progress in 8months), this would have been an average progress. Year 3/4 childrenmade a mean gain score in reading of 4.4 in both years. Year 5/6children made a mean gain score of 2.5 in 2007 and 3.5 in 2008.

For Maths, 40% of the children increased by at least one NationalCurriculum level in 2007, and 32% did so in 2008. For childrenprogressing at an average rate, the usual expectation would be that33% would make such an improvement.

Reading Rich(Finn 2008)

Evaluation frameworkincluded attendancerecords, but these datawere not collected.

No difference on reading ability found at post-test (n = 16).Of 22 children, 17 increased their reading frequency at post-interviews.

The authors conclude that the one-to-one sessions between carersand young people appeared to impact children’s reading ability, whilethe writers’ interventions had an impact on their writing.

Table 7 Tutoring results

StudyStaying

in school Attainment

Catch Up (Fraseret al. 2008)

Not measured All children gained in reading age (between 0.3 and 4.0 years). At a normal rate, achild is expected to improve his reading age by 1 month per month of observation(Clisby et al. 2000), so for this 1-year intervention the expected gain would be 1 year inreading age. This expectation was exceeded by three children, met by three children andtwo children fell below this rate.

Catch Up(Worsley &Beverley 2008)

Not measured The average gain in reading age was 17.15 months over 7 months of intervention. Thegains ranged from 1 to 30 months. This average gain is similar to when Catch Up isdelivered to mainstream pupils:http://www.catchup.org.uk/CatchUpLiteracy/CatchUpLiteracysuccess.aspx

Three tutoringmodels (Lustig2008)

Not measured Statistical significance from baseline was achieved in reading and sentencecompletion for two of the tutoring models, and in spelling for one tutoring model.

There were no statistically significant differences between the three models.



Most intervention components were adequatelydescribed. Some studies provided detailed descrip-tions of how the strategic roles and activities wereinterpreted but less on the original intentions for theseroles. Future evaluations should consider which com-ponents of an intervention need to be present acrossall sites and which elements can be adapted to fit localneeds and views.

The studies used a wide variety of measures to trackchanges in educational outcomes and it was notalways clear what tools were used. In addition to thoseshown in Fig. 2, studies measured behaviour, self-esteem, special educational need status, number ofschools attended and professionals’ attitude.

The issue of outcomes is potentially contentious.Education may be seen as primarily a middle-classvalue and professionals may see attitude and motiva-tion as more achievable than changes in attainment. Ayoung person who gets expelled frequently couldchange his behaviour in ways that are not caught bythe measuring tools used. Also needed is a discussionabout realistic expectations for children who haveexperienced long-term abuse or neglect as researchhas found a correlation between literacy andnumeracy problems and psychiatric disorders (Fordet al. 2007). Acknowledging the complexity and varia-tion within this population, Stein (2006) has arguedfor more use of theory in outcome research on looked-after children. Considering young people’s life coursetrajectories after care and applying a resilience frame-work, he found that people’s outcomes tended to varyaccording to whether they were ‘moving on’, ‘surviv-ing’ or ‘becoming victims’ (Stein 2006).

Evaluation focus and priorities

Many of the study reports in this review did notdescribe the intervention’s theory for change. Whenlooking at the nature of the interventions, it appearsthat they were based on one of the three explanations:

• Looked-after children’s education is not coordi-nated well and their educational progress is notmonitored. This means that support cannot betimely or targeted to individual needs.The strategicinterventions fit here.

• Looked-after children often fall behind at schoolbecause of placement moves and family problems,which means that they require help to catch up withtheir peers. Direct support interventions such astutoring fit here.

• Looked-after children’s home environment doesnot support their learning. The home environment

therefore needs to be changed to facilitate leisurereading and home work. The reading encourage-ment interventions and the community supportproject fit here.The UK policy evaluations indicate that there

might have been a gap between different stakeholders’understandings or expectations of the programme.None of the studies in this review asked children,carers or professionals about their desired outcomes,or involved them in the development of the design.Hawe (1994) has pointed out that the notions ofpopulation, intervention or outcome can differ signifi-cantly between stakeholders. This particularly playsout in policy evaluations of initiatives commissionedby central government departments where the inter-vention is designed centrally but with scope for localinterpretation. The gap between intended outcomesand service delivery and what actually happensbecomes evident in the evaluation, which may alsocontain elements that are at odds with the priorities ofthe practitioners.

CONCLUSION

Looked-after children as a group is not reaching theUK government’s set of educational targets andtheir achievements fall below those of their peersacross the class divide (http://www.education.gov.uk/childrenandyoungpeople/families/childrenincare).This review found that we do not know the effective-ness of programmes that aim to support looked-afterchildren in school. Some individual study resultsmerit further exploration. Studies indicated that part-nership working was beneficial but required ongoingcommitment. High-level support in a residentialschool might be a solution for children who cannot beplaced in foster care. Popular interventions weretutoring, creative-writing support and free books. Onestudy found that those who were coping well in schoolwere unhappy about being targeted for extra support.Achievement awards and acknowledgements werehighly appreciated.

The studies showed considerable effort by policy-makers, practitioners and researchers to develop,deliver and evaluate interventions. However, looked-after children themselves were not involved and thereappeared to be a gap in expectations between differentstakeholders.There is clearly a room for collaborationin this field.We need to develop clear definitions of theproblem, potential solutions and interventions, and toincorporate evaluation design from the programme-design stage onwards. While studies have asked



Figure 2 Outcome measures in the included studies. GCSE, General Certificate of Secondary Education.



looked-after children about their school experiences(Jackson & Sachdev 2001; Emond 2002; Martin &Jackson 2002; McLaughlin et al. 2006; Broad2008), we have not found any where young peoplehave informed the development of interventions orevaluations.

This review gives an overview of outcome evalua-tions. Other evaluations have focused on processissues and participant satisfaction (Jackson 1989; Prit-chard et al. 1998; Fletcher-Campbell 2001; Bryderup2004). Such studies and those included in this reviewcontain valuable information to the development ofsupport for this group. In light of considerable effortsto provide a coherent service, it is time to identify themost effective ways of providing equal opportunitiesto one of the most disadvantaged groups of childrenand young people.

ACKNOWLEDGEMENTS

We would like to thank the members of CAIS (Chil-dren’s Active Involvement Service in Islington) whoinformed different stages of the review, and particu-larly Helen who double screened more than 400abstracts, and Wintana Woldai who contributed to theprotocol and the searching, and who screened morethan 400 abstracts.

We would also like to thank Madeleine Stevens whoprovided valuable comments on an earlier version ofthis paper, and Sonja Weirauch who helped with thescreening.

Kristin Liabo was funded by a Doctoral ResearchFellowship award from the National Institute forHealth Research. This review is an independentresearch arising from this Fellowship. The viewsexpressed in this publication are those of the authorsand not necessarily those of the National HealthService, the National Institute for Health Research orthe Department of Health.

REFERENCES

Berridge, D. (2007) Theory and explanation in child welfare:

education and looked-after children. Child & Family Social

Work, 12, 1–10.

Berridge, D., Henry, L., Jackson, S. & Turney, D. (2009) Looked

after and learning: Evaluation of the Virtual School Head

pilot. London, Department for Education, Schools and

Families.

Broad, B. (2008) Aspirations: the views of foster children and their

carers. London, The Adolescent and Children’s Trust.

Bryderup, I.M. (2004) The educational principles of social edu-

cation and special education for children and youngsters in

care: a Danish study. Young Nordic Journal ofYouth Research, 12,337–356.

Clisby, C., Fowler, M.S., Hebb, G.S., Walters, J., Southcott, P. &

Stein, J.F. (2000) Outcome of Treatement of Visual Problems in

Children with Reading Difficulties. Learning Difficulties Clinic,

Royal Berks Hospital, Reading & University Laboratory of

Physiology, Oxford. Available at http://www.dyslexic.org.uk/

docs/Visual%20Problems%20-02-11-18.pdf (accessed 12

April 2012).

Connelly, G., Forrest, J., Furnivall, J., Siebelt, L., Smith, I. &

Seagraves, L. (2008) The Educational Attainment of Looked After

Children – Local Authority Pilot Projects: Final Research Report.

Edinburgh, Scottish Government Social Research.

Department for Education (2010a) Outcomes for children

looked after by local authorities in England, as at 31 March

2010. 16-12-2010. London, DfE.

Department for Education (2010b) Statistical First Release:

GCSE and Equivalent Attainment by Pupil Characteristics in

England, 2009/10. Department for Education, London.

Department of Health (2002) Education Protects. Collecting and

Using Data to Improve Educational Outcomes for Children in

Public Care. Department of Health, London.

Emond, R. (2002) Learning from Their Lessons: A Study ofYoung

People in Residential Care andTheir Experiences of Education. The

Children’s Research Centre, Dublin.

Finn, M. (2008) Evaluation of Reading Rich. Scottish Govern-

ment Social Research, Edinburgh.

Fletcher-Campbell, F. (2001) Issues of inclusion: evidence from

three recent research studies. Emotional and Behavioural Diffi-

culties, 6, 69–89.

Ford, T., Vostanis, P., Meltzer, H. & Goodman, R. (2007) Psy-

chiatric disorder among British children looked after by local

authorities: comparison with children living in private house-

holds. British Journal of Psychiatry, 190, 319–325.

Fraser, A., Barratt, G., Beverley, J. & Lawes, J. (2008) Compass/

Catch Up literacy intervention pilot for LAC. Thetford, Catch

Up.

Griffiths, R., Comber, C. & Dymoke, S. (2009) The Letterbox

Club 2007 to 2009:Final Evaluation Report. London, Booktrust.

Harker, R., Dobel-Ober, D., Berridge, D. & Sinclair, R. (2004)

Taking Care of Education: An Evaluation of the Education of

Looked after Children. National Children’s Bureau, London.

Hatton, A. & Marsh, J. (2007) Enhancing the reading of looked

after children and young people. In: Literacy and Social Inclu-

sion: Closing the Gap (eds E. Bearne & J. Marsh), pp. 53–71.

Trentham Books, Stoke-on-Trent.

Hawe, P. (1994) Capturing the meaning of community in

community intervention evaluation: some contributions

from community psychology. Health Promotion International,

9 (3), 199–210.

Heath, A.F., Colton, M.J. & Aldgate, J. (1994) Failure to escape:

a longitudinal study of foster children’s educational attain-

ment. British Journal of SocialWork, 24, 241–260.

Heptinstall, E. (2000) Research note: gaining access to looked

after children for research purposes: lessons learned. British

Journal of SocialWork, 30, 867–872.



Jackson, S. (1987) The education of children in care. Bristol, Uni-

versity of Bristol, School of Applied Social Studies.

Jackson, S. (1989) Residential care and education. Children &

Society, 2, 335–350.

Jackson, S. & Sachdev, D. (2001) Better education, better futures.

Research, practice and the views of young people in public

care. Barkingside, Essex, Barnardo’s.

Jones, L. & Lansdverk, J. (2006) Residential education: examin-

ing a new approach for improving outcomes for foster youth.

Children andYouth Services Review, 28, 1152–1168.

Lee, S., Plionis, E. & Lippino, J. (1989) Keep youth in school: A

community based practice model to keep at risk youth in school.

Washington, DC, National Catholic School of Social Service

and The Catholic University of America.

Lleras-Muney, A. (2005) The relationship between education

and adult mortality in the United States. Review of Economic

Studies, 72, 189–221.

Lustig, M.L. (2008) A silent and significant subgroup: closing the

achievement gap for students in foster care. PhD thesis, University

of California, San Diego, San Diego State University and

California State University, San Marcos.

McLaughlin, R., McConvey, V., Rodgers, P., Santin, O., Foster,

M., Hannan, M. et al. (2006) Branded a problem? Belfast, Save

the Children.

Martin, P.Y. & Jackson, S. (2002) Educational success for chil-

dren in public care: advice from a group of high achievers.

Child & Family SocialWork, 7, 121–130.

Pritchard, C., Cotton, A., Bowen, D. & Williams, R. (1998) A

consumer study of young people’s views of their educational

social worker: engagement as a measure of an effective rela-

tionship. British Journal of SocialWork, 28, 915–938.

Ritter, G., Denny, G., Albin, G., Barnett, J. & Blankenship, V.

(2006) The effectiveness of volunteer tutoring programs:

a systematic review. Campbell Systematic Reviews, 7, doi:

10.4073/csr.2006.7.

Stein, M. (2006) Research review: young people leaving care.

Child & Family SocialWork, 11, 273–279.

Stevens, M., Liabo, K., Witherspoon, S. & Roberts, H. (2009)

What do practitioners want from research, what do funders

fund and what needs to be done to know more about what

works in the new world of children’s services? Evidence &

Policy, 5, 281–294.

The Cochrane Collaboration (2008) Cochrane handbook for sys-

tematic reviews of interventions.Version 5.0.0 [Updated February

2008].

Weyts, A. (2004) The educational achievements of looked after

children: do welfare systems make a difference to outcomes?

Adoption and Fostering, 28, 7–19.

Wholey, J.S. (1987) Evaluability assessment: developing program

theory. New Directions for Program Evaluations, 1987, 77–92.

Worsley, J. & Beverley, J. (2008) Catch Up pilot for looked after

children 2007–8. Norfolk Virtual School.

Zetlin, A., Weinberg, L. & Kimm, C. (2004) Improving educa-

tion outcomes for children in foster care: intervention by an

education liaison. Journal of Education for Students Placed at

Risk, 9, 421–429.



a systematic review of interventions to support looked-after children in school

Documents