evaluation of unicef girls’ education project phase 3 (gep3) · evaluation of unicef girls’...
TRANSCRIPT
January 2017
Evaluation of UNICEF Girls’ Education Project Phase 3 (GEP3)
Tom Pellens, Rachel Outhred, Michele Binci, Zara Majeed, Johanna Wallin, Fatimah Kelleher, Monazza Aslam and Shefali Rai
Draft Baseline Synthesis Report prepared by EDOREN on behalf of UNICEF GEP3
September 2016
Nailatu (12) attends primary school in the town of Toro, Bauchi State, Nigeria. Nigeria continues to face challenges in ensuring quality education for children – especially girls.
Photo Credit: © UNICEF/NYHQ2014-0710/Eseibo
DisclaimerEDORENisaconsortiumofleadingorganisationsininternationaldevelopmentandeducation:OxfordPolicyManagement(OPM),andtheInstituteofDevelopmentStudies(IDS)attheUniversityofSussex,andissupportedbyUKAid.EDORENcannotbeheldresponsibleforerrorsoranyconsequencesarisingfromtheuseofinformationcontainedinthisreport.AnyviewsandopinionsexpresseddonotnecessarilyreflectthoseofOPM,IDSandEDORENoranyothercontributingorganisation.
EDOREN No2,16MafemiCrescent Tel +2348107278718EducationData,Research&EvaluationinNigeria Utako Tel +2348176678243 Abuja,Nigeria Email [email protected] Website www.edoren.org i
Acknowledgements
Weareverygratefultogovernmentdepartmentsandstaffinthefiveprojectstatesaswellasatnationallevel who facilitated the baseline research and shared their perspectives with the evaluation team, inparticular: the State Ministries of Education, State Universal Basic Education Boards (SUBEBs), StateAgenciesforMassEducation(SAMEs),CollegesofEducation,andtheGenderUnitoftheFederalMinistryofEducation.
WewouldliketothanktheUKDepartmentforInternationalDevelopment(DFID),whichcommissionedthisindependentevaluation,particularlyDFIDEducationAdviserMsLauraBrannellyand theDFIDEvaluationAdvisersforNigeria,MsKristenHopkinsandMrLaurieThompson,forusefuldiscussionsandadviceontheevaluationdesignandimplementation.
Thanks also go to the implementing organisation, the United Nations Children Fund (UNICEF): TerryDurnnian (Senior Education Specialist), Charles Avelino (Education Specialist), and Alassane Ouedraogo(MonitoringandEvaluationOfficer),whohavegivenmuchoftheirtimetotheevaluationteaminordertoparticipateintheevaluationdesignandshareinformationontheGirls’EducationProjectPhase3(GEP3).Special thanks are due to the GEP3 state teams and UNICEF staff in the GEP3 states for assistance inaccessinginformationandarrangingmeetingswithkeyinformants(KIs).Inaddition,wethanktheReadingand Numeracy Activity (RANA) implementation team and monitoring and evaluation support staff forcoordinating the RANA roll-out with the evaluation team, and for providing information on the RANAproject.
This evaluation is carried out in full independence by the Education Data, Research and Evaluation inNigeria (EDOREN) project. The EDOREN team responsible for the design of the evaluation and theimplementationandanalysisof thebaselinedataconsistedof:TomPellens (TeamLeaderandEvaluationSpecialist);RachelOuthred(ProjectManager,SeniorEducationSpecialist,QualitativeDesignLeadandLeadof InstrumentDevelopment); ZaraMajeed (InstrumentDevelopment andQuantitativeAnalysis Support);AndrejKveder(SurveyTechnicalLead);MicheleBinci(Econometrician);FatimahKelleher(QualitativeLead);Johanna Wallin (Qualitative Researcher); Victor Steenbergen (Qualitative Researcher); and Shefali Rai(Researcher).Thecoreteamwassupportedbyagroupofspecialistadvisers:AdrianBeavis,DavidMegill,JusteNitiema,OladeleAkogun,StuartCameron,SouroviDeandMonazzaAslam.MonazzaAslamconductedquality assurance of the Technical Report and the drafting of this Synthesis report. The key informantinterviews(KIIs)ofthebaselinetheoryofchange(ToC)assessmentwereconductedwiththeassistanceofDrHafsatLawalKontagora.ThequantitativebaselinefieldworkwasimplementedbytheOPMNigeriaofficesurvey team, under the leadership of Femi Adegoke (SurveyManager) andArogendade Ekundayo (FieldManager),withsupportbyBabatundeAkano(DataManager)andBukolaAdekolu(SurveyAdministrator).
Needless to say, during the fieldwork state education officers, local government officers, communityleaders,IntegratedQur’anicSchool(IQS)proprietors,headteachers,teachers,pupilsandparentsplayedacrucialrolebygivingtheirtimetoallowtheevaluationbaselineresearchtobeconducted;wewouldliketoextendabigthankyoutothem.
DisclaimerEDORENisaconsortiumofleadingorganisationsininternationaldevelopmentandeducation:OxfordPolicyManagement(OPM),andtheInstituteofDevelopmentStudies(IDS)attheUniversityofSussex,andissupportedbyUKAid.EDORENcannotbeheldresponsibleforerrorsoranyconsequencesarisingfromtheuseofinformationcontainedinthisreport.AnyviewsandopinionsexpresseddonotnecessarilyreflectthoseofOPM,IDSandEDORENoranyothercontributingorganisation.
EDOREN No2,16MafemiCrescent Tel +2348107278718EducationData,Research&EvaluationinNigeria Utako Tel +2348176678243 Abuja,Nigeria Email [email protected] Website www.edoren.org ii
For further information contact Rachel Outhred ([email protected]) or the EDOREN countryoffice([email protected]).ThecontactpersonatUNICEFisTerryDurnnian([email protected]),andtheresponsibleDFIDadviserisLauraBrannelly([email protected]).
©EDOREN iii
Executivesummary
Thisreportpresentsthefindingsofquantitativeandqualitativebaselinedatacollectionundertakenby
EducationData,ResearchandEvaluation inNigeria (EDOREN)aspartofamulti-yearevaluationof the
GirlsEducationProjectPhase3(GEP3).Itcomplementsthefulltechnicalreportofthebaselineevaluation,whichdescribesthemethodologyfortheevaluationandthefullsetofquantitativeandqualitativefindingsindetail.GEP3 is an eight-year project (2012–20) that seeks to improve school access, retention and learning
outcomesforgirlsinfivenorthernNigerianstates.ItismanagedbytheUNChildren’sFund(UNICEF)andfundedbytheUKDepartmentforInternationalDevelopment(DFID).Between2014and2017theprojectispiloting a series of interventions in primary schools and Integrated Qur’anic schools (IQS). Those mosteffective in improvingeducationoutcomes for girlswill be scaledup.Akey focusof theevaluation is toinformdecisionsrelatedtothescale-upofGEP3’sinterventions.Thescopeoftheevaluationinvolves:1. ahigh-levelexplicationandexaminationofGEP3’stheoryofchange(ToC);2. animpactevaluationofGEP3’searlylearningintervention;and3. aperformanceevaluationofGEP3’ssupporttoIQSs.
Thebaselineevaluationhasthreecoregoals:first,tocapturetheeducationsituation,andperceptionsofthissituation,atthestartofthere-designedGEP3,sothatchangescanbemeasuredduringfollow-updatacollection rounds, andproject attributionor contribution canbe assessed; second, to answer evaluationquestions about the relevance of the project; and third, to provide evidence to informGEP3’s ToC andprojectimplementation.
ExaminationofGEP3’sToC
Methodology
ThebaselineassessmentofGEP3’sToCfocusesontwoevaluationquestions:
• HowplausibleisGEP3’sToCinthecontextoftheGEP3states?
• HowappropriateareGEP3’sinterventionsintermsoftheimplementationstrategy?
Theplausibility of the ToC is examinedby reviewing stakeholders’ understandingof intendedoutcomes,interrogating the logic of the outcome chain, and identifying factors that have a key bearing on theachievement of the stated outcomes. GEP3’s implementation strategy is assessed on three dimensions:stakeholderinvolvement, implementationcapacityandtheextenttowhichtheprojectfocusesonunder-served groups. This assessment is based on key informant interviews (KIIs) conducted largelywithGEP3state-levelstakeholdersinAugust2015.KIswerepurposivelyselected,withthesupportoftheGEP3stateteams,basedontheirknowledgeoftheproject.
Keyfindings
The plausibility of GEP3’s ToC is supported by its coherent logic, synergies across interventions, and
stakeholders’commonunderstandingofitsmainobjectives–althoughtherearecaveatshere.Synergiesbetween the project’s interventions could enhance its impact, but also present risks, in thatimplementation difficulties on one dimension of the project could undermine its performance on otherfronts. A second key caveat is thatwhile stakeholders understandGEP3’smain objectives, the project’soperational design is not always understood. For instance, at the time of the interviews government
©EDOREN iv
stakeholdersinvolvedinteachertrainingdidnotyetunderstandwhatcapacitiesneedtobedeveloped,inwhatways, andwhen. This couldmake it difficult for them to assessGEP3’s performanceor be held toaccountfortheresultsachieved.
Stakeholdersnoted that theproject’sobjectiveof increasinggirls’enrolment is feasible,particularly in
primaryeducation,butthatimprovementsinretentionandlearningoutcomeswillbehardertoachieve.
Stakeholderswereoftheviewthatenrolmentdrivesandcashtransfersarebotheffectivestrategiesthataddressimportantbarrierstogirls’accesstoeducation,andthattheycomplementoneanother.Retentionis held to bemore challenging as it is shaped by supply-side factors, such as the presence of sufficientteachers,thequalityofteaching,andthequalityoftheschoolenvironment.Stakeholderswerelesslikelyto recognise learning as a key expected project outcome. When they did, they noted that learningoutcomesaremoredifficult to improve thanaccess,owing to the lowqualityof teaching,particularly inrural areas, and insufficient investment in the supply of quality education. There was wide consensusamongstKIs that tackling thequalityof teachingandteachers iscentral to improving learningoutcomes.ThisisconsistentwiththefocusonthisvariableinGEP3’sToC.
Awiderangeofstakeholdersendorsedthe importanceof theearly learning intervention.Stakeholderssupporttheemphasisonliteracyduringearlygradesasafoundationalskill.However,thereisnoconsensusamong stakeholders about the extent towhich the use of amother tongue is a necessary condition toimprovelearningoutcomes.ThebaselinefindingsoftheearlylearningevaluationconfirmthatHausaisthelanguage of the immediate environment in Katsina and Zamfara, but they also highlight that Hausaknowledgeamongteachersis low.Thisneedstobetakenintoaccountinthedesignoftheearlylearningintervention.
GEP3’s interventions to support school governance are considered to be largely promising, although
thereare some risks related to the targetedempowermentof School-BasedManagementCommittees
(SBMCs).SBMCempowermentisbothapivotalintermediaryoutcomeinGEP3’sToC,aswellasoneofitsmostprecariouslinks.TheexpectationsplacedonSBMCsintermsofrolesandresponsibilitiesareveryhigheventhoughtheseorganisationsareoftenstartingfromaverylowbase,particularlyinIQSs.Atthestatelevel, High-Level Women Advocate (HiLWA) members engage with decision-makers and schoolcommunitiesandhavethepotentialforindirectinfluence.TheGirlsEducationSteeringCommittee(GESC)alsohasthepotentialtosupportgovernance,totheextentthatitisactive,sufficientlylocallyownedandaddresses key challenges. GEP3’s support to the educationmanagement information system (EMIS) andAnnualSchoolCensus(ASC)willplausiblyimprovedatareliability,althoughthiswillnotnecessarilyleadtodatabeingusedinthepolicy-makingprocess.
The findings point to some specific risks to the scale-up process. The scale-up process is reliant ongovernmentfunding.However,thereissignificantuncertaintyabouttheextenttowhichthisfundingwillbeforthcoming.ThisisdespitecertainsupportivemeasuresbeingintroducedaspartoftherecentdesignofGEP3,notablyanincreasedemphasisonhigh-leveladvocacyandafocusonamoremanageablesetofoutcomes. The scale-up of GEP3’s support to IQSs is particularly uncertain because the institutionalmandate over IQSs is unclear, the number ofwell-established IQSs available for scale-up is limited, andthere are supply-side constraints, particularly with regards to facilitators. One further constraint is thatstakeholdersdonothaveaclearunderstandingofhowscale-upisexpectedtotakeplace.
Overall,thebaselinefindingspointtoanumberofriskstothecausallinksintheToC.Criticalassumptionsrelatedtothemanagementandresourcingoftheeducationsystemarehighlyuncertain,inparticularthereleaseofgovernmentfunding,school investmentandhumanresourcecapacityremainingonaparwithincreasedenrolment,effectivemonitoringattheschoolandinterventionlevels,andSBMCs’abilitytoplaythewide-rangingrolethatisbeingsoughtaspartoftheproject.
Baseline findings on implementation capacity are mixed. The project makes an appreciable effort toinvolve a variety of stakeholders. However, their involvement in operational planning is uneven across
©EDOREN v
interventions. Government capacity building is embedded in project design, but an effective system isneeded to identify and address themost important capacity development needs.Monitoring capacity isrightly emphasised in GEP3’s re-design but its operationalisation has yet to advance. With regards toequity,severalGEP3interventionsaredesignedtobeequityenhancing,butitisunclearwhetherthemostvulnerablehouseholdsandgroupsareabletobenefitfromandactivelyparticipateinGEP3interventions.Notably,GEP3isnolongeroperatinginLocalGovernmentAreas(LGAs)withthehighestgendergapinallstates.
ImpactevaluationofGEP3’searlylearningintervention
GEP3’searlylearninginterventionaimstoimprovetheearlylearningskillsofchildreninprimaryGrades
1to3(Primary1(P1)–Primary3(P3))inthemothertongue,whilealsopreparingchildrentolearnwithEnglish as a language of instruction by the time they transition to Grade 4. A key measure of theintervention’ssuccesswillbe improved literacyskills.The interventionwillbe implementedoverathree-yearperiod (2016–2018) insixLGAs inZamfaraandKatsina (threeperstate).The interventionhas threekeycomponentsatschoolandcommunitylevel:theprovisionofapackageofHausa-mediumteachingandlearning materials to schools; early grade professional development for teachers and head teachers(includingmonthlyschoolvisits);andasetofcommunityawarenessandengagementactivitiestosupportearlygradeliteracy.
Methodology
The evaluation of the early learning intervention uses a theory-based approach and is designed as a
clusteredrandomisedcontrolledtrial(RCT),stratifiedbyLGAandtypeofschool(primaryschoolvs.IQS),and randomised at the school level. The intervention’s ToC was used as a framework to formulate theevaluationquestions.TheRCTdesignallowstheevaluationteamtomeasuretheattributableimpactoftheearly learninginterventiononlearningoutcomesbycomparingoutcomechangesinatreatmentgroupofschoolswith those ina control group that is statistically similaronaverage.TheRCTdesign is combinedwith the overarching theory-based evaluation approach to measure not only changes in final outcomevariablesbutalsonetchangesinintermediaryoutcomevariablesalongtheassumedcausalchain.Thiswillallowtheevaluationteamtounpackhowchangetakesplace.
Inorder tomeasure changes inoutcomesapanel surveywasdesigned fordata collectionatbaseline,
midlineandendline.Datawillbecollectedateachofthesethreestagesinasampleof120publicprimaryschoolsand120IQSsdrawnfromthesixinterventionLGAsacrossKatsinaandZamfara.Halfofthepublicprimary schools and IQSs form part of the treatment group, while the other half serves as the controlgroup. Baseline data collection was conducted in October–November 2015 during the first term of the2015–2016 school yearbefore the startof implementationof theearly learning intervention.Within thesample schools, male and female pupils and teachers were randomly sampled to form part of a panelsurvey.Sevendatacollection instrumentswereadministeredwithineachschool:pupilEnglishandHausaliteracy assessments, pupil and teacher questionnaires, a teacher knowledge and skills assessment, ateacherclassroomobservationandaheadteacherquestionnaire.Theliteracyassessmentswerecarefullydesignedandpilotedtoensurethatitemdifficultymatchedpupils’ability.
Analysisofthebaselinedataindicatedthatrandomisationhadworkedtocreatecomparabletreatment
and control groups. We assessed whether the randomisation had achieved its intended purpose bychecking whether key outcome variables and school-, teacher- and pupil-level characteristics differedbetweenthetreatmentandcontrolgroupsatbaseline.Thegreatmajorityofvariablesinvestigateddidnotshowanystatisticallysignificantdifferencebetweenthetwogroups.
©EDOREN vi
Keyfindings
Thebaselinefindingshighlightthattheearlylearninginterventionwillbeimplementedinachallenging
context.Thevastmajorityofschoolsarelocatedinruralareasandhavepoorinfrastructure.Whilealargershare of IQSs than public primary schools have access to drinking water and electricity, they have lessrooms on average and have less access to books and functional toilets for girls. Teachers in IQSs arepredominantlymale. The baseline also found that 40% of IQSs only have one teacher who teaches theintegrated curriculum subjects, which raises questions about the applicability of the school-level peermentoringapproachintheseIQSs.
Head teachers make some effort to address teacher attendance, but are less likely to take action to
improvethequalityofinstruction.Almost90%ofheadteachersinpublicprimaryschoolsreportedtakingactiononteacherattendance.ThecorrespondingfigureforIQSswasjustover50%.Overhalfoftheheadteachers interviewedhadnotobserveda single lessonduring theprevious school term.Similarly,halfofheadteachersdonothavemeetingswithteachersormeetthemlessthanonceamonth.
Anotableshareofearlygrade teachersdonot teach languages.Roughlyhalfof teachers reported thattheyteachonlyonesubject,ofwhichjustover40%teachasubjectotherthanHausaorEnglish.GiventhattheReadingandNumeracyActivity (RANA)willemphasiseearlygradereading, thispoolof teachersmaynot be suitable candidates for the intervention. This needs to be taken into account when selectingteachersfortraining.
Theteacherssurveyeddemonstratedverylimitedknowledgeandskillsinrespectofmostofthedomains
covered by the teacher assessments. Less than 3% of teachers were able to display competence inidentifying lowperformers,evidencing judgementsanddiagnosingpupilperformance, interpretingwordsandphrases,andwritingskills.AlthoughallteachersreportedthattheyspeakHausa, lessthan40%wereabletodisplaycompetenceinprimaryGrade1and2-levelHausa.ThisissignificantforaninterventionthatfocusesonteachinginHausaasteachers’subjectknowledgehasimportantimplicationsforthequalityofteaching.Teachers’verypoorskills in identifying lowperformersanddiagnosingpupils’performancearelikely to present key hurdles to improving teaching quality given the wide recognition in the educationliteraturethatchildrenlearnbestwhenteachingistargetedtowhatthechildisreadytolearn.Therearesomeminorvariationsinperformanceacrossdifferentgroupsofteachers,butknowledgeandskillslevelsareconsistentlylowwithinallgroups.
Thesefindingsregardingteachers’knowledgeandskillshaveafewkeyimplicationsfortheearlylearning
intervention.Theverylowlevelsofknowledgeandskillsamongstteachersatbaselineindicatesthatthereis substantial scope for improvement in this area. However, it also presents challenges, in that there isoftenalackofbasicfoundationalskillsthattheprojectcanbuildon.Thefindingsshouldserveasausefulinput to the RANA implementation team as they highlight some of the key areas of weakness that theintervention would need to address. They also provide an indication of the scale of the challengesconfrontingtheproject,whichwillhaveimplicationsforimplementationdecisions–forinstancerelatedtothefrequencyandcontentoftraining,andthefocusoftheschoolvisits.
Classroom observations indicate that the quality of early grade instruction is low across all groups of
teachers.Theextentofpupil-centred learningobservedatbaselinewas low,aswas theextent towhichteacherslinkthelessontopreviouslearningandlearningobjectives.Incontrast,timeontaskwashigh,atanaverageof96%ofatotallesson.However,thisfindingshouldbeinterpretedwithcautionasitislikelythatthepresenceofobserversintheclassroomledtoanincreaseintimeontask.Thebaselinealsosoughttomeasuregender-sensitiveteachingpracticesusingclassroomobservationdata,butextremecomplianceeffectswereobservedacrossall items,raisingquestionsaboutthevalidityandreliabilityofthemeasure.Teachers’ responses to the assessments indicate that teachers are aware of objectives to target girls inclass,astheystatethatitisimportanttofocusongirls,buttheirresponsesalsopointtothepersistenceofdeeplyingrainedgenderbiases.
©EDOREN vii
Theavailabilityanduseof teachingand learningmaterialswasobserved tobevery low. This suggeststhatthedistributionoflearningmaterialsbytheRANAinterventioncouldfillanimportantgap.However,itwillbevital formaterialstobepitchedatanappropriate level, taking intoaccountthevery lowlevelsofknowledgeamongstbothteachersandpupils.
Hausa-basedteachingisprevalentintheearlygrades.Hausawasusedinallclassesobservedatbaseline.In roughly half of classes, a second languagewas used in addition to Hausa. However, as noted above,manyteacherslackcompetenceinbasicHausa.
The baseline sought to assess teacher motivation and attendance as these variables are likely to
influence the extent to which improved knowledge and skills amongst teachers translate into better
teaching.Teachermotivationwas relativelyhighon threedimensions: theeffortput into,andperceivedimportance, of teachers’ work; their enjoyment of teaching; and interaction between teachers. Scoresrelated to pressure andwork-related tension and teachers’ perceived self-efficacywere low. Therewaslimited variation across teachers. Absenteeism was higher in IQSs than in public primary schools, withfacilitators reporting that theywere absent for an averageof eight days over theprevious 60days. Thecorrespondingfigureforpublicschoolswasthreedays.
ThebaselinefindingsindicatethatinbothEnglishandHausa,veryfewpupilshavelevelsofknowledge
that are appropriate for their grade. In both subjects, the vast majority of P2 pupils assessed haveknowledgeand skills expectedofpre-school children (pre-literacy skills). Furthermore, inboth cases, thepeak of the distribution of performance falls well below the cut-off point between pre-literacy andemergingliteracy.Thissuggeststhatsubstantialeffortwouldbeneededtoachieveasignificantincreaseintheshareofpupilsmovingfrompre-literacytoemergingliteracy.
Pupils’knowledgeofphonic isparticularly low.Thepsychometricanalysis foundthat itemsthat requireknowledgeofphonicsrankasthemostdifficultitemsinboththeHausaandEnglishassessments.Correctlysoundingoutlettersandidentifyingsimilarsoundswasmoredifficultforpupilsthanwritingorreadingfullpassages.
Learningoutcomesdiffer by age and gender.Older pupils in P2 performbetter than younger ones andboys perform better than girls. Gender differences in performance are small in the younger years, butincreaseoncegirlsreachpuberty(around12yearsofage).
Regressionanalysis indicatesthatpupilcharacteristics,socio-economicbackgroundandschooltypeare
associatedwithlearningoutcomes,butteachercharacteristicsarenot.Thelatterfindingmayreflectthefactthatcompetencylevelsareverylowacrossallteachers.Itislikelythatastrongercorrelationwouldbedetected if there were greater variation in competency levels across teachers, and if their ability toinfluencepupils’learningoutcomeswashigher.Thisalsoimpliesthatiftheearlylearninginterventionleadsto substantial improvements in teachers’ knowledge and skills we should see a more significant linkbetweenteachercharacteristicsandlearningoutcomesatendline.
EvaluationofGEP3’ssupporttoIQSs
GEP3’s support to IQSs (IQSS) seeks to improve education outcomes at these schools, with the aim ofprovidinganacceptablealternativeformofqualitybasiceducationforgirls.Theinterventionseekstoraiselearning outcomes in basic literacy and numeracy for pupils at IQSs (especially girls), improve retentionamong girls, and (to a lesser extent) increase girls’ enrolment. The project targets registered Islamiyya,Qur’anicTsangayaEducation(IQTE)centresthatofferanintegratedcurriculumandinwhichatleast40%ofpupilsaregirls.TheseIQSslargelyoperateascommunity-basedinitiatives,butarewillingtobuildlinkswithgovernmentforthepurposesofmonitoringandtechnicalsupport.GEP3’sIQSSinvolvestheprovisionof training and mentoring of IQS facilitators, training for head teachers, the distribution of classroom
©EDOREN viii
teachingandlearningmaterials,capacitybuildingforCentre-BasedManagementCommittees(CBMCs)andtheprovisionofmini-grants.
Methodology
Theevaluationapproachdrawsontheprinciplesofcontributionanalysisandreliesonastrongmixof
quantitative and qualitative methods. It does not make use of a comparison group to make causalinferences but rather seeks to make credible causal claims about the intervention’s contribution toeducationoutcomesbyverifyingthechainofexpectedresultsandassumptionsasperacredibleToC,aswell as assessing alternative explanations for the outcomes observed. The evaluation will assess threecontributionclaims:
• GEP3’sIQSScontributestomoreeffectiveteachingofformalsubjectsinIQSs;
• GEP3’sIQSScontributestoanimproved,girl-friendlyenvironmentwithintheschools;and
• More effective teaching of formal subjects and an improved, girl-friendly environment contribute toimprovedlearninglevels,particularlyamonggirls.
ThequantitativecomponentoftheevaluationconsistsofrepresentativesamplesurveysamongGEP3IQSsinthe12GEP3LGAsinBauchiandNiger.Thebaselinesurveytookplaceinasampleof60IQSs,stratifiedbyLGA.Itinvolvedtheadministrationofthesameseveninstrumentsusedfortheearlylearninginterventionevaluation, as well as a pupil numeracy assessment and CBMC questionnaire. The baseline qualitativeresearchtookplaceinsixIQSsthatwerepurposivelysampledusingtypicalandextremecasesamplingandthat were also included in the quantitative survey. The same set of IQSs will be visited at midline andendline.
Keyfindings
ContributionClaim1:GEP3’sIQSScontributestomoreeffectiveteachingofformalsubjectsinIQSs
The landscape of IQSs is complex. Variations in school structures, pupil–teacher ratios, facilitators’qualifications,schoolleadershipandmanagement,andthenumberandhoursforwhichintegratedsubjectsaretaughtperschoolsuggestthattheoutcomesofinterveningintheseschoolsarelikelytodifferfromoneanother. Some IQSsmaybemobile,whichwouldhave implications for continued teachingand learning,particularly for girlswho are unlikely tomovewith theMallam.1 The extent of integration varies acrossIQSs,andisloweronaverageinBauchithaninNiger.Acrossthetwostates,headteachersreportedthatanaverageof threehoursperweek is spentonteaching the integratedcurriculum,which iswellbelowtheeight hours recommended by official guidelines. A quarter of IQSs only have one facilitator, which willconstrainamentoringprocessbasedonschool-basedpeerinteraction.Onecrucialsupportivefactoristhatintegrationdoesappeartohavegainedacceptanceamongparentsandthecommunity.
Facilitators’knowledgeandskillsinkeydomainsassociatedwitheffectiveteachingareverylow.Only2%of facilitators were able to display competence in identifying low performers. Less than 1% hadcompetence in writing skills and none of the facilitators assessed were competent in evidencingjudgementsanddiagnosingpupilperformance.Only34%offacilitatorswereabletodisplaycompetenceinGrade1and2-levelHausa.Mostfacilitatorsdisplayalackofunderstandingofwhatstepstheycantaketoimprovepupils’performance.Aswiththeearly learning intervention,thesefindings indicatethatthere istremendous scope for improvement in teachers’ performance, but also that it will be crucial for theinterventiontobecarefullytargetedtotheverylowlevelsofskillsandknowledgeamongstfacilitators.
1ThetermMallamcanhavedifferentinterpretations.WeconsidertheMallamtobetheheadofthereligiousschool,responsibleforreligiouseducation.
©EDOREN ix
Facilitators scoredverypoorlyona composite indexof effective classroompractices. Inparticular, thedepthofpupil-centredlearningwasfoundtobelowandtherewaslittleeffortamongfacilitatorstolinkthelesson to previous learning and learning objectives. Time on task was generally high: in 65% of lessonsobservedpupilsspent100%ofthelessonontask.However,thismayhavebeeninfluencedbythepresenceofobservers.
Thereismixedevidenceonfacilitators’attitudesandclassroompracticestowardsgirls.Thefacilitatorsinthesixcasestudyschoolsexpressedpositiveattitudestowardsgirls’education,althoughtheseresponsesarepronetosocialdesirabilitybias.Indeed,bothqualitativeandquantitativedatapointtothepersistenceofgenderbiases–forinstancerelatedtogirls’abilities.Thefindingsindicatethatfacilitatorsareawareofand sometimes practice gender-sensitive techniques, such as actively engaging both girls and boys.However, thesepractices seem tobeperformed in a tokenisticway—in linewithdevelopmentprojects’sensitisationefforts—andgender-biasedclassroompracticescontinuetotakeplaceinIQSs.
Aswiththeearlylearningintervention,thebaselineevaluationsoughttoassessfacilitators’motivation
and attendance. On average, facilitators think that their role is important and they enjoy working asteachers, but they have poor perceptions of their teaching efficacy. The qualitative research found thatfacilitatorsfeelintrinsicallymotivatedbywhattheydo,andthatschoolstakeholders,includingpupils,thinkthatfacilitatorsasgenerallydedicated.However,italsoindicatesthatthefactthatmostfacilitatorsarenotpaidmakesitdifficulttoattractqualifiedfacilitatorsandholdthemaccountable.Thequantitativefindingsconfirmthataverysmallshareoffacilitatorsarepaidasalaryorastipend(3%inBauchi,33%inNiger)andpointtoapositivelinkbetweenreceivingremunerationandteachermotivation.Roughly75%offacilitatorsreported that they had been absent at least once in the previous threemonths. Furthermore, the casestudy findings indicate that few IQSshave set timetables, and formal subjects are taughtasandwhenafacilitatorisavailable.
Theavailabilityanduseofteachingandlearningmaterialsisverylimited.AlmostnoHausamaterialsareavailable,althoughitisimportanttonotethatHausaisnotthemothertongueofallpupils.InBauchi,93%of sampledchildren reported speakingHausaathomebut the corresponding figure inNiger is just54%,with43%speakingNupe.
SchoolleadershipinIQSsiscomplex,whichraisessomechallengingquestionsaboutwhopreciselyshouldbe targeted by the pedagogical leadership component of the IQS intervention. IQSs have a variety ofdifferent leadership roles (Mallam, head teachers, proprietor). These may be held by one person ordifferentpeopleandthewayinwhichresponsibilitiesaresplitacrosstheseindividualsvariesacrossIQSs.Theprofessionalandacademicqualificationsofheadteachers inIQSsaregenerally low,and,onaverage,below those of facilitators. This raises questions about head teachers’ technical capacity to exercisepedagogical leadership over facilitators. The appointment of a head teacher is not necessarily based onability and qualifications but has to dowith social statuswithin the community and perceptions aroundleadershipmoregenerally.
ContributionClaim2:GEP3’sIQSScontributestoanimproved,girl-friendlyschoolenvironmentinIQSs
CBMCs have some potential to bring about improvements in the learning environment in IQSs. Themanagement of IQSs is seen as collective, without clearly defined and delegated roles, which providesopportunitiesforcommunity-basedmanagementstructureslikeCBMCs.Thebaselinefindingsalsoindicatethatmost CBMCs are established and active, as indicated by regularmeetings andmonitoring activities.CBMCs have considerable community representation, although women are under-represented and veryfewmembersarechildren.This indicatesthatatpresentCBMCsareunlikelytoprovideaforuminwhichgirls’voicesandneedscanbeheard. Inall IQSs included in thequalitativestudytheMallamholdsakeypositionintheCBMC,suchaschairperson.ThisraisesquestionsabouttheextenttowhichCBMCscanplayaneffectiveroleinholdingschoolleaderstoaccount.
©EDOREN x
CBMCsdomakeanefforttoraiseresourcesfromthecommunity,butthisappearstoofferlimitedscope
inregardtobridgingresourcegapsinIQSs.ThequalitativeresearchindicatesthatCBMCsfeelthatalackof fundshinders theirability to ‘solve’problemsat IQSs.Theyalsonote thatwhileparentsarewilling tocontributeresources,theyoftenhavelimitedcapacitytodoso.ThesefindingshighlightthecaseforCBMCtrainingtogobeyondanarrowfocusoncommunityresourcemobilisationtoencompassthemobilisationofresourcesfromalternativesources.
CBMCmembersseemtolargelyunderstandtheirrolesandresponsibilitiesbutoftenlackthecapacityto
enact them. Key areas in which CBMCs are under-performing are school planning and financialmanagement.CBMCmembersdomakeaneffort tomonitor thequalityof the teachingand learning, inparticular pupils’ and facilitators’ attendance. CBMC members assert that the main constraint thatunderminestheirperformanceisalackoffunds,particularlytoimproveinfrastructureandpayfacilitators.
Most IQSs do not offer a girl-friendly school environment at present. The vast majority face majordeficiencies inphysical infrastructure.Only3%of the schoolshaveaccess toawater source,20%of theIQSsdonothaveaphysicalclassroomstructureforthestudents,andlessthan25%havefunctioningtoiletsforpupils,withonly8%havingfunctioningtoiletsforgirls.Only3%ofIQSshaveamothers’group,teacher–student association or pupil groupwhere students can discuss their concerns. Gender-biased classroompracticesandattitudesprevail,whichcanaffectgirls’learningopportunitiesandself-confidence.However,there appears to be an openness among school leaders to try and address these issues. ResourcesmobilisedbyCBMCsareinvestedinschoolsandgirl-friendlyinvestmentsareconsidered,buttheamountsraisedappearinsufficientgiventhepoorconditionoftheschoolenvironment.
Theextentofgovernmentengagementwith IQSs is limited.Themajorityof IQSs in thesamplehadnotbeen visited by a government official during the previous term. Government officials say that they facechallenges in reaching all the schools on a regular basis. Communities generally view support from thegovernmentnegatively,consideringgovernmentactorstobeunreliable.
ContributionClaim3:Moreeffectiveteachingofformalsubjectsandanimproved,girl-friendlyschool
environmentcontributetoimprovedlearninglevels,particularlyamonggirls
Thebaselineevaluationidentifiessomekeyfindingsthatarerelevanttothiscontributionclaim.Girlsandboys are both equally likely to attend IQSs. However, both face challenges in regard to attending andremaining fully engaged with the learning process because of out-of-school responsibilities, which arelinked to household poverty. A large share of childrenwho study in IQSs attend another school aswell.There were significant cross-state differences, with 72% of pupils in Bauchi reporting attending otherschools,ascomparedtoonly13%pupilsinNiger.InBauchi,mostofthesechildrenwereattendingapublicprimary school (94%), as compared to 55% in Niger. This indicates that interventions at local primaryschoolsmayconstituteanalternativeexplanationforanyimprovementsinlearningoutcomesatIQSs.
Pupils’ learningoutcomesarevery low. InbothHausaandEnglishover90%ofGrade2pupilsassesseddisplayed literacy levels associatedwith pre-school pupils. Furthermore,most of these pupils are a longway from the cut-off for emerging literacy skills (those associated with the P1 curriculum). Numeracyoutcomeswerebetter,with69%ofpupilsdisplayingemergingnumeracyskills,comparedto21%withpre-numeracyskills.However,only11%ofpupilshadnumeracyskillsassociatedwiththeP2curriculum.Boysandgirlsbothperformpoorly,althoughgirls’performancedeclines,comparedtothatofboys,ataroundpuberty(roughly12yearsofage).
Continuedbarrierstogirls’accessto,andretentionin,schoolingexist.Attitudestowardsgirls’educationarenotalwayssupportive,althoughthisappearstobechanging.Changingattitudesalone,however,maynot be sufficient to bring about change in behaviour as poverty is often cited as a critical reason whyparentsdonotsend theirchild toanyschool,publicprimaryor IQS.Bothboysandgirlsoftenengage insomeformofincome-generatingactivity,whichparentstendtoviewasafinancialandpracticalnecessity.
©EDOREN xi
Thisadverselyaffectsschoolattendance.Earlymarriageisalsocitedasakeyreasonforlowenrolmentandretentionamonggirls.
Recommendations
ThebaselinefindingspointtocertainkeyrecommendationsforGEP3.ThebaselineassessmentofGEP3’sToC highlights that continued political engagement is required to ensure that government funds aremobilisedforprojectscale-up.Theprojectshouldspecifyandcommunicateitsoperationalobjectivesandstrategies to government and other implementing partners, strengthen itsmonitoring of assumptions inthe ToC, and ensure that monitoring information is used to facilitate learning and accountability. Inparticular,emphasisshouldbeplacedonmonitoringGEP3’ssupporttoSBMCs/CBMCs,giventheirpivotalroleintheproject’sToC.
Baseline findingson theearly learning interventionhighlight that theRANA interventionwillneed tobecarefully tailored to thevery low levelsofknowledgeandskillsamongst teachers toensure that trainingcontentandmaterialsarepitchedattherightlevel,andthattrainingissufficientlyintensivetofillthelargegapsinteachers’subject,pedagogicalandcurriculumknowledge.Teachercapacitydevelopmentalsoneedsto incorporate actions to change teachers’ awareness of their own potential for influencing learningoutcomes,andtheirunderstandingofhowbesttosupportpupils’learning.Thepeermentoringapproachneedstobeadaptedfor the largenumberof IQSsthatonlyhaveoneteacherwho iseligible forsupportunder the intervention. RANA also needs to sufficiently strengthen the capacity of IQS governmentstakeholders to ensure that they effectively monitor and provide support supervision. Finally, thecomplexityoftheIQSleadershipmodelindicatesthatspecialattentionneedstobegiventoensuringthatappropriateindividualsatIQSsaretargetedfortheprogramme’spedagogicalleadershiptraining.
ThefindingsontheIQSSinterventionhighlighttheneedfortheprojecttostrengthenitsmonitoringoftheintervention. The IQS context is diverse, flexible and evolving. In order to adapt the intervention to thiscontext,quicklearningandfeedbackbasedonmonitoringdataisneeded.Theselectionoffacilitators,headteachersandCBMCmembersfortrainingrequirescloseattention,verificationandmonitoring.FacilitatortrainingandmentoringneedstobecarefullyadaptedtotheverylowlevelsoffacilitatorcompetencyandtheIQScontext.Similarly,learningandteachingmaterialsneedtobetailoredtotheverylowlevelsofskillsandknowledgeamongstbothfacilitatorsandpupils,andtothelanguageoftheusers(particularlyinNiger,where Nupe is widely spoken). Training and mentoring on gender-sensitive class practices needs to gobeyond a focus on gender-sensitive teaching practices to try and tackle entrenched gender biases, forinstancerelatedtogirls’abilitytolearn.Womenandgirls’participationinshapingeducationandtheschoolenvironmentneedsfurther investigation:atpresent,bothareunder-representedonCBMCs,which limitsthe scope for thesebodies toact as forums forwomenandgirls’ participation in the sector. Finally, thebaseline findings highlight the need for GEP3 to promote the mobilisation of resources beyond thecommunityandtoadvocateforfacilitatorstobeadequatelyremuneratedbythegovernment.
©EDOREN xii
Tableofcontents
Acknowledgements Error!Bookmarknotdefined.
Executivesummary iii
Listoffigures,tablesandboxes xiv
Listofabbreviations xvi
1 Introduction 1
2 Projectandevaluationbackground 2
2.1 Objectives,scopeandinterventionstrategy 2
2.2 ObjectivesandscopeofGEP3evaluation 3
2.3 Evaluationdesignandmethods 3
2.3.1 Evaluationquestions 32.3.2 Evaluationdesignandmethods 42.3.3 Inclusionandethics 42.3.4 Externalvalidityofthefindings 4
3 BaselineToCassessment 5
3.1 Methodology 5
3.2 PlausibilityofGEP3’sToC 5
3.2.1 Generalassessment 63.2.2 Output1:Increasedenrolmentandretentionofgirlsinbasiceducation 73.2.3 Output2:Improvedcapacityofteacherstodelivereffectivelearningforgirls 73.2.4 Output3:Improvedgovernancetostrengthengirls’education 83.2.5 GEP3’spilot-to-scale-upapproach 9
3.3 AppropriatenessofGEP3’simplementationstrategy 10
3.3.1 Stakeholderinvolvement 103.3.2 Governmentcapacity 103.3.3 Equity 11
4 Baselineoftheearlylearninginterventionevaluation 12
4.1 GEP3’searlylearningintervention 12
4.2 Baselinemethodology 12
4.3 Comparisonofbaselinecharacteristicsbetweentreatmentandcontrolschools 13
4.4 Baselinefindings 14
4.4.1 Teachingintheearlygrades 144.4.2 Pupils’literacyinearlygrades 234.4.3 Analysisofrelationships 26
5 BaselineoftheIQSSevaluation 28
5.1 GEP3’sIQSS 28
5.2 Baselinemethodology 29
5.3 Baselinefindings 30
5.3.1 ContributionClaim1:GEP3’sIQSScontributestomoreeffectiveteachingofformalsubjectsinIQSs 30
5.3.2 ContributionClaim2:GEP3’sIQSScontributestoanimproved,girl-friendlyschoolenvironmentinIQSs 42
©EDOREN xiii
5.3.3 ContributionClaim3:Moreeffectiveteachingofformalsubjectsandanimproved,girl-friendlyschoolenvironmentcontributetoimprovedlearninglevels,particularlyamonggirls 48
6 Conclusionsandrecommendations 55
6.1 Conclusions 55
6.2 Recommendations 60
Bibliography 64
AnnexA ToCoftheearlylearningintervention 65
AnnexB Evaluationquestions 68
B.1 Evaluationquestionsfortheearlylearninginterventionevaluation 68B.2 EvaluationquestionsfortheIQSSevaluation 68
AnnexC Samplesizeofbaselinesurvey 70
C.1 Samplesizeofearlylearninginterventionbaseline 70C.2 SamplesizeofIQSSbaseline 70
AnnexD Randomisationchecksfortreatmentandcontrolgroup 72
AnnexE Regressionoflearningoutcomesonkeyinfluencingfactors 75
AnnexF UpdatedToCofIQSSintervention 78
©EDOREN xiv
Listoffigures,tablesandboxes
Figure1: SchoolmanagementactionsinPSandIQS............................................................................17Figure2: Percentageofteachersachievingwithinthelower,middleandupperbandsofcompetence
acrosstheteacherknowledgeandskillssubscales...............................................................19Figure3: FrequencyofHausauseintheclass......................................................................................21Figure4: Comparisonofmotivationsubscales.....................................................................................22Figure5: DistributionofHausaliteracyproficiency.............................................................................25Figure6: MeanHausascalescorebyageandgender(95%confidenceinterval)...............................26Figure7: CoresubjectstaughtinIQSs..................................................................................................33Figure8: Percentageofteachersachievingwithinthelower,middleandupperbandsofcompetence
acrosstheteacherknowledgeandskillssubscales...............................................................36Figure9: Comparisonofmotivationsubscales,IQSSsample...............................................................37Figure10: Facilitatorlanguageuse:Shareoffacilitatorswhousedthelanguageatleastonceduring
thelesson...............................................................................................................................39Figure11: Availabilityanduseofteachingandlearningmaterials........................................................40Figure12: HeadteacherpedagogicalleadershipandmanagementactionsinNigerandBauchi..........42Figure13: Languagespokenatpupils’homes,bystate.........................................................................50Figure14: DistributionofHausaliteracyproficiency.............................................................................52Figure15: Distributionofnumeracyproficiency....................................................................................53Figure16: DiagramdepictingToCoftheearlylearningintervention....................................................67Figure17: DiagramdepictingToCoftheIQSS........................................................................................79Figure18: Causalpackageandunderlyingassumptionsof‘moreeffectiveteaching’...........................82Figure19: Causalpackageandunderlyingassumptionsof‘improved,girl-friendlyschoolenvironment’
...............................................................................................................................................85Figure20: Causalpackageandunderlyingassumptionsof‘improvedlearningoutcomes’...................87
Table1: OverviewofGEP3interventionsduringthe2014–2017pilotperiod......................................2Table2: Priorityevaluationquestions...................................................................................................3Table3: SchoolcharacteristicsinpublicprimaryschoolsandIQSs....................................................16Table4: Summaryofteachercharacteristics......................................................................................18Table5: Summaryoffacilitatorcharacteristics...................................................................................34Table6: Summaryofheadteachercharacteristics.............................................................................41Table7: SummaryofCBMCcharacteristics.........................................................................................44Table8: Targetedandfinalsamplesizesoftheearlylearninginterventionbaselinesurvey.............70Table9: TargetedandfinalsamplesizesoftheIQSSbaselinesurvey................................................71Table10: School-levelbalance(allheadteachersinpublicprimaryschoolsandIQSs)........................72Table11: Teacher-levelbalance(allteachersinpublicprimaryschoolsandIQSs)..............................73Table12: Pupil-levelbalance(allpupilsinpublicprimaryschoolsandIQSs)........................................74Table13: Regressioncoefficientsandtheirstatisticalsignificanceofthemainfactorsinfluencing
Hausascaledscores...............................................................................................................75Table14: Regressioncoefficientsandtheirstatisticalsignificanceofthemainfactorsinfluencing
Englishscaledscores..............................................................................................................76Table15: Hausaschoolfixedeffectsmodel..........................................................................................77Table16: Englishschoolfixedeffectsmodel.........................................................................................77
Box1: RANAactivitiesatschoolandcommunitylevel.....................................................................12Box2: Measurementofteachermotivation....................................................................................22Box3: Learningoutcomemeasurement..........................................................................................24Box4: Explanatoryfactorsincludedinthepupilscorrelation..........................................................26Box5: IQSSactivitiesatschoolandcommunitylevel.......................................................................28Box6: EvaluationquestionsforGEP3’searlylearningintervention................................................68
©EDOREN xv
Box7: EvaluationquestionsforGEP3’sIQSS....................................................................................68
©EDOREN xvi
Listofabbreviations
ASC AnnualSchoolCensus
CAPI Computer-assistedpersonalinterviewing
CBMC Centre-BasedManagementCommittee
DAC DevelopmentAssistanceCommittee(OECD)
DEEPEN DevelopingEffectivePrivateEducationinNigeria
DFID UKDepartmentforInternationalDevelopment
EDOREN EducationData,ResearchandEvaluationinNigeria
EMIS Educationmanagementinformationsystem
ESSPIN EducationSectorSupportProgrammeinNigeria
FTTSS FemaleTeacherTraineeScholarshipScheme
G4G GirlsforGirls
GEP3 GirlsEducationProjectPhase3
GESC GirlsEducationSteeringCommittee
HiLWA High-LevelWomenAdvocates
HWI HouseholdWealthIndex
IQS IntegratedQur’anicSchool
IQSS IQSsupport
IQTE Islamiyya,Qur’anicTsangayaEducation
LGA LocalGovernmentArea
MS MothersAssociation
NCE NationalCertificateinEducation
NGN NigerianNaira
NGO Non-governmentalorganisation
OECD OrganisationforEconomicCo-operationandDevelopment
OPM OxfordPolicyManagement
P1 Primary1
RANA ReadingandNumeracyActivity
RCT Randomisedcontroltrial
SAME StateAgencyforMassEducation
SBMC School-BasedManagementCommittee
SCCE SeniorSchoolCertificateExamination
SUBEB StateUniversalBasicEducationBoard
ToC Theoryofchange
UNICEF UnitedNationsChildren’sFund
WCDP WholeCentreDevelopmentPlan
©EDOREN 17
1 Introduction
ThisreporthasbeenpreparedbyEDOREN.EDORENiscontractedbyDFIDNigeriatoevaluateGEP3,inlinewiththeGEP3evaluationframework(EDOREN,2015).GEP3isaneight-yearproject(2012–20)thatseekstoensuremoregirlscompletebasiceducationandacquireskillsforlifeandlivelihoodsinfivenorthernNigerianstates.ItismanagedbyUNICEFandfundedbyDFID.
Thisreportprovidesasynthesisofthefindingsofthebaselinedatacollectionaspartofamulti-yearevaluationofGEP3.ItisaccompaniedbyacomprehensiveBaselineTechnicalReport,whichincludesadetailedpresentationoftheanalysis,methodologyandinterventionsbeingevaluated.Theobjectiveofthesynthesisreportistoprovideasummaryofthekeybaselinefindingswithoutenteringintotechnicaldetails.
ThescopeofEDOREN’sevaluationworkconsistsofthreepiecesofanalysis:
• ahigh-levelexplicationandexaminationofGEP3’sToC;
• animpactevaluationofGEP3’searlylearningintervention;and
• aperformanceevaluationofGEP3’sIQSS.
Theobjectivesofthebaselineanalysisare:
• tocapturetheeducationsituation,andperceptionsofthissituation,atthestartofthere-designedGEP3projectsothatchangescanbemeasuredduringfollow-updatacollectionroundsandprojectattributionorcontributioncanbeassessed;
• toanswerspecificevaluationquestionsabouttherelevanceoftheproject;and
• toprovideevidencetoinformtheGEP3ToCandprojectimplementation.
InlinewiththescopeofEDOREN’sevaluationwork,thereportisstructuredasfollows:Chapter2providesasummaryofGEP3anditsevaluationframework.Chapter3presentsthemainfindingsofthebaselineassessmentoftheGEP3ToC.Chapter4startswithabriefpresentationofGEP3’searlylearninginterventionandtheimpactevaluationmethodology,followedbythemainfindingsofthebaselinedatacollection.Chapter5summarisesthebaselinefindingsoftheevaluationofGEP3’sIQSS.Chapter6concludes.
©EDOREN 18
2 Projectandevaluationbackground
2.1 Objectives,scopeandinterventionstrategy
GEP3isaneight-yearproject(2012–20)thatseekstoensuremoregirlscompletebasiceducationandacquireskillsforlifeandlivelihoodsinfivenorthernNigerianstates:Katsina,Sokoto,Bauchi,NigerandZamfara.ItismanagedbyUNICEFinpartnershipwithfederalandstategovernmentsandfundedbyDFID.Theprojectaimstoimproveaccess,retentionandlearningoutcomesforgirls,andtoreducethedisparitiesbetweengirls’andboys’educationoutcomes.OverthelongtermtheprojectseekstocontributetoimprovedsocialandeconomicopportunitiesforgirlsinnorthernNigeria.
In2014theprojectwasre-designedanditsToCrevised,resultinginareducednumberofinterventions.GEP3worksonapilot-to-scale-upapproach.Overtheperiod2014–2017interventionsarebeingpilotedin210publicprimaryschoolsand200IQSsineachstate.2Duringthispilotperiod,interventionsaremainlyfundedbyGEP3.BytheendofthisperiodGEP3aimstosecurestategovernments’buy-intoinvesttheirownresourcestoscaleuppilotedinterventionsthathavehaddemonstrableresults.
GEP3supportsinterventionsinthreeoutputareas:
• Output1:increasedenrolmentandretentionofgirlsinbasiceducation;
• Output2:improvedteachingandlearningsupportforgirlsinbasiceducation;and
• Output3:improvedgovernancetostrengthengirls’education.
Intermsoffinances,40.7%oftheGEP3budgetisallocatedtooutput2,26.6%tooutput3and12.9%tooutput1.
Table1providesanoverviewoftheGEP3interventionsthatarebeingpilotedin2014–2017inthedifferentstates.GEP3’sinterventionsareintendedtobebroadlydeliveredthroughgovernmenteducationstructuresatstateandlocalgovernmentlevels.Governmentbodiesattheselevelsthereforeconstitutetheprogramme’sprimarystakeholders.
Table1: OverviewofGEP3interventionsduringthe2014–2017pilotperiod
Output Intervention States
Output1
Unconditionalcashtransferprogrammeinpublicprimaryschools NigerandSokoto
Enrolmentdrivesincatchmentareasofpublicprimaryschools Allstates
GirlsforGirls(G4G)groupsinpublicprimaryschoolsandIQSs Bauchi,Katsina,Zamfara
Output2
TeachercapacitydevelopmentinpublicprimaryschoolsandIQSsPublic primary schools:BauchiIQSs:Bauchi,Niger,Sokoto
HeadteachertraininginpublicprimaryschoolsandIQSsPublic primary schools: allstatesIQSs:Bauchi,Niger,Sokoto
EarlylearninginterventioninpublicprimaryschoolsandIQSs KatsinaandZamfara
Output3
CapacitybuildingofSBMCs/CBMCsinprimaryschoolsandIQSs(includingschoolgrants)
Public primary schools: allstatesIQSs:Bauchi,Niger,Sokoto
SupporttotheASC,theEMISandIQSdatacollection Allstates
Enhancedpoliticalengagementforgirls’educationthroughGESC Allstates
2InKatsinaandZamfara,interventionswillonlytakeplacein40IQSsineachstate,basedonaninitiallistof200pilotschools.
©EDOREN 19
SupporttoHiLWAmembers Allstates
AdvocacyforFemaleTeacherTraineeScholarshipScheme(FTTSS) Allstates
2.2 ObjectivesandscopeofGEP3evaluation
ThemainobjectiveoftheGEP3evaluationistoassesswhetherandhowwellinterventionscontributetoGEP3’sobjectives.InlinewithGEP3’semphasisonlearningoutcomes,theevaluationwillprioritisemeasuringchangesinlearningoverchangesinaccesstoeducation.Theevaluationfocusesoninterventionsthatabsorbahighshareofthebudget,havearelativelyweakevidencebase,areinnovativeinnatureandareofstrategicinteresttoGEP3managementandDFID.Thesearetheearlylearningintervention,GEP3’sIQSSandthecashtransferprogramme.ThefirsttwowillbesubjecttoEDOREN-ledevaluationanddatacollection,withGEP3researchstudiesprovidingadditionalevidence.EvidenceforthethirdinterventionwillbegeneratedthroughresearchstudiesandmonitoringconductedunderGEP3.
2.3 Evaluationdesignandmethods
During2014–2015,EDOREN,incollaborationwithUNICEFandDFIDNigeria,developedanevaluation
frameworkinparallelwithGEP3’sre-design.Theevaluationframeworkdefinedtheobjectives,scope,questions,designandpartnerresponsibilitiesoftheevaluation.Theevaluationfocusesontheperiod2014–2017inordertoinformdecision-makingaroundscale-up.Therestofthissectionoutlinesthemainelementsoftheevaluationframework.SubsequentchapterscontainmoreinformationonthedesignofthedifferentevaluationcomponentsledbyEDOREN.
2.3.1 Evaluationquestions
EvaluationquestionswereselectedconsideringtheOrganisationforEconomicCo-operationandDevelopment(OECD)DevelopmentAssistanceCommittee’s(DAC)criteriaforevaluatingdevelopmentassistance.Table2providesanoverviewofthekeyevaluationquestions.3ThefirstthreequestionswillbeaddressedthroughEDOREN-ledevaluation;thequestionrelatedtothecashtransferprogrammewillbeaddressedbyUNICEF-commissionedindependentresearch;andtheefficiencyquestionwillbeaddressedthroughtheDFID-managedannualreviewprocess.ThisreportfocusesontheEDOREN-ledevaluation.
Table2: Priorityevaluationquestions
DACcriteria Evaluationquestion
Relevance • IsGEP3’sToCappropriatetoachieveitsplannedobjectiveswithinthecontextof
theGEP3states?
Effectiveness/
Impact• Towhatextenthastheearlylearninginterventionimprovedlearningoutcomesin
theearlygradesofschool,especiallyforgirls?
• HowwellhasGEP3’sIQSScontributedtobetterlearningoutcomes,especiallyforgirls,byimprovingteachingandschoolleadershipandmanagement?
• Howhasthecashtransferprogrammechangedattitudesandbehaviourinregard
togirls’educationamongrecipientcaregivers?
Efficiency • Howwellhaveprojectorganisationalprocessesfacilitatedthedeliveryofexpected
outputs?Howcouldsuchprocessesbeimproved?
3TheGEP3EvaluationFrameworkdocumentpresentsacomprehensiveoverviewofallevaluationquestionsidentified(EDOREN2015).
©EDOREN 20
2.3.2 Evaluationdesignandmethods
Theevaluationquestionswillbeansweredbyapplyingvariousevaluationapproachesusingmixed-methodsdatacollection.Approacheshavebeenselectedinlinewiththecontext,availableresourcesandmethodologicalconstraints.Theuseofatheory-basedapproachisparticularlyappropriatefortheevaluationofGEP3asithelpsunpickhowGEP3interventionsaremakingadifferencewithintheirspecificcontext.Thisisusefulformakingjudgementsaboutwhethertheoutputsandoutcomesobservedduringthepilotphasewouldbelikelytooccurduringscale-up,whichinturnwouldinformdecisionsrelatedtoscale-up.Furthermore,itallowscrediblejudgementstobemaderegardingthecausalrelationshipbetweenGEP3interventionsandtheiroutcomes,evenwhenavalidcounterfactualcannotbeconstructed.
EDORENwillimplementthreesetsofevaluationmethodologiesandactivitiestoanswertheevaluationquestionsthatareitsresponsibility:
• Thefirstsetofevaluationactivitiescentreonahigh-levelexplicationandexaminationofGEP3’sToCbasedonstate-levelKIIsanddocumentanddatareview.
• Second,EDORENwillconductanimpactevaluationoftheearlylearninginterventionbasedonarandomisedcounterfactualdesignusingmainlyquantitativeschoolsurveys.
• Third,GEP3’sIQSSwillbeevaluatedusingamixed-methodsapproachbasedontheprinciplesofcontributionanalysis.
ThedifferentdesignsaresummarisedintheirrespectivechaptersanddiscussedextensivelyintheBaselineTechnicalReportaccompanyingthissynthesisreport.
2.3.3 Inclusionandethics
TheevaluationadherestoDFID’sprinciplesofethicsinresearchandevaluation(DFID,2011)andfollowsUNICEF’sguidelinesonconductingresearchwithchildren(Grahametal.,2013).Stepstakentoensureethicsandinclusioninclude:ethicsreviewoftheevaluationdesignanddatacollectionprotocols;4attentiontoage,mothertongueanddisabilityduringdatacollectionwithchildren;anavoidanceofgenderbiasinmeasurement;5anevaluationdesignthatensuresdisaggregationbygenderwithsufficientstatisticalpower;analysisoflearningthatfocusesonpupilsatthelowendofthelearningdistribution;andintensivefieldresearchertraining.
2.3.4 Externalvalidityofthefindings
ThefindingsofthequantitativebaselinesurveysarerepresentativeoftheGEP3schoolsinwhichtheearlylearningandIQSSinterventionsarebeingimplemented.TheearlylearninginterventionisbeingimplementedinasubsetofGEP3LGAsthatwerepurposivelyselectedfortheintervention.ThequalitativecomponentoftheIQSSevaluationwasnotdesignedtoproduceresultsthataregeneralisable,butinsteadprovidesinsightsthatcomplementthequantitativefindings.Thetheory-basedapproachenablesustounpickhowandunderwhatconditionsimpactonlearningoccurs,whichallowsformoregeneralisablefindingsthatgobeyondtheinterventiongroups.
4TheevaluationdesignanddatacollectionprotocolswereapprovedbyOPM’sEthicalReviewCommittee.Inaddition,datacollectionprotocolsweresubmittedtotheNationalHealthResearchEthicsCommitteeofNigeria,whichprovidedexemption.5ARaschpsychometricanalysisofpupiltestitemswasundertakeninordertoidentifywhethertestitemswouldbeinterpreteddifferentlybygirlsorboys.Itwasfoundthattheitemsdidnotdiscriminateonthebasisofgender.
©EDOREN 21
3 BaselineToCassessment
3.1 Methodology
ThischapterpresentsthebaselineassessmentofGEP3’sToC.ItfocusesontwoevaluationquestionsrelatedtotherelevanceofGEP3:
• HowplausibleisGEP3’sToCinthecontextoftheGEP3states?
• HowappropriateareGEP3interventionsintermsofimplementationstrategy?
TheplausibilityoftheToCisexaminedby:(i)reviewingstakeholders’commonunderstandingofitsintendedoutcomes;(ii)interrogatingthelogicofitsoutcomechain;and(iii)makingexplicitthosefactorsthatareeithernecessaryfor,orahindranceto,achievingthestatedoutcomes.TheappropriatenessofGEP3’simplementationstrategyisassessedonthefollowingdimensions:(i)stakeholderinvolvementinGEP3implementation;(ii)implementationcapacity;and(iii)thelikelihoodthattheprogrammewillreachthetargetpopulationinanequitablemanner.
ThisassessmentisbasedonKIIs,largelywithGEP3state-levelstakeholders,conductedinAugust2015.KIswerepurposivelyselected,withsupportofGEP3stateteams,basedontheirknowledgeoftheprojectandspecificinterventions.KIsweregovernmentstaffinmanagementpositionsintheeducationministries,departmentsandagenciesthatGEP3partnerswith;representativesofgovernmentornon-governmentagenciesinvolvedinGEP3implementation(includingGEP3stateteamsandUNICEFfieldofficestaff);andnon-governmentinformantsthatcouldprovidefurtherinsightsregardingspecificinterventions.
3.2 PlausibilityofGEP3’sToC
Summaryanswerstoevaluationquestions
Q1:HowplausibleisGEP3’sToC
inthecontextoftheGEP3
states?
Theplausibilityofthere-designedGEP3ToCisbackedbyitspotentiallysynergetic,coherentlogicandstakeholders’commonunderstandingofitsmainobjectivesandoverarchingstrategy.Criticalassumptionsrelatedtothemanagementandresourcingoftheeducationsystemarehighlyuncertain–inparticular,thereleaseofgovernmentfunding,schoolinvestmentandhumanresourcecapacityremainingonaparwithincreasedenrolment,effectivemonitoringatschoolandinterventionlevel,andSBMC’sabilitytodeliveronitsmultipleprojectresponsibilities.
SBMCsandMothersAssociations(MAs)haveapivotalroleinthesuccessofmanyinterventions.SBMC/CBMC/MAfunctionalityandcapacitytoundertakethemultiplerolesassignedaspartofGEP3interventionsisacriticalassumptionofGEP3’sToCandrequiresclosemonitoring.
TheplausibilityofGEP3’sscale-upapproachisdependentonuncertaingovernmentfundingandvariesbyintervention.Thescale-upoftheIQSSisparticularlyuncertainbecausetheinstitutionalmandateoverIQSsisunclear,andduetotheSAME’slimitedanduncertainaccesstoresources,thelimitednumberofwell-establishedIQSsavailableforscale-up,andsupply-sideconstraints(particularlywithregardstofacilitators).
©EDOREN 22
3.2.1 Generalassessment
StakeholdersgenerallyhaveacommonunderstandingofGEP3’sintendedoutcomesandoverarching
strategy,butlackanunderstandingoftheproject’soperationaldesign.Stakeholderscommonlyrecogniseincreasedgirls’enrolmentasthemainintendedoutcomeofGEP3.Whilstretentionandlearningareconsideredimportant,theyarereferredtolessfrequently,particularlyamonggovernmentKIs,whichsuggestsGEP3isforemostassociatedwithimprovinggirls’accesstoschool,representedbyitstargetofgetting1millionadditionalgirlsintoschool.StakeholdershaverelativelygoodknowledgeofGEP3’soverallstrategyandacknowledgethatthere-designhasimprovedtheproject’sfocus.Atthesametime,theproject’soperationaldesignisnotalwaysunderstood.Attributesofintervention-specificoutputsandoutcomes—suchasprioritytargetgroups,thespecificationandmeaningofoutputs/outcomes,andtimelines—areoftennotcleartostakeholders.ThiscouldmakeitdifficultforthemtoassessGEP3’sperformanceorbeheldaccountablefortheresultsachieved.
GEP3’sdesignisperceivedtobecoherent,withpotentialsynergiesbetweenoutputsandintervention.
However,thesesynergiesalsoconstituteariskofinterventionproblemsspreadingacrossthe
programme.GEP3seekstoholisticallyimproveeducationaccess,qualityandgovernanceinthefivestatesinwhichitworks.Therearesynergiesacrossitsinterventions,whichhavethescopetoreinforceoneanother’soutcomes.However,thisalsomeansthatifacertaininterventionispoorlyimplemented,thiscouldunderminetheimpactofotherGEP3interventions.Forinstance,successfuleffortstobuildthecapacityofSBMCswouldsupporttheoutcomesofinterventionsseekingtoimproveaccess,suchasenrolmentdrives,whichareimplementedbySBMCs.Conversely,ifeffortstobuildSBMCs’capacitywereunsuccessful,thiswouldunderminetheimplementationandefficacyofenrolmentdrives.
Stakeholdersemphasisedthatthetimelyreleaseoffundsbystategovernmentsisacentraluncertain
factorthatcouldunderminethesuccessoftheproject.Stakeholdersacknowledgethattheprojectre-designhasmitigatedtheriskofstategovernmentsnotprovidingtheircommittedfinancialcontributions.Thishasbeenachievedthroughanemphasisonhigh-leveladvocacyandafocusonamoremanageablesetofoutcomes.The2015electionsarealsoseentohaveresultedinincreasedstatesupporttoeducation.Nonetheless,stakeholdersindicatethatthenon-releaseoffundsremainsanimportantconcern,withfundreleasedependentonthedecisiveinfluenceofthegovernor.
EffectivemonitoringwashighlightedbyseveralKIsaskeytothesuccessoftheproject.ThiscorrespondstobothmonitoringandoversightofschoolactivitiesbytheSBMC,headteacherorlocal/stategovernmentofficers,aswellasmonitoringofGEP3projectimplementation.SomeKIslinkthistoaneedforinterventionstohaveastrongerresultsorientation,withpeopleheldtoaccountforresults.KIsacknowledgethatmonitoringsystemsareweak,althoughinvestmentshavebeenmadeinthepasttostrengthenmonitoringandqualitycontrol.6Therefore,GEP3rightlyemphasisesmonitoringinitsToC.Effectivemonitoringrequiresacommonunderstandingofitsobjectives,rolesandresponsibilities,andapproach,andsufficientresourcesforimplementation.
Contextualfactorsarelikelytoinfluenceprojectoutcomes.InsecurityandconflicthavehadanimpactonschoolinginBauchi,ZamfaraandKatsina.WhilestakeholdersindicatedthatGEP3schoolswerenotstronglyaffectedatthetimeoftheinterviews,insecuritycontinuestoposeriskstotheproject.GovernmentpoliciesandstructuresmayalsoleadtovariationsinGEP3outcomesacrossstates.Forexample,somestateshaveagenciestargetinggirls’education.Theseagenciesmayintroduceinitiativesthatcouldinfluencegirls’educationoutcomes.Similarly,otherdonor-fundedprojectsoperatingintheGEP3statescouldcontribute
6InSokoto,schoolqualityassuranceofficershavebeenscreenedandtrainedwiththesupportoftheNorthernEducationInitiativeproject.InZamfara,theSUBEBrecentlysuppliedqualityassuranceofficerswithnewmotorcyclesandfuelallowances,accordingtooneKI.InBauchi,theSUBEBincreasedthenumberofitszonalofficesfromthreeto10,soastohavemorequalityassuranceunitsclosertotheschools.
©EDOREN 23
tosomeoftheproject’soutcomes.ThiswillmakeitdifficulttoattributeimprovementsineducationoutcomestoGEP3.
3.2.2 Output1:Increasedenrolmentandretentionofgirlsinbasiceducation
Achievingtheproject’sobjectiveofincreasinggirls’enrolmentisplausible,particularlyinprimary
education.Stakeholderswereoftheviewthatenrolmentdrivesandcashtransfersarebotheffectivestrategiesthataddressimportantbarrierstogirls’education,andthattheycomplementoneanother.Enrolmentdrivesareaninterventionthatissupportedbystakeholdersatdifferentlevels.Thesedrivesareabletopenetratethecommunities,withtheactiveinvolvementofthosecommunities.Similarly,thecashtransferinterventionscancountonahighlevelofinterestinthetargetedcommunities,accordingtotheprojectimplementationunits,whichhavereceivedstrongcapacitysupporttosuccessfullyimplementtheintervention.Fundingdeficienciesresultingindelayedcashtransfersconstitutearisktoachievingenrolmentobjectives.Inareaswherebothenrolmentdriveandcashtransferinterventionsareimplemented,girls’enrolmentisexpectedtoincrease.AstheGEP3re-designhasshiftedtheproject’sfocusfrombasiceducationtoprimaryeducation,favourableresultsareexpectedtobeachievedinprimaryeducation.
Retentionisuncertainduetosupply-sidechallenges.Stakeholderswereoftheviewthatitwillbemoredifficulttoretaingirlsinschool.Retentionisshaped,amongotherthings,bysupply-sidefactors,suchasthepresenceofsufficientteachers,thequalityofteaching,andthequalityoftheschoolenvironment.AkeyassumptionoftheGEP3ToCisthatgovernmentcansupplyprimaryschoolsandteacherstomeetincreasingdemand.InallstatesKIsperceivedthatincreasedenrolmentisnotsufficientlyaccommodatedintermsofkeyresourcessuchasteachersandclassrooms,althoughsomeseethisasaproblemthatispresentmainlyinurbanareas
SBMCsandMAsareconsideredcentraltothesuccessofinterventionstoimproveaccess.StakeholdershighlightthecentralrolethatSBMCsandMAsplaynotonlyinsensitisingcommunitiesabouttheimportanceofgirls’schoolingbutalsoinfacilitatinginterventionimplementationandsupply-sideresponsestoincreasedenrolment.TheirfunctionalityandcapacitytoundertakethesemultiplerolesisacriticalassumptioninGEP3’sToC,andonethatrequiresclosemonitoring.ThisisreflectedinGEP3’sSBMCeffectivenessmonitoringsystem.
3.2.3 Output2:Improvedcapacityofteacherstodelivereffectivelearningforgirls
Thequalityofteachingandteachersisconsideredcentraltoimprovinglearningoutcomes.Stakeholdersarelesslikelytorecogniselearningasakeyexpectedprojectoutcome,thanenrolmentandretention.Whentheydo,theyacknowledgethatlearningoutcomesaremoredifficulttoachievethanincreasingaccessforgirlsduetothelowqualityofteaching,particularlyinruralareas,andalackofinvestmentinthesupplyofqualityeducation.ThereisawideconsensusamongtheKIs,inlinewithGEP3’sToC,thattacklingqualityofteachingandteachersiscentraltoimprovinglearningoutcomes.Factorssuchasteachercommitmentandmotivation,theirdeploymentandtransfers,initialskillsandknowledge,theschoolenvironment,andmonitoringandsupervisionarelikelytoplayanimportantroleindeterminingthefinaloutcomeofteachercapacitydevelopmentinterventions.
Stakeholdersarenotsufficientlyawareofthemoreconcreteattributesofthetrainingapproachand
outcomes.Althoughtheteachercapacitydevelopmentcomponenthasbeenre-designed,atthetimeoftheinterviewsgovernmentstakeholdersinvolvedinteachertrainingdidnotyetunderstandwhatcapacitiesneedtobedeveloped,inwhatways,andwhen.WhileawelltargetedteachercapacitydevelopmentapproachispostulatedasanassumptionintheToC,whatthistargetingconcretelyentailsremainsunclear.TheplausibilityoftheToCwouldbeincreasedifsuchattributesweremorecommonlyunderstood,sotheycanbemonitoredandaccountedfor.
©EDOREN 24
ThewidevariationinthebackgroundofIQSfacilitatorswarrantsspecificattentionandadaptation.7ThereappearstobelargevariationinthebackgroundoftheIQSfacilitators,andamongtheIQSsinwhichtheyareteaching,intermsoffacilitators’teachingexperience,teachinghours,theavailabilityoflearningmaterials,teachingfacilities,pupilcharacteristicsandthetypeofIQS.HeadteachersupportwillvaryintermsofhowthisroleisfilledandbywhomsinceaheadteacherpositionisnotnecessarilyaseparatepositionfromtheproprietororMallamintheIQScontext.Therefore,notonlydoesthecapacitybuildingapproachneedtobeadaptedtotheIQScontext,butitalsoneedstotakeintoaccountlargevariationsacrossIQSs.RemunerationofIQSfacilitatorsappearstobeparticularlyproblematicforIQSfacilitators.ThebaselinefindingsoftheIQSSevaluationconfirmthataverysmallshareoffacilitatorsarepaid,despiteanallowancebeingpolicyinthestates.Whilefacilitatorsarenotmotivatedsolelybyremuneration,thebaselinefindingsfindapositiveassociationbetweenreceivingremunerationandteachermotivation,andindicatethatschoolstakeholdersstatethatitisdifficulttoholdnon-paidfacilitatorstoaccount.
Awiderangeofstakeholdersunderlinetheimportanceoftheearlylearningintervention.Stakeholderssupporttheemphasisonliteracyduringearlygradesasafoundationalskill.Thereisnooverallconsensusamongstakeholdersabouttheextenttowhichtheuseofamothertongueisanecessaryconditiontoimprovelearningoutcomes.ThebaselinefindingsoftheearlylearningevaluationhasconfirmedthatHausaisthelanguageoftheimmediateenvironmentinKatsinaandZamfara,whichvalidatestheToCassumptionforthisintervention.However,whileallteacherssurveyedreportedspeakingHausa,Hausaliteracylevelsamongteachersarelow,whichneedstobetakenintoaccountinthedevelopmentofHausateachingandlearningmaterials.ImplementingearlylearninginIQSswillrequirecarefuldesignbecauseofthedifferenceingradelevelscomparedtopublicprimaryschools,theuseofamothertongueinthisgradestructure,andthevarietyofwaysinwhichformaleducationisorganisedintheIQS.
3.2.4 Output3:Improvedgovernancetostrengthengirls’education
GEP3providessupportattheschoollevelandthestateleveltoimprovegovernancetostrengthengirls’education.
Attheschoollevel,SBMCempowermentisbothapivotalintermediaryoutcomeinGEP3’sToCaswellas
oneofitsmostprecariouslinks.TheexpectationsplacedonSBMCs,intermsofrolesandresponsibilities,areveryhigheventhoughthecapacitybuildingandempowermentprocesswillbegradual,andoftenstartsfromalowbase.ThecompositionoftheSBMCs,theiracceptanceintheschoolandcommunity,theiraccesstoschools,andtheinstitutionalisationofgovernmentsupporttoSBMCsarekeyfactorsthatwillinfluencetheircapacitydevelopment.TheyfaceamorechallengingsituationinIQSssincetheirempowermentisbasedonalargersetofassumptions,suchasSBMCsbeingestablishedinthefirstplace,theirbeingacceptedbytheIQSproprietorandtheacceptanceofintegrationinthecommunity.ItisalsoassumedthatIQSsarenotmobileorthatiftheyare,thisdoesnotaffecttheirfunctionality.
StakeholderssupportthesignificantroleofSBMCsintheschool-levelgovernanceofeducationbutthe
scopeoftheirgovernanceroleremainstobeclarified.StakeholdersconsiderSBMCsakeyinstrumentbywhichtopromotecommunityownershipofschoolsandaddressdemand,supplyandgovernanceproblems.TheynotethatchannellingfinancialsupporttoschoolsthroughSBMCs(forinstancethroughtheprovisionofGEP3grants)canmakeanimportantcontributiontotheirempowerment.WhileSBMCs’roleinschool-levelgovernanceisbroadlysupported,theirroleinregardtoinfluencingwidereducationprocesses—suchasholdingstateandlocalgovernmentstoaccountforservicedelivery—isconsideredpromisingbynon-governmentKIsbutremainsunaddressedbygovernmentKIs.WhileGEP3doesnotaimtosupportagovernanceroleforSBMCsbeyondtheschoollevel,thisislikelytohaveitslimitationsbecauseseveralaspectsofschools’performance(suchashumanresourcemanagement,provisionoflearningmaterials,schoolinfrastructure)dependonstateandlocalgovernmentaction.
7TeachersinIQSsarereferredtoasfacilitators.
©EDOREN 25
Atthestatelevel,HiLWAmembersengagewithdecision-makersandschoolcommunitiesandhavethe
potentialforindirectinfluence.HiLWAmembersseemtobeinapositiontoengagewithdecision-makerstopromotewomen’sandgirls’participationintheeducationsector.TheyalsohavethescopetopromoteGEP3’soutcomesattheschoollevelthroughtheirinvolvementinenrolmentdrivesandengagementwithSBMCs.HiLWAmembers’influencecomesfromthecredibilityoftheirmembers.Theircapacitytoinfluenceismainlyindirectandshouldnotbeoverstatedasmostmembersdonotholdpositionsofdirectinfluence.
TheGESCsoffersamulti-stakeholderplatformtosupportgovernancearoundgirl’seducationtothe
extentthatitisactive,sufficientlylocallyownedandaddresseskeychallenges.GEP3seekstoimprovegovernancethroughtheGESCsatthestateandnationallevels.Stakeholdersappreciatethemulti-sectorandmulti-stakeholdercharacteroftheGESCs.TheGESCsmetinfrequentlyduringthefirstpartof2015duetotheprojectre-designandelections,althoughinseveralstatesitseemstohavebeenreactivated.Itseffectivenesscanlikelybestrengthenedbymoregovernmentownership.ItwouldalsobeinabetterpositiontocontributetoGEP3’soutcomesifmonitoringandpoliticalengagementaroundGEP3fundingwereexplicitlyincorporatedinitsmandateandplannedfor.
GEP3’ssupporttoEMISandASCwillplausiblyimprovedatareliability,althoughthismaynotnecessarily
translateintodatabeingusedinpolicy-makingprocesses.ThecapacityneedsintermsofgeneratingreliabledataarecommonlyunderstoodandGEP3supportseemstobewelltargeted.Althoughstakeholdersexpectthatreliabledatawillbeusedinpolicy-makingprocesses,GEP3’sToCdoesnotconsidertheinterfacebetweendataproductionandusageandleavesthetranslationofdataproductionintouseasanimplicitassumption.ItwouldbeworthexaminingwhetherthedemandsideofEMISneedsstrengtheninginorderachieveeffectiveuseinevidence-basedplanning,resourceallocationandpolicyorientationtoimprovegirls’education.IncorporatingIQSdatainEMIS(beyondthoseIQSsthatarewellalignedwiththeformaleducationsystem)doesnotseemplausibleatthismoment,notleastbecauseofalackofreliableschoolidentificationdataandthecapacityconstraintsfacedbySAMEsinregardtoregularlyandsystematicallycapturingIQSdata.
3.2.5 GEP3’spilot-to-scale-upapproach
Thenatureofscale-upisnotcommonlyunderstoodanditsimplementationisdependentonuncertain
governmentfunding.AkeyassumptionunderlyingGEP3’sstrategyisthatstategovernmentsarewillingandabletoimplementlarge-scalechangeforgirls’education.GovernmentfundingavailabilitywascitedasthemainuncertainconditionforGEP3scale-up.Stakeholdersalsodidnothaveaclearunderstandingofhowscale-upwouldhappen.Limitedplanninghadtakenplaceatthetimeoftheinterviewsin2015.StateGEP3teamsandUNICEFstaffinterviewedrecognisethatplanningforscale-upisimportantandneedstostartwellaheadof2017.
Theplausibilityofscale-upvariesacrossinterventions.Thescale-upofteacherandSBMCcapacitybuildingisrelativelymoreplausiblebecausetheneedfortheseinterventionsiswellrecognisedandasupportinginstitutionalframeworkisinplace.However,scalingupthecashtransferschemeislessplausiblebecauseitiscapitalintensive,isnotwellinstitutionalised,andisseenasadonorpriority.Thescale-upofIQSSisparticularlyuncertainbecauseofalackofclarityaboutwhichinstitutionisresponsibleforintegration,theSAMEs’limitedanduncertainaccesstoresources,thelimitednumberofwell-establishedIQSsavailableforscale-up,andsupply-sideconstraints–particularlywithregardstofacilitators.
©EDOREN 26
3.3 AppropriatenessofGEP3’simplementationstrategy
Summaryanswerstoevaluationquestions
Q2:HowappropriateareGEP3’s
interventionsintermsof
implementationstrategy?
Theprojectismakinganappreciatedefforttoinvolveavarietyofstakeholders,althoughtheirinvolvementinoperationalplanningremainsunevenacrossinterventions.
Governmentcapacitybuildingisembeddedinprojectdesign,butaneffectivesystemisneededtoidentifyandaddressthemostimportantcapacitydevelopmentneeds.MonitoringcapacityisrightlyemphasisedinGEP3’sre-designbutitsoperationalisationhasyettoadvance.
SeveralGEP3interventionsaredesignedtobespecificallyequityenhancing,butempiricalverificationisrequiredastowhetherthemostvulnerablehouseholdsandgroupsareabletobenefitandactivelyparticipateinGEP3interventions.GEP3isnolongeroperatinginLGAswiththehighestgendergapinallstates.
3.3.1 Stakeholderinvolvement
Theprojectismakinganappreciatedefforttoinvolveavarietyofstakeholders,althoughtheir
involvementinoperationalplanningremainsunevenacrossinterventions.GEP3’soperationalplansupportsstakeholderownershipbyemphasisingimplementationwithandthroughstateeducationpartners.TheoperationofGEP3fromwithinstategovernmentscontributestogovernmentinvolvementandcoordinationbetweentheGEP3stateteamandgovernmentstaff.However,governmentinvolvementremainstosomeextentexternallydrivenandgovernmentpartnersarenotalwaysawareof,orlackclarityabout,operationalplanning.Theoperationalisationofstructuresandsystemsthatfacilitategovernmentinvolvement,liketheGESC,havebeeninfluencedbychangesinstaffingandtheelectionsin2015.Stakeholdersconsidertheinvolvementofabroadrangeofstakeholdersatdifferentlevels(community,local,stateandnationallevel)astrengthofGEP3’simplementationstrategy(inparticulartheparticipationofSBMCsandMAs).
3.3.2 Governmentcapacity
Governmentcapacitybuildingisembeddedintheprojectdesign,butaneffectivesystemisneededto
identifyandaddressthemostimportantcapacitydevelopmentneeds.GEP3providesdirectsupporttogovernmentstaffandmobilisesspecifictechnicalexpertisetoincreasegovernmentcapacity.Thisisdemonstratedmoreatthestatethanatthelocallevel,andvariesacrossinterventions.Giventhelimitedresourcesandthecapacityconstraints,particularlyatthelocallevel,identifyingandaddressingwherecapacityneedsarehighest,isessential.GEP3’simplementationislikelytobeinfluencedbyinstitutionalcapacityconstraints,particularlythelackofclarityregardingthedivisionofrolesandresponsibilitiesfortheintegrationofformaleducationinQur’anicschoolsacrosstheSAMEsandStateUniversalEducationBoards(SUBEBs).
MonitoringcapacityisrightlyemphasisedinGEP3’sre-designbutitsoperationalisationhasyetto
advance.Moststakeholdersnotedthatmonitoringandsupportivesupervisionareessentialforprojectsuccess.However,theyalsohighlightedthatthegovernmenthasinadequateorganisationalcapacitytomonitorinterventions.Therefore,GEP3hasrightlyemphasisedstrengtheningmonitoringcapacityaspartofthere-design.However,theresultsinthisregardwilldependonitsoperationalisation,whichhadnotyetadvancedatthetimeoftheinterviews.Thiswasindicated,forinstance,bythelowqualityinformationaboutGEP3-supportedIQSsthatwasprovidedforthebaselinesampling.
©EDOREN 27
3.3.3 Equity
GEP3isnolongeroperatinginLGAswiththehighestgendergapinallstates.EquityineducationisacentralconsiderationforGEP3,astheprojectintendstoreducegenderdisparitiesineducation.WhiletheprojecttargetedLGAswiththehighestgendergapineducationatthestart,atthetimeoftheGEP3re-designthetargetedLGAswerenolongeralwaysthosewiththehighestgendergap.Basedon2014–2015schoolcensusdata,GEP3ismostlyoperatinginLGAswitharelativelyhighprimarygendergapinprimaryeducationinBauchiandNiger,whileinKatsina,ZamfaraandSokotoGEP3isimplementedinLGAswithrelativelylowgenderdisparitiesinprimaryschools.
SeveralGEP3interventionsaredesignedtobespecificallyequityenhancing.Thecashtransferprogrammetargetsschoolswiththehighestproportionofout-of-schoolgirls.TheIQSSismeanttoexpandaccesstoqualitybasiceducationformarginalisedchildreninrurallocations.However,thedegreetowhichthemostvulnerablegroupsareactuallyreachedwillbeinfluencedbyhowequityconsiderationsareoperationalisedontheground.Forexample,empiricalverificationisrequiredastowhetherthemostvulnerablehouseholdsareabletoreceivethecashtransferasplanned.ActiveparticipationinSBMCs/CBMCsmaybelimitedtoonlyspecificcommunitymembers,duetotheconstraintsimposedbysocialnormsandschoolaccessibility.
©EDOREN 28
4 Baselineoftheearlylearninginterventionevaluation
4.1 GEP3’searlylearningintervention
GEP3’searlylearninginterventionaimstoimprovetheearlylearningskillsofchildreninprimaryGrades
1to3(P1–P3)inthemothertongue,whilealsopreparingchildrentolearnwithEnglishasalanguageofinstructionbythetimetheytransitiontoGrade4.Akeymeasureoftheintervention’ssuccesswillbeimprovedliteracyskills.Tothisend,UNICEFhascontractedathirdpartytoimplementtheRANAproject.
RANAwillbeimplementedoverathree-yearperiod(2016–2018)insixLGAsinZamfaraandKatsina
(threeLGAsperstate).Theinterventiontargets120publicprimaryschoolsand80IQSsthatformpartofGEP3’spilotschoollists(60publicprimaryschoolsand40IQSsperstate).ThemainactivitiesthatRANAwillimplementattheschoolandcommunitylevelarepresentedinBox1.
Box1.RANAactivitiesatschoolandcommunitylevel
• ProvisionofapackageofteachingandlearningmaterialsinHausa
• Earlygradeteacherin-serviceprofessionaldevelopment,includingcluster-leveltraining,weeklyschool-basedpeermentoring,on-sitemonthlysupervisorysupportandmonitoring,andheadteachertraining
• Communityawarenessandengagementactivities,includingcommunitysensitisationandtheappointmentofaliteracychampionamongtheSBMC/CBMCtocoordinateliteracyactivities
ThematerialsandteacherprofessionaldevelopmentwillberolledouttogradesP1andP2duringthesecondandthirdtermofthe2015/2016academicyear.P3willbecoveredfromthestartofthe2016/2017academicyear.TheinterventionpackagewillbesimilarforpublicprimaryschoolsandIQSs.TheRANApilotprojectisscheduledtoendinAugust2018.
AnnexApresentstheToCoftheearlylearningintervention.Itdescribesandvisualisesthecausalpathwaysofhowtheearlylearninginterventionisassumedtoimprovepupils’literacyinearlygrades,aswellasdiscussingunderlyingassumptions.ThemaincausalassumptionunderlyingtheToCisthatliteracylearningoutcomes,particularlyinHausaasthemothertongue,willimproveinearlygradesifteachingpracticesimprovethroughtheuseofimprovedteachingandlearningmaterialsandthepresenceofmoreknowledgeableandskilledteachers.
4.2 Baselinemethodology
TheevaluationofGEP3’searlylearninginterventiontakesatheory-basedapproach.Theintervention’sToCwasusedasaframeworktoformulatetheevaluationquestions(listedinAnnexB).ThesequestionsaddresstheimpactoftheearlylearninginterventiononarangeofintermediaryandfinaloutcomesidentifiedintheToC.Besidestheintervention’sToC,thechoiceofquestionshasbeenbasedonthefollowingconsiderations:anemphasisonmeasuringimpactonlearning;theinclusionofanequityperspectivebyseekingtounderstandtheimpactonthelowestperformingpupils;apreferenceforquantifyingattributableimpact;andafocusonthelinkagebetweenteachers’knowledgeandskills,andteachers’classroompractices,andlearning.
TheevaluationisdesignedasaclusteredRCT,stratifiedbyLGAandtypeofschool(primaryschoolvs.IQS),andrandomisedattheschoollevel.TheRCTdesignallowstheevaluationteamtomeasuretheattributableimpactoftheearlylearninginterventiononlearningoutcomesbycomparingoutcomechangesinatreatmentgroupofschoolswiththoseinacontrolgroupthatisstatisticallysimilaronaverage.Foreach
©EDOREN 29
typeofschoolanequalsizedsampleofschoolswasrandomlyselectedfromamongallGEP3schoolsineachofthesixGEP3LGAsinwhichRANAisimplemented.IneachLGA,halfofthesamplewasthenrandomlyassignedtoatreatmentgroupandtheotherhalftoacontrolgroup.TheRCTdesigniscombinedwiththeoverarchingtheory-basedevaluationapproach,whichmeasuresnetchangesinintermediaryandfinaloutcomesalongtheassumedcausalchain.Thiswillallowtheevaluationteamtounpackhowchangetakesplace.
Inordertomeasurechangesinoutcomesapanelsurveywasdesignedfordatacollectionatbaseline,
midlineandendline.Datawillbecollectedateachofthesethreestagesinasampleof120publicprimaryschoolsand120IQSsdrawnfromthesixRANALGAsacrossKatsinaandZamfara.HalfofthepublicprimaryschoolsandIQSsformpartofthetreatmentgroup;thisconsistsofschoolsthatreceivetheearlylearningintervention.Theotherhalfservesasthecontrolgroup,inwhichtheearlylearninginterventionwillnotbeimplemented.BaselinedatacollectionwasconductedbyOPMNigeriainOctober–November2015,duringthefirsttermofthe2015–2016schoolyearbeforethestartofimplementationoftheearlylearningintervention.8Thesurveywassuccessfulinachievingtheplannedschoolsamplesize9.
Withinthesampleschools,maleandfemalepupilsandteacherswererandomlysampledtoformpartof
apanelsurvey.10Thetargetpopulationswere:pupilsenrolledinP2grade(oritsequivalentinIQSs)andteachersteachinginP1–3grades(orequivalentinIQSs).Thefinalsampleconsistedof2,653pupils,ofwhich45%aregirls,11and477teachers(seeAnnexC).12Inaddition,theheadteacherineachschoolwassurveyed.Alldatahavebeenanalysedusingsamplingweights,andthereforerepresenttheentiretargetedpopulation.
Sevendatacollectioninstrumentswereadministeredwithineachsampledschool:pupilEnglishandHausaliteracyassessments,ashortpupilquestionnaire,ateacherquestionnaire,ateacherknowledgeandskillsassessment,ateacherclassroomobservationandaheadteacherquestionnaire.Exceptfortheteacherassessments,alldatawerecollectedelectronicallyusingcomputer-assistedpersonalinterviewing(CAPI).
Literacyassessmentswerecarefullydesignedandpilotedtoensureitemdifficultymatchedpupils’
ability.Raschmodelling(usingitemresponsetheory)wasusedtotesttheitems’psychometricpropertiesandtoplacepupilsanditemsonthesamemetric.LiteracyproficiencybandsweredrawnthroughabenchmarkingworkshopundertakenwithstakeholdersofseveraleducationprojectsinnorthernNigeria.13Teacherassessmentswereanalysedusingsubscalesforsubject,curriculumandpedagogicalknowledge.
4.3 Comparisonofbaselinecharacteristicsbetweentreatmentandcontrol
schools
Theevaluationdesignusesrandomisationtoensurethatnosystematicdifferencesexistbetweentreatmentandcontrolschools.Itthusaddressestheissueofselectionbiasinimpactmeasurement.Weassessedwhethertherandomisationhadachieveditsintendedpurposebycheckingwhetherkeyoutcomevariablesandschool,teacherandpupil-levelcharacteristicsdifferedbetweenthetreatmentandcontrol
8Midline data collection is scheduled forMay–June 2017,whichwill be the fifth school term of RANA implementation. This isbeforetheinterventionisscheduledtofinish.ThetimingofthemidlinehasbeenchosentoallowforevaluationfindingstoinformGEP3scale-updecision-makingin2017.Theexacttimingoftheendlinedatacollectionhasyettobedecided.922%oftheoriginallysampledIQSshadtobereplacedatthestartofthebaselineduetotheIQSsnotbeingconsideredeligibleforthesurvey(seeBaselineTechnicalReport).10Thesamepupilsandteacherssurveyedatbaselinewillberesurveyedatmidline.11Anequalnumberofgirlsandboysweretargetedforsamplingwithineachschool–thatis,sixgirlsandsixboys.12Thefinalpupilsampleequals92%ofthetargetedsamplesize(96%inpublicprimaryschoolsand88%inIQSs),whilethefinalteachersamplerepresents80%ofthetargetedsample(83%inpublicprimaryschoolsand74%inIQSs).13AbenchmarkingworkshopwiththeTeacherDevelopmentProgramme(TDP),GEP3,DevelopingEffectivePrivateEducationNigeria(DEEPEN)andEducationSectorSupportProgrammeinNigeria(ESSPIN)stakeholderstookplaceinNovember2015.
©EDOREN 30
groups.Thegreatmajorityofvariablesinvestigateddidnotshowanystatisticallysignificantdifferencebetweenthetwogroups.Thisindicatesthattherandomisationhadworkedtocreatecomparablegroups.Forthesmallminorityofvariablesthatshowedadifference,themagnitudeandthedegreeofstatisticalsignificancearelowanddonotrepresentaconcern.AnoverviewofthemainrandomisationchecksareincludedinAnnexD.
4.4 Baselinefindings
Thissubsectionpresentsthemainfindingsofthebaselineoftheearlylearningevaluation.ThefindingsarestructuredaccordingtotheToC.Throughoutthetextwewillmakereferencetotheevaluationquestionsinsummaryboxes.Sincetheevaluationquestionsarecausalquestionsrelatedtotheearlylearningintervention,theycanonlybeansweredatmidline/endline.Nonetheless,wesynthesiseevidencethatprovideabaselineassessmentoftheoutcomesexpectedandassumptionsunderlyingthecausalchain.
4.4.1 Teachingintheearlygrades
TheearlylearninginterventionaimstoimproveHausa-basedliteracyteachinginearlygradesasanintermediaryoutcomethatmayincreasepupilliteracylevels.Inthissectionwefirstdiscusstheteachingcontextandsomecharacteristicsoftheteachersinearlylearningschools.Next,wepresentthebaselinedataonteacherknowledgeandskills.Wefollowthiswithadiscussionofteacherinstruction,theuseofteachingandlearningmaterials,andtheuseofHausaintheclassroom.Weendthesectionwithfindingsonteachermotivationandattendance,whichareconsideredtobeimportantassumptionsintheToC.
Summaryofevidencerelatedtoevaluationquestions
Q2.Towhatextentdoes
teachers’knowledgeinliteracy
andlanguageacquisitioninearly
gradesimproveasaresultofthe
intervention?
Teachersdemonstrateverylimitedknowledgeinthemajorityoftheareasrequiredtofunctioneffectivelyasateacher.WhileallteachersreportthattheyspeakHausa,lessthan40%areproficientinprimaryGrade1and2-levelHausa.
Theteacherknowledgeandskillstestcarriedoutatbaselineshowsthatthegreatmajorityofteachersdonothavesufficientcurriculum,pedagogicalorsubjectknowledgetofunctioneffectivelyasteachers.Inparticular,whileallteachersreporttospeakHausa,lessthan40%ofteacherswereabletodisplaycompetenceinGrade1and2-levelHausa.Thebaselineevaluationalsofoundthatthevastmajorityofteachersarenotabletodisplaycompetenceinevidencingjudgementsanddiagnosingpupils’workorwriting.Theseskillsareessentialforteacherstotargettheirteachingtopupils’learninglevels,whichinturniswidelyrecognisedasanimportantcomponentofeffectiveteaching.
Theverylowlevelsofknowledgeandskillsamongstteachersatbaselineindicatesthatthereissubstantialscopeforimprovementinthisarea.However,italsopresentschallenges,inthatthereisoftenalackofbasicfoundationalskillsonwhichtheprojectcanbuild.ThefindingsshouldserveasausefulinputtotheRANAimplementationteamwithregardstoensuringthattheinterventiondesignistailoredtoteachers’existingknowledgeandskills.
Thebaselineevaluationalsofoundthatmanyteachersintheearlygradearesubject-specificteachers.ThishighlightsthattrainingwillneedtobeproperlytargetedsothatteacherswhoteachHausaliteracyaretrained.
Q3.Towhatextentdoteacher
skillsinearlygrade,gender-
sensitiveinstructionimproveasa
resultoftheintervention?
Teachers’instructionalskillsareweak.
Teachersperformpoorlyonacompositeindexthatwasconstructedtoassesstheirskillsinearlygradeteaching.Inparticular,theextentofpupil-centredlearningislowandmostteachersmakelimitedeffortstolinkthelessonto
©EDOREN 31
previouslearningortolearningobjectives.Incontrast,timeontaskwashighintheclassesobserved,althoughthisneedstobeinterpretedcarefullyasthepresenceofanobservermayhaveledtoanincreaseintimeontask.
Theearlylearninginterventionreliespartlyonapeermentoringapproachtoimproveteacherskills.Thisrestsontheassumptionthatschoolshavemorethanoneteacherengagedinearlygradeliteracyteaching.However,thebaselinefoundthataconsiderableshareofIQSsonlyhaveonefacilitatorteachingtheintegratedcurriculum.
TheToCalsoassumesthatimprovementsinteachers’skillswillbesupportedbyheadteachers’pedagogicalleadership.Thebaselineevaluationfoundthatlessonobservationandfrequentsupportiveteachermeetingsarepractisedbyonlyhalfofheadteachersinthesample.Thisunderlinesthescopeforimprovementonthisfront.
Thebaselinesurveyattemptedtomeasuregender-sensitiveteachingpractices,butacrossallitemsextremecomplianceeffectswereobserved,significantlycallingintoquestionthevalidityandreliabilityofthemeasure.Therefore,gender-sensitiveteachingcouldnotbemeasuredreliablythroughclassroomobservations.Theteacherassessmentsdoindicatethatteachersareawareofobjectivestotargetgirlsinclass,astheystatethatitisimportanttofocusongirls,buttheirresponsesalsopointtothepersistenceofdeeplyingrainedgenderbias.
Q4.Towhatextentandhowdo
teachersadjustandchangetheir
classroompracticesasaresultof
theintervention?
Hausainstructioniscommon.However,pedagogicalcompetencetoenablepupilstolearnislow.Teacherattendanceandsupportivemonitoring—conditionsthatsupportclassroompractice—requireattentioninIQSs.
Thevastmajorityofteacherstestedatbaselinewereunabletodemonstrateproficiencyinknowingthelearnersintheirsetting,knowledgeofhowtoprovidetheconditionsthatenablepupilstounderstandorproficiencyintheselectionofappropriatelearningandteachingmaterials.RegardingHausainstruction,Hausawasusedinallclassesobserved.Inhalfoftheclassesanotherlanguagewasusedaswell.
Teachermotivationandattendancemayinfluencethetranslationofimprovedknowledgeandskillsintoteachingpractices.Overall,teachersconsidertheirroletobeimportantandenjoyworkingasteachers,whilstbeingconsciousofthefactthattheyarelimitedintheirabilitytocontributetopupils’learning.Atbaseline,60%ofteachersself-reportedthattheyhadbeenabsentatleastonceinthepreviousthreemonths.AbsenceratesarehigherinIQSsthaninpublicprimaryschools.Mostheadteachersinpublicprimaryschoolsreportthattheytakeactiontoimproveteachers’attendance,whilejustoverhalfdoinIQSs.ThefindingsindicatethatteacherattendanceinIQSsmeritsattention.Lessonobservationsindicatedthatstudentswereengagedduringmostoftheclass.
Thelargemajorityofschools(80%)reportthattheyreceivevisitsbygovernmentorotherorganisations,andthevisitsareregular.ThisismuchlessthecaseinIQSs.ItwillbeimportanttopayparticularattentiontomonitoringpatternsinIQSssoastoensurethatvisitsbyGEP3trainersandotherstakeholdersoccurasfrequentlyasforpublicprimaryschools.
Q5.Aremorereadingand
learningmaterialsinHausaused
intheclassroomduetothe
intervention?Dotheycontribute
tomoreeffectiveteachingand
learning?
AccesstoanduseofHausamaterialiscompletelyinadequate.
ThedistributionofapackageofteachingandlearningmaterialsaspartofRANAisthereforehighlyrelevantintermsofprovidingaccesstosuchmaterials.ItwillbeimportantforthematerialstobecarefullyadaptedtothelowHausaliteracylevelsamongstteachers,andthatpeer/supervisorysupportisregularlyavailabletofacilitatetheiruse.
©EDOREN 32
Teachingcontextandteachercharacteristics
Schoolsaremostlyruralandhavepoorinfrastructure.Thegreatmajorityofearlylearningschoolsarelocatedinruralareas,withonly14%locatedinurbanareas.Theaverageschoolhasslightlymorethanfourteachersteachinginearlygrades.Thisnumberissubstantiallyhigherinurbanareas(seventeachersinurbanareas,versusonaverageafewlessthanfourteachersinruralareas).Intermsofinfrastructure,almostallschoolsarereportedlyinneedofmajorrepairs.Onlyaminorityhaveaccesstoasourceofdrinkingwater.Withregardstoschoolcharacteristicsoftenassociatedwithagirl-friendlyenvironment,only26%ofschoolshaveseparatefunctionaltoiletsforgirlsand9%ofteachersarefemale.
TherearenotabledifferencesbetweenIQSsandpublicprimaryschools,withimplicationsforproject
implementation.IQSsaresubstantiallysmallerthanpublicprimaryschoolsintermsofnumberofteachers.Importantly,40%ofIQSsonlyhaveasingleteachertoteachtheintegratedcurriculumsubjects(thecorrespondingfigureforpublicprimaryschoolsis3%).Thisfindingsuggeststhatschool-levelpeermentoringisnotapplicabletoalargeproportionofIQSs.Sincepeer-to-peermentoringisanimportantassumptionunderpinningtheeffectivenessofthetrainingsenvisaged,itisrecommendedthattheprojectidentifiesalternativewaysofensuringthattrainedteacherscanprovidepeersupport,possiblybyengagingwithteachersfromdifferentbutnearbyschools.WhilealargershareofIQSshaveaccesstoasourceofdrinkingwaterandelectricity,theyhavelessroomsonaverageandfewerhaveaccesstobooksandplaygrounds.Theyalsohavelessaccesstoseparatefunctionaltoiletsforgirls,andalmostallteachersaremale.
Table3: SchoolcharacteristicsinpublicprimaryschoolsandIQSs
Publicprimary
schoolsIQSs Allschools
%schoolslocatedinruralarea 86%
Averagenumberofrooms 9 2 6
Averagenumberofteachersinearlygrades* 6 2 4
%femaleteachersinschoolteacherroster 14% 4% 9%
%schoolsinneedofmajorrepairs 92% 97% 95%
%schoolswithsourceofdrinkingwater 41% 47% 44%
%schoolswithaseparatefunctionaltoiletforgirls 46% 6% 26%
*Thisonlyincludesteachersteachingnon-religioussubjects.Source:GEP3Baselinesurvey2015
Headteachersmakeanefforttoaddressteacherattendancebutthereislessattentiontoteacher
instruction.TherearedifferencesbetweenpublicprimaryschoolsandIQSsinthisregard.Acrossallschools,71%ofheadteachersreporthavingtakenactiontoimproveteacherattendanceduringthepreviousterm.ThisdiffersstarklybetweenpublicprimaryschoolsandIQSs,asisshowninFigure1Error!Referencesourcenotfound..ThisdifferencemaybeduetothedifferenttypeofleadershipstructureinIQSs,whereheadteachersarenotalwaystheschoolprivateproprietorandmaynotbe,ormaynotperceivethemselvestobe,responsibleforteacherattendance.Itwillbeimportantfortheearlylearninginterventiontoensurethattherelevantchainofteachingresponsibilitywithineachschoolisclearlyunderstoodanditsinfluencemaximisedwhenrunningthetraining.Inbothtypesofschool,headteachersarelessactiveinmonitoringactualteaching(seeFigure1).Lessthan50%ofheadteachersobservedalessonduringthepreviousschooltermandaround50%neverhavemeetingswithteachers,ormeetthemlessthanonceamonth.Thisisacauseforconcern.Headteacherpedagogicalleadershipshouldtherefore
©EDOREN 33
improveaspartoftheearlylearninginterventiontofacilitateimprovementsinteaching.Teacheradministration,intermsofkeepingupdatedteacherattendancerecords,appearstobewellorganisedinpublicprimaryschools,butisalmostnon-existentinIQSs.Withregardstoexternalmonitoringbygovernmentorotherorganisations,almostallpublicprimaryschoolsreportedreceivingavisitoverthepreviousschoolyear.ThecorrespondingfigureforIQSswasonly58%.ItwillthereforebeimportanttopayparticularattentiontomonitoringpatternsinIQSs,soastoensurethatvisitsbymentorsandgovernmentstaffoccurasfrequentlyasforpublicprimaryschools.
Figure1:SchoolmanagementactionsinPSandIQS
TeachersspeakHausa,theyoftenteachonesubjectandtheyhavevaryingdegreesofqualificationsand
experience.AllteacherssurveyedreportbeingabletospeakHausa.ThispartlyvalidatestheassumptionthatteacherswillbeabletouseHausaduringinstruction.However,asdiscussedbelow,thisdoesnotmeanthattheyareliterateinHausa.MostteachersalsoreportthattheyspeakEnglish,withdifferencesbetweenpublicprimaryschoolteachersandIQSfacilitators.OtherteachercharacteristicsaresummarisedinTable4.Itisworthnotingthataroundhalfoftheteacherssaytheyteachonlyonesubject,ofwhich25%teachonlymathematicsandanother18%teachonlynon-languagesubjects.GiventhatRANAwillemphasisetheinstructionofearlygradereading,therelevanceofthetrainingforthisproportionofteacherswhoteachmathematicsorother,non-languagesubjectsisworthyofattentionwhenselectingteachersfortraining.PublicprimaryschoolteachersandIQSfacilitatorsalsodifferintermsofprofessionalandacademicqualificationsandyearsofteachingexperience.Theteachertrainingwillneedtobeadaptedtothiskindofvariationinteacherbackground.
0 10 20 30 40 50 60 70 80 90 100
HTobservedalesson
HTtookactiontoimproveteacherattendance
Updatedteacherattendancerecordsexist
Percentageofschools
PS IQS
Source:GEP3Baselinesurvey2015
©EDOREN 34
Table4: Summaryofteachercharacteristics
Forsurveyedteacherstheaverage… Ofteachers…
• …ageis36years.
• …teachingexperienceis10years(11yearsinpublicprimaryschool,eightyearsinIQS);89%ofteachershaveatleasttwoyearsofteachingexperienceand71%havehadthatexperienceinthecurrentschool(79%inpublicprimaryschools–52%inIQSs).
• …teachersarelikelytoteachone(non-religious)
subject;thisbeingmostlymaths,HausaorEnglish.
• …all(100%)speakHausa;85%speakEnglish(91%inpublicprimaryschools,73%inIQSs).
• …only15%arefemale(21%inpublicprimaryschools,3%inIQSs).*
• …themajority(54%)holdanNationalCertificateinEducation(NCE)qualification(70%inpublicprimaryschools,22%inIQSs)and72%havepassedtheSeniorSchoolCertificateExamination(SCCE)(74%inpublicprimaryschools,67%inIQSs).
• …40%attendedtrainingduringthelasttwoyears(41%inpublicprimaryschools,46%inIQSs).
*ThepercentageoffemaleteachersslightlydiffersfromthedatapresentedinTable3becausethedatasourcesaredifferent.DatainTable3arebasedontheteacherschoolroster,whiledatainthistablearebasedontheteachersurvey.
Source:GEP3Baselinesurvey2015
Teachers’knowledgeandskills
TheGEP3earlylearninginterventionseekstoimproveteachers’knowledgeinliteracyandlanguageacquisitioninearlygrades.Teachersdrawonthreetypesofknowledgewithinclassroompractice:subjectknowledge,pedagogicalknowledgeandcurriculumknowledge.14
Theteacherssurveyeddemonstratedverylimitedknowledgeandskillsinthemajorityoftheareas
requiredtofunctioneffectivelyasateacher.Rudimentarylevelsofskillswereobservedamongasmallshareoftheassessedteachersinidentifyinglowperformers,evidencingjudgementsanddiagnosingpupils’performance,teacherwritingskillsandinterpretingwordsandphrases(seeFigure2).AgreatershareofteacherswasabletodisplayknowledgeandskillsinGrade1and2-levelHausaandincomprehension.Nevertheless,itisstrikingthat60%ofteacherswereunabletodisplaycompetenceinGrade1and2-levelHausa.15ThisfindingissignificantforaninterventionthatfocusesonteachinginHausabecauseteachers’lowHausaknowledgeislikelytonegativelyaffectteachers’abilitytopasstheirknowledgeontopupils.Also,teachers’inabilitytoidentifylowperformers,evidenceteacherjudgementsconcerningpupilperformance,anddiagnosethenextstepsofteachingpresentchallengestoimprovingpupillearningasastudentlearnsbestwhenteachingistargetedtowhats/heisreadytolearn.Regardingcurriculumknowledge,alargeminorityofteachers(about43%)wereunabletodemonstrateproficiencyinknowingwhatshouldbetaughttoagroupofstudents.
14Subjectknowledgereferstoknowingtheessentialquestionsofthesubject,thenetworksofconcepts,theoreticalframeworkandmethodsof inquiry.Pedagogicalknowledgereferstoknowledgeofthe learners inthesetting,knowledgeofhowtoprovidetheconditionsthatenablepupilstounderstand,andtheselectionoflearningandassessmentmaterials.Curriculumknowledgereferstoknowingwhatshouldbetaughttoagroupofstudents,knowledgeofthenationalsyllabus,understandingtheschool-andgrade-levelplanningdocumentsandknowledgeofthecontentofexaminations.15Itemstestedincludedbasicgrammarandtheinitiallettersofeverydayobjectsandanimals.
©EDOREN 35
Figure2: Percentageofteachersachievingwithinthelower,middleandupperbandsof
competenceacrosstheteacherknowledgeandskillssubscales
Therearefewdifferencesintheknowledgelevelsdemonstratedbydifferentgroupsofteachers.Althoughteacherswithaprofessionaleducationalqualificationdemonstrateslightlyhigherlevelsofknowledgeandskills,levelstendtobelowacrossallgroupsofteachers.Forexample,weobservenodifferencesbetweenteachersinpublicprimaryschoolsandIQSs,orbetweenteachersthatdidordidnotattendpreviousGEP/UNICEFtraining.
Thefindingssuggestthatteachersdonothavetheknowledgeorskillstodriveimprovementsinpupils’
learning.Aqualitativereviewofincorrectteacherresponsestoitemsintheteacherknowledgeandskillsassessmentindicatesthatteachersareunabletoascertainwhichactivitiesarebestsuitedtoimprovepupilperformanceinaspecificsubject.Teachersgenerallyperceivedpupilefforttobethecauseoflowperformanceandthereforeperceivedincreasingpupil(andtosomeextentteacher)effortasthebestresponsetolowperformance.Thefocusonpupileffortasacauseoflowperformancewithinthiscontextislikelytobeareflectionofthelowlevelsofcompetenceamongstteachers–teacherslackboththeknowledgeandskillstoimprovepupils’learning,whichleavesfewavailableoptionsforimprovingpupillearning,beyondpupils‘tryingharder’.Whileteacherssaidthattheyshouldfocusongirls’learningintheclass,theywerenotabletoexplainwhysuchafocusisnecessary.Inaddition,teachersdidnothaveaclearunderstandingofgirls’learningneeds,andinsomecaseshelddeeplyingrainedbiasesregardingthelimitedcapacityofgirlsortheirroleinsociety.
Thesefindingshaveafewkeyimplicationsfortheearlylearningintervention.First,theyhighlightsomeofthekeyissuesthattheinterventionneedstoaddressthroughanappropriatecombinationofteachertrainingandcarefullytailoredteachingandlearningmaterials.Second,theyprovideanindicationofthescaleofthechallengeconfrontingtheproject,whichcouldfeedintodesigndecisionsrelatedtothefrequencyoftrainingandthelevelatwhichitispitched.Third,theysuggestthatthereistremendousscopetoimproveknowledgeandskillsamongstteachers,although,equally,teachers’verylowlevelsofskillsandknowledgeatbaselinemayalsopresentchallengesforaninterventionthatworksprimarilythroughin-servicetrainingandsupport.
Abilitytoidentifylowperformers
Abilitytoevidence
judgementsanddiagnose
Teacherwritingskills
TeacherGrade2Hausa
Knowledge
Teachercomprehension
skills
Interpretingwordsandphrases
UpperBand 2.3 0 0 39.9 33.6 2.0
MiddleBand 6.2 .2 2.1 23.1 15.3 2.9
LowerBand 91.5 99.8 97.9 37.0 51.0 95.2
0
10
20
30
40
50
60
70
80
90
100
Percen
t
Note:Twocut-pointsweredefinedforeachsubscaletocreatethreeproficiencylevelsperscale.Theseproficiencylevelsare:LowBand–noevidenceofskill;MiddleBand–evidenceofrudimentaryskill;andUpperBand–evidenceofcompetenceSource:GEP3Baselinesurvey2015
©EDOREN 36
Teachers’instructionalskillsandpractices
Theclassroomobservationscarriedoutatbaselineindicatethatteachers’instructionskillsinearlygrade
teachingarelowacrossallgroupsofteachers.Acompositeindexwasconstructedtomeasurethisoutcomevariable.Thisconsistedofmeasuresoftheextentofpupil-centredlearningactivitiesobservedintheclassroom,observationsoftheteacherlinkingthelessontopreviouslearningandlearningobjectives,andtimeontaskinclass.Overall,teachersscoredpoorlyontheindex.Thiswasdrivenbytheirperformanceonthefirsttwocomponentsoftheindex.Incontrast,timeontaskwashigh:pupilswereobservedtobeengagedinanactivityrelatedtolearningfor96%oflessontime.Roughly85%ofclassroomswereobservedtobe‘ontask’forover90%ofthelesson.However,thesefindingsshouldbeinterpretedcarefullyasitislikelythatthepresenceofobserversintheclassroomincreasedthepercentageofontasktimeinlessons.
Therewaslimitedvariationinscoresontheindexacrossdifferentgroupsofteachers.PublicprimaryschoolteachersscoredslightlyhigherthanthoseinIQSs,whiletherewerenocleardifferencesbystate.Therewerealsonoobservablepatternsbetweenteacherpracticesandteachermotivationorperceivedteacherefficacy.Thisindicatesthatteachers’perceivedself-efficacyiseithernotrepresentativeoftheiractualcompetenciesorthelevelsofcompetencyinteacherpracticesaresolowthatdifferencescannotbedetected.
Thebaselinesurveyattemptedtomeasuregender-sensitiveteachingpractices,butacrossallitemsextremecomplianceeffectswereobserved,significantlycallingintoquestionthevalidityandreliabilityofthemeasure.Therefore,gender-sensitiveteachingcouldnotbemeasuredthroughclassroomobservations.Theteacherassessmentsindicatethatteachersareawareofobjectivestotargetgirlsinclass,butthatdeeplyingrainedgenderbiasesremain.Teacherassessmentresponsesstatingthatteachersshouldfocusongirlsintheclasswerenotsupportedbyjustificationsforsuchafocus,didnotreflecttheneedsofgirlsintheclassroomandinsomeinstancesmadeanargumentregardingthelimitedcapacityorsocietalrolesofgirls.
Useofteachingandlearningmaterials
AspartoftheearlylearninginterventionHausateachingandlearningmaterialswillbedistributed.ThisisexpectedtocontributetoanimprovementinHausa-basedteaching,insofarasthematerialsareusedandalignedwiththecompetencylevelsoftheteachers.Wethereforeexaminedtheuseofteachingandlearningmaterialsatbaseline.
Teachingandlearningmaterialsarerarelyused.Duringtheclassroomobservationsatbaseline,veryfewteachersusedatextbookorothermaterials:17%ofobservedteachersusedatextbook,while19%usedothermaterials.
TheuseofHausamaterialsisespeciallyrare.MaterialsinHausawereobservedbeingusedin2.4%ofobservedlessons.MaterialsinHausawereavailableforthesubjectobserved,butwerenotusedinafurther1%ofobservedlessons.Interestingly,almost18%ofteachersindicatedthattheyhavetheHausamaterialstheyneedtodotheirjobs.Thiswouldsuggestthatroughly14%ofteacherseitherdonotseeaneedforHausamaterials,orhaveaccesstomaterialsthattheydonotroutinelyuseduringlessons.
Hausa-basedteachinginearlygrades
TheearlylearninginterventionseekstoimproveHausa-basedliteracyteachinginearlygrades.AcompositeindexwasdevelopedinordertomeasurechangesinHausa-basedteachingintheearlygrades.16Typically,
16ThemeasuresincludedinthecompositeindexweredevisedusingthepercentageoftimetheteacherspokeHausainclassandteachers’Hausaskills.
©EDOREN 37
teachersinIQSsscoredhigherinHausa-basedteachingintheearlygradesthanteacherswithinpublicprimaryschools,asdidteacherswhodonotspeakEnglishorArabic.
Hausawasusedinallclassesobserved.InallclassroomstheuseofHausawasobservedonatleastoneoccasion.InapproximatelyhalfoftheclassesasecondlanguagewasusedinadditiontoHausa.ThefrequencyofHausausewasdocumentedduringclassroomobservationsandameasureofthepercentageoftimeforwhichtheteacherspokeHausawascalculated.Injustunder30%ofobservedlessons,Hausawasspokenbytheteacherfor50%–60%ofthelesson(asdepictedbythegreenbarinFigure3).Inalmosthalfoftheobservedlessons,Hausawasspokenbytheteacherfor30%–50%ofthelesson(depictedbytheblueandpurplebarsinFigure3).
Figure3: FrequencyofHausauseintheclass
Teachermotivationandattendance
Teachers’motivationandattendanceareassumptionscontainedintheToCthatareconsideredtoinfluencetheintermediaryoutcomesregardingteaching.LowteachermotivationandattendancemayleadtoHausa-basedliteracyteachingnotimproving,despitetheearlylearninginterventiontakingplace.Wemeasuremotivationaccordingtofivedimensionsthataretranslatedintomeasurementsubscales(seeError!Referencesourcenotfound.).17
17TheteachermotivationscalesweredrawnfromapreviousstudyundertakenbyEDORENfortheTDP.Inthisstudy,‘motivation’wasunderstoodas:amixtureofperceivedefficacyandeffort/importanceattachedtoteachingwork;butisalsomeasuredlessdirectlythroughinterest/enjoymentandpressure/tensionitems.Basedonthisdefinition,amotivatedteacherisonewho:seesthemselvesaseffectiveandasmakinganeffort;seestheirworkasimportant;isinterestedinandenjoystheirwork;andmanagesworkpressureandtension.DetailsofthetheoryunderpinningtheteachermotivationscalescanbefoundinSection3.2.9oftheBaselineTechnicalReport.
.0
5.0
10.0
15.0
20.0
25.0
30.0
ShareoftimethatHausawasusedduringlesson
Shareoflesson
sob
served
Lessthan10
10-19.99
20-29.99
30-39.99
40-49.99
50-59.99
Source:GEP3Baselinesurvey2015
©EDOREN 38
Box2.Measurementofteachermotivation
Teachermotivationisinvestigatedthroughtheuseofarangeofmotivationscaledscores,whichformthebasisfortheconstructionofacompositemotivationindex.Thesubscalescoverthefollowingdimensions:
• perceivedteacherefficacy;
• interestin,andenjoymentof,teaching;
• effortputinto,andperceivedimportanceof,teaching;
• pressureandwork-relatedtension;and
• teacher-to-teacherinteraction
Overall,teachersconsidertheirroletobeimportantandenjoyworkingasteachers,whilstbeing
consciousofthefactthattheyarelimitedintheirabilitytocontributetopupils’learning.Ananalysisofthedifferentmotivationsubscalesshowsthat,onaverage,thesurveyedteachersscorehighestoneffortandlowestonperceivedteacherefficacy(seeFigure4).Interestinandenjoymentofteachingisrelativelyhighamongteacherswhencomparedtopressureandwork-relatedtension.Itisalsonotablethattheteacher-to-teacherinteractionscoreseemstoindicatearelativelyhighlevelofcollaborationamongstteachers,whichcouldhelpthedevelopmentofspillovereffectswithinschoolsofanyteacher-specificintervention,includingteachertraining.
Figure4: Comparisonofmotivationsubscales
Overallmotivationappearsrelativelyhomogeneousacrossteachers.Acompositemotivationindexthatcomprisesallmotivationsubscalesshowsarelativelyhomogeneoussituationacrossteachers.Motivationamongurbanandruralteachersisverysimilar,andnodifferenceatallisdetectablebetweenteachersacrossstatesandtypesofschools(publicprimaryschoolsandIQSs).Whilereceivingasalaryseemstobesomewhatpositivelyassociatedwithmotivation,thedifferencebetweenteacherswhoreceiveanddonotreceiveanyremunerationissmall.
AbsenteeismisfoundtobemoreprevalentinIQSsthaninpublicprimaryschools.Thebaselinedataonabsenteeismshowsthat60%ofteachersself-reportedhavingbeenabsentatleastonceinthelastthreemonthsacrossallschooltypes.Theestimatedaveragenumberofdaysthattheyreportedhavingbeenabsentoverthisperiodwasjustunderfivedays.AbsenteeismismoreprevalentinIQSsthaninpublicprimaryschools.IQSfacilitatorsreportedhavingbeenabsentforroughlyeightdaysonaverage,compared
1.99
3.57
3.73
2.19
3.47
0 0.5 1 1.5 2 2.5 3 3.5 4
Perceivedteacherefficacy
Interestandenjoymentofteaching
Effortandimportance
Pressureandworkrelatedtension
Teacher-to-teacherinteraction
Source:GEP3Baselinesurvey2015
©EDOREN 39
tojustoverthreedaysamongpublicprimaryschoolteachers.However,attendancedataonIQSsneedstobeinterpretedwithcautionastheIQSteachingscheduleisnotasstructuredasinpublicprimaryschools.Thiscouldpartlyexplainthedifferenceinabsenteeismrates.Teacherswhodonotreceivesomeformofremunerationaremorelikelytobeabsent,andtobeabsentforlongerperiods.Itwillbeimportanttoincentivisetrainedteachersandleadteacherstoregularlyattendtheirclassessoastoensurethattheirimprovedknowledgeandteachingpracticesarepassedontopupilsinschool.
4.4.2 Pupils’literacyinearlygrades
Inthissectionwediscusspupils’literacyintheearlygradesample.First,weprovidebackgroundinformationonkeypupilcharacteristicsthatareimportantfortheanalysisandtheearlylearningintervention.WethendescribethelearninglevelsofpupilsinHausaandEnglishliteracy,whichisfollowedbyaninvestigationofthefactorsassociatedwiththeselevels.Section4.4.3theninvestigatestherobustnessofsomeofthedescriptiveassociationsthroughregressionmodelling.
Summaryofevidencerelatedtoevaluationquestions
Q1.Towhatextentdoesthe
earlylearningintervention
improveHausaliteracyand
Englishlanguagelearning
outcomesamonggirlsandboys
intheearlygradesinpublic
primaryschoolsandIQSs?
Towhatextentdoestheearly
learninginterventionreducethe
gapbetweenthelearning
outcomesofthelowest
performingpupilsandthe
expectedlearningoutcomes,as
expressedinthecurriculum?
Pupilliteracyperformanceisverylowandiswellbelowtheexpectedlearningoutcomesasexpressedinthecurriculum.Whileteachercompetenciesarenotfoundtobeassociatedwithlearningoutcomes,thismaybeduetoverylowteachercompetencylevelsandineffectiveteaching.Pupil-levelfactorsaresignificantlycorrelatedwithlearningoutcomes.
Baselinefindingsonpupils’characteristicsshowthatHausaisthemothertongueofeverypupil,whichisanassumptionoftheprogramme’sToC.However,thefindingsoftheHausaliteracyassessmentindicatethatveryfewpupilshaveacquiredtheknowledgeandskillsappropriatefortheirgrade.Furthermore,thepeakofthedistributionofperformanceinHausaliteracyfallswellbelowthecut-offpointbetweenpre-literacyandemergingliteracy.WhenitcomestoEnglish,96%ofpupilscouldnotdemonstrateliteracyskillsinEnglishbeyondpre-literacyskills.AswithHausa,thepeakofthedistributionofperformancefallswellbelowthecut-offpointbetweenpre-literacyandemergingliteracy.Thissuggeststhatsubstantialeffortwouldberequiredtoachievesignificantgainsinthepercentageofpupilsmovingfrompre-literacytoemergingliteracyinbothHausaandEnglish.
Theregressionanalysisshowsthatpupils’individualcharacteristicsandsocioeconomicbackgroundareassociatedwithpupils’achievement,whilemostoftheschool-levelvariables,suchaspresenceofasourceofdrinkingwaterorseparatetoiletsforgirls,donotshowanysignificantcorrelationwithHausaliteracy.
Q6.Towhatextentdoesthe
earlylearningintervention
improvepupilretention,
especiallyretentionofgirls?
Noevidenceaboutretentionisavailableatbaseline.
Pupilcharacteristics
Hausa is the language of the immediate environment. All pupils report speaking Hausa at home (withalmostnostudentsreportingspeakingEnglishoranyotherlocallanguageathome).ThisvalidatestheToCassumptionthatHausaisthepupils’mothertongue.
©EDOREN 40
PupilcharacteristicsvarybetweenPSandIQS.IQSpupilsareolderonaveragethanPSpupils.Asubstantialproportionofpupilsareaged11yearsandover in IQS (41%) compared tohardlyany inPS (3%).18 IQSpupils in the sample are also frommoredisadvantagedbackgrounds in termsof householdwealth thanpupilsinPS.19
Pupils’learningoutcomes
In this subsectionwepresent the findings aboutpupils’ learningoutcomes.Error!Reference sourcenotfound.summariseshowlearningoutcomeshavebeenmeasured.
Box3.Learningoutcomesmeasurement
Hausa and English literacy outcomes aremeasuredbasedon two assessment tools. TheHausa andEnglish literacy assessments are each designed to test the same literacy knowledge and skills. TheEnglish literacy assessment contains 13 items,with each item beingmade up of several sub-items.RegardingtheHausaliteracyassessment,itemsarenotmerelytranslated,butratherparallelitemsaredevelopedtotestsimilarconceptsasappliedtotheHausalanguage.
The assessments test a rangeof literacy knowledge and skills across thepre-literacy, emerging andbasicliteracyranges.Knowledgeandskillstestedincludeletterrecognition,phonologicalknowledge,printconcepts,oralliteracy,verbalcomprehension,initialsoundsandletters,readinghighfrequencywords, verbal and written grammar, writing high frequency words, reading fluency, copying andspellinghighfrequencywords.
English literacy and Hausa literacy assessments were constructed following five steps: clarifyingconstructs,testtargeting,administration,psychometricanalysis,drawingbenchmarksandsecondarydataanalysis.
Theoverwhelmingmajorityofpupilshavenotyetacquiredtheknowledgeandskillsexpectedbythe
NigeriancurriculuminHausaliteracy.ItisexpectedthatpupilswhoarebeginningP2willhavemasteredtheP1curriculumandwillbereadytoreceivetheP2curriculum.However,thelearningassessmentresultsindicatethatonly5.3%ofpupilsareperformingwithintheexpectedrange.Afurther2.6%ofpupilsareabletodemonstrateemergingliteracyskills,while92%ofthepupilscouldnotdemonstrateliteracyskillsinHausabeyondpre-literacyskills.AscanbeseeninFigure5,mostpupilsscorewellbelowthecut-offpointbetweenpre-literacyandemergingliteracy,indicatingthatachievinglargegainsinthepercentageofpupilsmovingfrompre-literacytoemergingliteracywillrequiresubstantialeffort.
ThemajorityofpupilshavenotyetacquiredeitheremergingorbasicEnglishliteracyskills.Lessthan1%ofpupilsdemonstratedatleastsomeskillsthatfallwithinthebasicEnglishproficiencyrangeand3.3%ofpupilsdemonstratedatleastsomeoftheskillsthatfallwithintheemergingliteracyrange.However,96%ofpupilscouldnotdemonstrateliteracyskillsinEnglishbeyondpre-literacyskills.TheresultsoftheEnglishliteracyassessmentsuggestthatpupilproficiencyinHausaaftermorethanayearofschoolingisnotsignificantlyhigherthanpupilproficiencyinEnglish.SimilartothedistributionofHausaliteracyscores,thecuspofthedistributionofEnglishliteracyfallswellbelowthecut-offpointbetweenpre-literacyandemergingliteracy.
Knowledgeofphonicsisverylow.ThepsychometricanalysisfindsthatitemsthatrequireknowledgeofphonicsrankasthemostdifficultitemsinboththeHausaandEnglishassessments.Correctlysoundingoutlettersandidentifyingsimilarsoundswasmoredifficultforpupilsthanwritingorreadingfullpassages.Thisindicatesthatcurrentlypupilswithinthecontextwhohaveachievedbasicliteracyaredoingsowithoutsignificantexposuretophonicsknowledge.Evidencesuggeststhatsystematicphonicsteachingis
18Theactualpercentagenumbersneedbehandledwithcautionasalargenumberofpupils(35%)inthesampledidnotreportage.ThismissingdatawereevenlyspreadacrosspublicprimaryschoolsandIQSs.19PovertydifferentiationisbasedonaHouseholdWealthIndex(HWI)usingaseriesofhouseholdassetsasanindicationofwealth.
©EDOREN 41
associatedwithbetterprogressinreadingaccuracythoughnotalwaysinreadingcomprehension.20Whilsttheuseofphonicsinstructionhasadvantages,ifliteracyisunderstoodasbeingabletounderstandandinteractwithwrittentext,readingfluencyisabridgetoliteracy,ratherthanameasureofliteracyitself.Furthermore,theevidencebaseonthelinksbetweenphonologicalawarenessandreadingaccuracyandcomprehensionmainlycomesfromstudiesincontextswithhighlyliterateandqualifiedteachers.Asdiscussedabove,theseconditionsdonotholdinthecontextinwhichGEP3isbeingimplemented.Theearlylearninginterventionwillneedtotakethesecaveatsintoaccountwhenpromotingaphonicsapproachtoteaching.
Figure5: DistributionofHausaliteracyproficiency
Learningoutcomesdifferbyageandgender.Malepupilsachievedhigheraveragescoresthanfemalepupils,althoughthiswasmainlydrivenbydifferencesinIQSs.Thiscanbepartlyexplainedbytheassociationbetweenage,learningoutcomesandthegendergap.Thebaselinedatahighlightthatolderpupilsperformbetterthanyoungerones(seeFigure6).Italsosuggeststhatlearningoutcomesaremorecloselyassociatedwithcognitivedevelopmentthatwithyearsofschooling–asaverageyearsofschoolingdonotdiffersubstantiallybetweenolderandyoungerchildreninthesample.WhendisaggregatingliteracyachievementbyageandgenderatrendemergesacrossbothHausaandEnglish(seeFigure6).Genderdifferencesinperformancearesmallintheyoungeryears,butincreaseoncegirlsreachpuberty(around12
20Torgeson,Brooks,andHall,2006.
©EDOREN 42
yearsofage).Inlinewiththis,thelargergendergapinIQSscouldpartlyreflectthefactthatpupilsatIQSsalsotendtobeolder.
Figure6: MeanHausascalescorebyageandgender(95%confidenceinterval)
4.4.3 Analysisofrelationships
Inadditiontothedescriptiveanalysispresentedabove,theevaluationteamconductedregressionanalysistomakemorestatisticallyrobustclaimsabouttherelationshipbetweenlearningoutcomes(HausaliteracyandEnglishliteracy)andawidersetofschool-,teacher-andschool-levelexplanatoryfactors.InError!Referencesourcenotfound.wepresentthecategoriesofexplanatoryvariablesthatareincludedinthepupilregressionmodel.TableswithregressioncoefficientsareincludedinAnnexE.
Box4.Explanatoryfactorsincludedinthepupilregressionmodel
• Pupils’personalcharacteristics,includinggenderandagegroup• Pupils’socioeconomicstatus,asdefinedbytheHWItertilecategorisation• Informationaboutpupils’schools,includingwhethertheyattendpublicprimaryschoolsorIQSs,
whethertheyattendotherschoolsinparallelandwhethertheycanwrite• Locationinformation,includingwhethertheschoolisinKatsinaorZamfaraandwhetheritislocated
inanurbanorruralarea• Schoolinfrastructure21,includingwhethertheschoolhasseparatetoiletsforgirlsandwhetherithas
accesstoasourceofdrinkingwater• Teachers’averagemotivationlevel,asdefinedinouroverallmotivationindexillustratedabove,in
theschoolattendedbythepupil• Teachers’averagesubjectandpedagogicalknowledge,asdefinedintherelevantindexesillustrated
inthesectionabove,intheschoolattendedbythepupil• Teacherknowledge1:Teachers’subjectknowledge
• Teacherknowledge2:Teachers’syllabusandcurriculumknowledge• Teacherpedagogy1:Teachers’pedagogicalknowledge
21Othervariablesonschoolcontextinformation,includingforinstanceelectricityorpupil/teacher,teacher/classroom,pupil/classroomandpupilandteachergenderratioswerenotincludedastheywouldhavereducedtheregressionsamplesize.Dataonthefactorsaboveandotherschoolaspectswereinfactcollectedonlyfromalimitednumberofschools.
300
400
500
600
700
800
900
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Scalescore
Pupils'ageBoy Girl
Source:GEP3Baselinesurvey2015
Cutpointbetweenpre-literacyandemergingliteracy
©EDOREN 43
• Teacherpedagogy2:Mothertongueteachinginearlygrades.
Teacher-levelfactorsdonotappeartobesignificantlycorrelatedwithliteracylearningoutcomes.Inparticular,noneoftheteachermotivation,knowledgeorpedagogyindexesarefoundtobesignificantlyassociatedwitheitherHausaorEnglishlearningoutcomes.Thismayreflectthefactthatcompetencylevelsareverylowacrossallteachers.Itislikelythatastrongercorrelationwouldbedetectediftherewasgreatervariationincompetencylevelsacrossteachersandthattheirabilitytoinfluencepupils’learningoutcomeswouldbehigher.Thisalsoimpliesthatiftheearlylearninginterventiondoesleadtosubstantialimprovementsinteachers’knowledgeandskills,weshouldseeamoresignificantlinkbetweenteachercharacteristicsandlearningoutcomesatendline.
Thelocation(urban/rural)andtypeofschool(publicprimaryschool/IQS)areassociatedwithlearning
outcomes.Mostoftheschool-levelvariables,suchaspresenceofasourceofdrinkingwaterorseparatetoiletsforgirls,donotshowanysignificantcorrelationwithHausaliteracy.Similarly,whilstbeinginKatsinaorZamfarastatedoesnothaveanyassociationwithliteracyoutcomes,rural/urbanlocationdoes,withpupilsattendingschoolsinruralareasachievinglowerHausaandEnglishscoresthanthoseinurbanareas.Finally,pupilsinpublicprimaryschoolsappeartoachievelowerliteracyscoresinbothlanguagestestedthanpupilsinIQSs.TheregressionanalysisindicatesthatthestatisticallysignificantbetterperformanceofIQSpupilsremainsevenaftercontrollingforthepotentiallyconfoundingeffectofpupils’age.
Pupil-levelexplanatoryfactorsaresignificantlycorrelatedwithlearningoutcomes.Age,unsurprisingly,issignificantlycorrelatedwithlearning,witholderpupilsdoingbetterthanyoungerones.Householdsocioeconomicstatusisalsoassociatedwithliteracyoutcomes,withrelativelywealthierpupilsfoundtoperformbetterthantheirpoorercounterparts,althoughaneconomicthresholdexistsbelowwhichimprovementsinsocioeconomicstatusdonotappeartoaffectlearning.Atthesametime,giventhegenerallylowlevelofliteracyamongstthesurveyedpupils,itseemsreasonabletopresumethatwealthisnotsufficienttoachievegoodlearningoutcomes.Genderiscorrelatedwithlearning:malepupilsperformbetterthantheirfemalecounterparts.However,thecorrelationonlyemergeswhencontrollingforallpossibleschool-levelcharacteristicsandthestatisticalsignificanceofthecorrelationislow.RANA’scommunityengagementactivitiescouldhelptocountersomeoftheeffectsofsocioeconomicbackgroundonlearningoutcomesbypromotingasupportivehomelearningenvironmentforallchildren.
©EDOREN 44
5 BaselineoftheIQSSevaluation
5.1 GEP3’sIQSS
GEP3’sstrategyforintegratedQur’aniceducationfocusesontheimprovementofeducationinIQSs,with
theaimofprovidinganacceptablealternativeformofqualitybasiceducationforgirls.Thefinaloutcomesexpectedare:improvedlearningoutcomesinbasicliteracyandnumeracy(especiallyforgirls),improvedretentionofgirls,and,toalesserextent,increasedenrolmentofgirls.
GEP3targetsregisteredIQTEcentresthatimplementanintegratedcurriculum(becomingIQSs)andthatlargelyoperateascommunity-basedinitiatives,butthatarewillingtobuildlinkswithgovernmentforthepurposesofmonitoringandtechnicalsupport.Atleast40%ofthestudentsenrolledshouldbegirlsfortheIQStobesupportedbyGEP3.Duringthe2015–2017period200IQSsperstateacrossthesixGEP3LGAsinNiger,BauchiandSokotowillreceiveafullschool-levelsupportpackage.22Box5summarisestheIQSSpackagethatwillbeimplementedatschoolcommunitylevel(amoredetaileddescriptionispresentedinChapter4oftheBaselineTechnicalReport).
Box5.IQSSactivitiesatschoolandcommunitylevel
• TrainingandmentoringofIQSfacilitators,with,atitscore,a1.5-yearcycleofmonthlycluster-leveltraining/mentoringmeetingsinitiatedbyfive-dayinductionworkshop
• Provisionofapackageofclassroomteachingandlearningmaterials
• Trainingofheadteachers,consistingofthree-daytrainingsessionspertermoveratwo-yearperiod
• CapacitybuildingforCBMCs,consistingofaninitialmulti-daycluster-leveltrainingwithfollow-upmentoringvisits,atleastonceaterm,overaperiodofnineto12months
• Provisionofmini-grants,atmosttwiceoveratwo-yearperiod,andconditionalonthedevelopmentofaWholeCentreDevelopmentPlan(WCDP)andattendanceatCBMCtraining
GEP’sIQSSbeganinAugust2015andinvolvesa2–2.5-yearperiodofmentoringandfollow-uptraining.AfirstcohortofonefacilitatorperIQSattendedtheinductionworkshopinAugust/September2015.AsecondfacilitatorfromeachIQSwillstartthetrainingandmentoringcycleinJuly–September2016.TheheadteachertrainingwasoriginallyplannedtostartbetweenAprilandJune2016butthishasyettobeconfirmed.ThematerialsareplannedtobedeliveredtotheIQSsbyApril–May2016.TheinitialCBMCcluster-leveltrainingisscheduledtotakeplaceinJuly–September2016.
AnnexFpresentstheToCoftheIQSSintervention.TheToCisdiscussedindetailasitformsanintegralpartofthecontributionanalysisapproachthatisusedfortheIQSSevaluation.ThemainlogicofGEP3’sIQSSisthatgirls’learningcanimproveinanIQSwhentheIQScanprovidequalitybasiceducation,whichrequiresthatfacilitatorsteachmoreeffectivelyandthattheschoolenvironmentimprovesandbecomesmoregirl-friendly.EffectiveteachingisexpectedtoimprovewithGEP3’ssupportforfacilitatortrainingandmentoring,headteachertrainingandthedistributionofteachingandlearningmaterials.CBMCtrainingandtheprovisionofmini-grantsareexpectedtocontributetoanimprovementintheschoolenvironment.Atthegovernmentlevel,GEP3’sstateadvocacyandLG(E)Acapacitysupportareexpectedtocontributetotheimprovementofmonitoringandsupportsupervisionbygovernmentstaff,aswellastosustainedgovernmentfinancialsupport.Thisinturnisexpectedtocontributetomoreeffectiveteachingandanimprovementintheschoolenvironment.
22InNigerandSokotothetargetedIQSsareregisteredwiththeSAME,whichistheleadagencycoordinatingtheimplementationofthesupport.InBauchi,191GEP3IQSsaremanagedbytheSUBEB.TheremainingnineIQSsfallunderBauchiSAME.
©EDOREN 45
5.2 Baselinemethodology
TheToCofGEP3’sIQSSwasusedasaframeworktoformulatetheevaluationquestions.Theevaluationquestionsinterrogateawiderangeofthecause–effectassumptionsunderlyingdifferentstepsintheToC,inordertobetterunderstandhowchangemaycomeabout.Error!Referencesourcenotfound.AnnexBpresentsalistoftheevaluationquestions.Wewillalsosynthesisebaselineevidenceperevaluationquestionaspartofthepresentationofthebaselinefindings.TheevaluationprioritisesthequestionofhowGEP3’ssupportiscontributingtochangesinlearningoutcomes(ratherthanretentionor
enrolment),particularlybylookingatchangesinteachingandtheschoolenvironment.Similartotheearlylearningevaluation,weapplyanequityperspectiveinmeasuringlearningoutcomes.BecauseofthemultifacetednatureofIQSSandthecomplexIQScontexttheevaluationquestionsaimtoobtainanunderstandingofhowIQSScontributestointendedchangeswithintheIQScontext,ratherthantoquantifytheattributableeffectoftheintervention.Furthermore,theevaluationwillpayattentiontounintendedconsequencesoftheIQSS.Atmidlinein2017evaluationquestionsrelatedtoGEP3’scontributiontoteachingandtheschoolenvironmentwillbeanswered,whiletheeffectonlearningoutcomeswillbethesubjectoftheendlineanalysis.
ToevaluateIQSSweapplyanevaluationapproachthatdrawsontheprinciplesofcontributionanalysis
andreliesonastrongmixofquantitativeandqualitativemethods.Contributionanalysisisapragmatictheory-basedevaluationapproachthatassessescausalpathwayswithintheToC,reportsonwhethertheintendedchangesoccurredornot,andidentifiesthemaincontributionstosuchchanges.TheIQSSevaluationdoesnotmakeuseofacomparisongrouptomakecausalinferencebutseekstomakecrediblecausalclaimsaboutthecontributionIQSSismakingtoobservedoutcomesbyverifyingthechainofexpectedresultsandassumptionsofacredibleToC,aswellasassessingthecontributionofalternativeexplanationstoobservedoutcomes(Mayne,2012;DelahaisandToulemonde,2012).23BasedontheIQSSToCandinlinewiththeevaluationquestionsthreecontributionclaimswereidentified,onwhichevidence
willbecollectedduringseveralroundsofdatacollection:
1. ContributionClaim1:GEP3’sIQSScontributestomoreeffectiveteachingofformalsubjectsinIQSs.
2. ContributionClaim2:GEP3’sIQSScontributestoanimproved,girl-friendlyschoolenvironmentinIQSs.
3. ContributionClaim3:Moreeffectiveteachingofformalsubjectsandanimproved,girl-friendlyschoolenvironmentcontributetoimprovedlearninglevels,particularlyamonggirls.
Datacollectiontakesamixed-methodsapproach,combiningquantitativeandqualitativemethods.QuantitativedatacollectionconsistsofrepresentativesamplesurveysamongGEP3IQSsinthe12GEP3LGAsacrossBauchiandNiger.Baseline,midlineandendlinesurveyswillbeconductedinacohortofIQSssampledatbaseline.ThequalitativeresearchtakesplaceinpurposivelysampledcasestudyIQSsthatarealsoincludedinthequantitativesurveysample.ThesamecasestudyIQSswillbevisitedatbaseline,midlineandendline.BothquantitativeandqualitativebaselinedatacollectiontookplaceduringOctober–November2015.2425
23Wedecidednottouseacomparisongroupforseveralreasons.First,theevaluationquestionsprioritisedunderstandingofhowIQSScontributes to intendedchangeswithin the IQScontext, rather than toquantify theattributableeffectof the intervention.Second, itwasdifficulttoconstructacomparisongroupasnosampleframeof IQSssimilartotheGEP3pilot IQSswasavailable.Third,resourceswerenotavailabletoconstructcomparisongroupsforalloftheinterventionsunderevaluation.Prioritywasgiventotheearlylearningintervention.24AtthattimetheinductionworkshopofthefirstIQSfacilitatorcohorthadalreadytakenplace.Becauseoflogisticalreasonsthebaselinecouldnottakeplaceearlier.Wedonotexpectthistosignificantlyaffectthevalidityofthebaselinedatasincethecluster-levelmonthlymeetingsareconsideredtobethecoreofthecapacitydevelopmentinterventionandhadnotyetstartedatthetimeofthebaseline.Furthermore,whileteachingknowledgeandskillsmayhavebeenaffectedtolimitedextent,itishighlyunlikelythatpupillearningoutcomeswillalreadyhavebeeninfluenced.25MidlinedatacollectionisscheduledforMay–June2017,togetherwiththeearlylearninginterventiondatacollection.
©EDOREN 46
TheIQSsurveytookplaceinasampleof60IQSs,stratifiedaccordingtothe12GEP3LGAsinBauchiandNigerKatsina(fiveIQSsperLGA,30IQSsperstate).BaselinedatacollectionwasconductedbyOPMNigeria.Thesurveywassuccessfulinachievingtheplannedschoolsamplesize.26ItisimportanttonotethatahigherproportionofthesampledIQSsarelocatedinurbanareasinBauchicomparedtoNiger(27%inBauchi,versus10%inNiger).Withintheschoolsmaleandfemalepupilsaswellasteacherswererandomlysampledaspartofthesurvey.27Similartotheearlylearninginterventionbaseline,thetargetpopulationswere:pupilsenrolledinaP2equivalentgradeandteachersteachinginP1–3equivalentgrades.Thefinalsampleincludes576pupils(ofwhich48%aregirls)and96teachers(seeAnnexC).Inaddition,theheadteacherineachschoolwassurveyed,aswellasCBMCrepresentativesin52IQSs.Alldatahavebeenanalysedusingsamplingweights,andthereforerepresenttheentiretargetedpopulation.ThesamesevendatacollectioninstrumentsaswereusedintheearlylearningsurveywereadministeredusingCAPI.In
addition,apupilnumeracyandCBMCquestionnairewereadministeredtopupilsandCBMCmembers
respectively.
ThequalitativecasestudiestookplaceinsixIQSsusingpurposive‘typicalcasesampling’and‘extreme
casesampling’.CaseswereselectedfromthreedifferentLGAsperstatethatwerethemselvesselectedafteracategorisationaccordingtoaveragepublicprimaryschoolperformanceongirls’educationindicators.28Thequalitativeresearchmadeuseoffourresearchtechniquesorinstruments:qualitativeclassroomobservationandunstructuredteacherpracticediscussions;KIIswiththeIQSproprietors/Mallams,headteachers,communityleadersandlocalgovernmentIQSofficers;andfocusgroupdiscussionswithparents,girlpupilsandboypupils.AppliedthematicanalysiswasusedtointerpretthedataaccordingtothemesthatwerebasedontheIQSSToC.
5.3 Baselinefindings
Inthissectionwepresentasynthesisofthebaselinefindingspercontributionclaimsetoutinthemethodology.Inlinewiththemixed-methodsapproachwecombinequantitativeandqualitativedatainthesynthesis.Amoredetailedpresentationofthedataisincludedintheevaluationtechnicalreport.Throughoutthetextwewillmakereferencetotheevaluationquestionsinsummaryboxes.SincetheevaluationquestionsarecausalquestionsrelatedtotheIQSSintervention,theycanonlybeansweredatmid/endline.Nonetheless,wesynthesiseevidencethatprovideabaselineassessmentoftheoutcomesexpectedandtheassumptionsunderlyingthecausalchain.
5.3.1 ContributionClaim1:GEP3’sIQSScontributestomoreeffectiveteachingofformal
subjectsinIQSs
GEP3’sstrategyforintegratedQur’aniceducationaimstoprovideaccesstoqualityeducationinIQSs,particularlyforgirls,byimprovingeffective,gender-sensitiveteachingoftheformalsubjectsincludedintheharmonisedintegratedcurriculum.GEP3’scontributiontomoreeffectiveteachingassumesthatIQSfacilitators’knowledgeandskillsandgender-sensitiveclasspracticescanbeimprovedthroughtrainingandmentoring.Mentoringisalsoassumedtoincreaseteachermotivation,whichcanagaincontributetomoreeffectiveteaching.Inaddition,thedistributionofHausateachingandlearningmaterialsisexpectedtosupporteffectiveteachingprocesses,asisimprovedpedagogicalleadershipandschoolmanagementbytheheadteacher.Inthissectionwesummarisethebaselinefindingsonthesedifferentintermediaryoutcomes.Wefirstdescribethecontextinwhichtheteachingtakesplacesandsomeofthefacilitators’
2635%oftheoriginallysampledIQSshadtobereplaced,mainlyduetotheIQSnotbeingconsideredeligibleforthesurveygiventhestudyuniverse(seeBaselineTechnicalReport).27Unliketheearlylearninginterventionpanelsurvey,acrosssectionalsurveyisusedforpupil-leveldatacollection(seeBaselineTechnicalReport).28Thefollowingindicatorswereused:thegenderparityindex,girls’transitionrateinprimaryschoolandacompositeindexofthepupil-to-qualified-teacherrate,theshareofqualifiedfemaleteachersandthepupil–classroomratio.
©EDOREN 47
characteristics.Amorein-depthdiscussionoftheschoolenvironmentisprovidedinthediscussionofContributionClaim2.
Summaryofevidencerelatedtoevaluationquestions
Q1:HowwellhasGEP3teacher
capacitydevelopment(training
andmentoring)contributedto
improvedteacherknowledgeand
skills,andmoreeffective
teachingintheclassroom?
Levelsoffacilitatorknowledgeof,andcompetencein,effectiveteachingareverylow.Additionalcapacitydevelopmentisthereforehighlyrelevantbutneedstobewelltargetedtotheneedsofthefacilitators,toreachtheappropriatefacilitators,andtotakeintoaccountthediverseandchallengingteachingcontextinIQSs.Facilitators’motivationisshapedbyissuesofremuneration,socialstatusandcommunityrolesandrelations.
Facilitatorsdemonstratedverylowlevelsofpedagogical,curriculumandsubjectknowledge.Forinstance,lessthan3%offacilitatorscoulddemonstratecompetenceinidentifyinglowperformers,interpretingwordsandphrases,anddisplayingwritingskills.Theuseofpupil-centredapproacheswasalsolimited.Thiswasdespite43%offacilitatorsreportinghavingreceivedtraininginthepasttwoyears.Theappropriatefacilitatorsarenotalwaystrainedandonlyalimitedshareoffacilitatorshaveprofessionalteachingqualifications.Facilitatorsshowinterestinbeingtrained,particularlyinsubjectknowledge.
Facilitatorsconsidertheirroletobeimportantandenjoyworkingasteachers,whilsthavingamorenegativeperceptionoftheirteachingefficacy.Facilitatorsthemselves,aswellascommunitymembers,donotperceivetheirlackofcompetencetobeacauseoflowpupilperformance.WhileremunerationdoesnotappeartobethemainmotivationforfacilitatorstoteachinIQSs,thelackofadequateremunerationmakesitdifficulttoattractqualifiedfacilitatorsandholdthemaccountable.
AquarterofIQSsonlyhaveonefacilitator,whichwillconstrainamentoringprocessthatisbasedonschool-basedpeerinteraction.90%ormoreofthefacilitatorsreportspeakingHausaandEnglish,althoughthisdoesnotmeanthattheyareliterateinHausaorEnglish.Notably,only34%offacilitatorswerefoundtobefullycompetentinGrade1and2-levelHausa.
TheteachingoftheintegratedcurriculuminIQSsappearstohavegainedacceptancebutisimplementedtovaryingdegrees.Thisislikelytoinfluencethescopeforgainsineffectiveteaching.Facilitatorattendanceseemstobeflexible,withnon-religioussubjectsbeingtaughtasandwhenafacilitatorisavailable.Amorestructured,intensifiedteachingprogrammewillbehardtoenforcebecausebaselineresultssuggesttheneedforintegrationtobegradualandfortheschooltoretainitsQur’aniccharacter.
Q2:HowwellhasGEP3teacher
capacitydevelopment
contributedtoanimprovement
ingender-sensitiveteaching?
Whilefacilitatorsdisplaysomegender-sensitivetechniques,gender-biasedclassroompracticesprevail.
Facilitators,whoaremostlymale,expresspositiveattitudestowardsgirls’education.Theyarealsoareawareof,andsometimesdisplay,gender-sensitivetechniques.However,genderbiasescontinuetoaffectclassroomorganisationandpracticesduringtheteachingoftheintegratedcurriculum.Thisreflectsentrenchedgenderedattitudesandexpectationsaboutthebenefitsofeducationforgirls,andculturalnormsabouthowgirlsandboysrelate.
Q3:HowwellhasGEP3head
teachercapacitydevelopment
(trainingandmentoring)
contributedtoimproved
pedagogicalleadershipand
SchoolleadershiprolesinIQSsarenotclearlydefined,arepossiblyshared,andarenotalwaysassignedbasedonabilityandqualifications.Thisislikelytocomplicatetheprocessofidentifyingwhopreciselyshouldbetargetedfortraining.
SchoolleadershipinIQSsiscomplexandissharedamongstmany
©EDOREN 48
schoolmanagement? stakeholders.IQSsattimeschooseheadteachersbasedonsocialstatusandperceptionsregardingleadership,ratherthanbasedonteachingknowledge.Thelackofclarityaboutleadershiprolescanmakeitdifficulttoidentifywhoseleadershipcapacitiestobuild.GEP3needstoconsiderwhetherthoseattendingtrainingarethemostqualifiedtocarryoutaleadershiprole,particularlywithregardstopedagogicalleadership.
Headteachersdemonstratesomedegreeofpedagogicalleadership(particularlyinNiger)andtakeactionstoimprovepupilandteacherattendance.However,record-keepingispoor.43%hadattendedtrainingduringthelasttwoyears.Headteachers’professionalandacademicqualificationsaregenerallylowand,onaverage,belowthoseoffacilitators.
Q4:Towhatdegreehave
teachingandlearningmaterials
suppliedthroughthe
interventionbeenperceivedby
teachersandheadteachersas
appropriateandwelltargeted?
Havetheybeenusedformore
effectiveteaching?
TheavailabilityofteachingandlearningmaterialsinIQSsisverylimited.GEP3’sdistributionofsuchmaterialscanfillanimportantgap.However,materialsneedtobeappropriateforthelanguagesspokenintheIQS,thesubjectstaughtandtheskilllevelsofthefacilitators.
Theavailabilityofteachingandlearningresourcesisverylimited.AlmostnoHausamaterialsareavailable,althoughHausaisnotnecessarilythemothertongueinallcases.InNiger43%ofpupilsreportedlyspeakNupeathome.IQSsmostlyteachmathematicsandlanguagesascoreintegratedsubjects,whichmakesthedistributionofnumeracyandliteracymaterialsbyGEP3wellalignedwiththesubjectstaughtintheIQSs.GiventhelowHausaliteracylevelsoffacilitatorsHausamaterialswillneedtobefit-for-purpose,notonlyinrelationtowhatneedstobetaught,butalsowithrespecttotheskilllevelsoffacilitators.
Teachingcontextandteachercharacteristics
IQSsprovideadiverseandchallengingteachingcontext.20%oftheIQSsampledidnothaveclassrooms,and30%hadonlyoneclassroom.Asshowninthequalitativecasestudies,someIQSsprovideclassesintheopenair,withablackboardhangingontheouterwalloftheMallam’shouse.Gradeprogressionisorganisedinvariousways.Aroundtwo-thirdsoftheIQSsinthesurveysampleseemtobeorganisedinsixlevels,likepublicprimaryschools;theremainderorganisetheteachingofthecurriculumintooneortwostages.Classroomobservationdataindicatethatthepupil–teacherratioisquitevaried,withfromfourto183childrenbeingtaughtbyoneteacher,withameanof45.Therefore,thefacilitatortrainingwillneedtotakeintoaccountthediverseteachingcontextofthefacilitators.
ThepotentialmobilityofthetraditionalTsangayaQur’anic29schoolsislikelytoposesomechallenges.
Thiswouldlikelyinterruptteachingoftheintegratedcurriculumandmayaffectthegenderinclusivenessoftheschool,asthefacilitatormaynotmovewiththereligiousteacher(theMallam),theMallamwouldmoveonlywiththeboys,andthefacilitatormaynotcontinueteachingtheintegratedcurriculumintheabsenceoftheMallamandthemalepupils.Atbaselinewedonothavequantitativedataonthemobilityoftheschoolsample.TheheadteachersofthecasestudyIQSsallstatedthattheirschoolsaresedentary,butinoneschoolinBauchipupilsindicatedthatmovingwasplannedforthefuture.GEP3needstocloselymonitorthemobilityoftheIQSs.Schoolmobilityanditsconsequencesforeffectiveteachingoftheintegratedsubjectscouldbediscussedaspartoftheheadteachertraining,inordertosensitiseschoolleadership.
IntegrationhasbeenimplementedacrossBauchiandNiger,buttovaryingdegrees.AconditionforeffectiveteachingofformalsubjectsisfortheQur’anicschooltobeactuallyintegrated–thatis,providingtheintegratedcurriculum.Thisisnotnecessarilythecase:22%oftheGEP3IQSscontactedforthebaseline
29TheTsangayaelementmeansthattheinstitutionhasamobileelement,inwhichtheMallammoveswithhispupils‘inthebeliefthatanitinerantlifeisessentialforthemtofullyconcentrateontheirstudy’(Antoninis,2012).
©EDOREN 49
surveywerenon-integratedand14%hadbeenintegratedforlessthanoneyear.Amongthesurveyedschools,IQSswereintegratedonaveragefortwoyearsinBauchiandthreeyearsinNiger.FewIQSsprovideallfivecoresubjectsoftheintegratedcurriculumintheearlygrades.30Interviewswithheadteachersindicatethatonly12%ofIQSsteachmathematics,Hausa31,English,basicsciencesandsocialsciences,whileEnglishandmathematicsareprovidedinaround90%ofIQSs(seeError!Referencesourcenotfound.).Accordingtotheheadteachersinterviewed,Hausaisalsofrequentlytaught,althoughonly32%offacilitatorsinterviewedindicatedthattheytaughtHausa.32Thefactthatnotallcoresubjectsaretaughtmayberelatedtothelimitednumberofteachinghoursfornon-religioussubjects.Headteachersreportedthatonaveragethreehoursperweekarededicatedtotheintegratedcurriculum,whichissubstantially
belowtheeighthoursrecommendedintheNationalBenchmarkforIntegratedBasicEducation.Furthermore,thequalitativecasestudiessuggestthatactualinstructionaltimeismorelimitedthanthetimedeclaredbytheheadteachers.
Figure7: CoresubjectstaughtinIQSs
ThedegreeofintegrationissomewhatdifferentinNigerandinBauchi.Intheformerstate,20%ofIQSsprovidethefivecoresubjects,whilethisisonlythecasefor3%ofIQSsinBauchi.ReportedinstructionaltimeisfourhoursonaverageinNiger,comparedtotwohoursinBauchi.Hence,IQSsinNigerseemtobemoreintegratedthaninBauchi,whichisconfirmedbythedifferenceinthenumberofyearsofintegrationandinthefactthatmoreIQSsinNigerareorganisedintosixlevels,likepublicprimaryschools.Thevariationinthedegreeofintegrationislikelytoaffectthegainsineffectiveteachingthatcantakeplace.Also,thevariedadoptionoftheintegratedcurriculumdeservesattentionduringimplementation,totheextentthatGEP3seekstoimproveeffectiveteachingofallcoresubjects.ThefactthatmostsurveyedIQSsareteachingmathematicsandlanguagessupportstheplausibilityofliteracyandnumeracybeingthecoreGEP3learningoutcomestoimprove.
Integratededucationappearstohavegainedacceptance.Inorderforeffectiveteachingoftheintegratedcurriculumtotakeplacefacilitators,proprietors,parentsandtheschoolcommunityneedtobesupportive
30AccordingtotheNationalBenchmarkforIntegratedBasicEducationinQur’anicschools(2013)thefirststagecurriculum(P1–P3)ismadeupofmathematics,thelanguageoftheimmediateenvironment,English,socialstudiesandbasicsciences.31Hausaisnotnecessarilythelanguageoftheimmediateenvironmentinallschoolcommunities–particularlyinNiger,whereFulfuldecanbethemothertongue.WefocusonHausasinceGEP3willbeprovidingteachingandlearningmaterialsinHausa.32ThiscouldbeduetothefactthatanotherfacilitatorteachesHausa,oritmaybethatheadteachersareover-reportingtheteachingofHausa.
0
10
20
30
40
50
60
70
80
90
100
English Mathematics Hausa Basicscience Socialstudies
PercentageofIQS
Source:GEP3Baselinesurvey2015
©EDOREN 50
ofintegratedQur’aniceducation.Thequalitativecasestudiesindicatethatthisisthecase.Thisisunderpinnedbyarecognitionthatbothreligiousandnon-religiousformsofeducationareimportant,andthatneithertypeofeducationissufficientinisolation.However,interviewedstakeholdersindicatedthatintegrationneedstobegradualandthattheschoolsshouldnotlosetheirQur’aniccharacterandbeturnedintoformalschools.
MostIQSshaveoneortwofacilitators.Theaveragenumberofteachersofnon-religioussubjectsis2.4inBauchiand3.2inNiger.Thisdifferencesuggests,again,thatinNigerIQSsaremoreintegrated.25%ofIQSshaveonlyoneteacherofnon-religioussubjects,while37%havetwo.Facilitatorsmostlyteachoneortwosubjects.MorefacilitatorsteachmultiplesubjectsinNigerthandosoinBauchi.InIQSswithonlyonefacilitatorschool-basedpeerinteractionasamechanismtostrengthenknowledgeandskillswillnotbefeasible,whichmaylowertheimpactoftheIQSSintervention.InIQSswithmorethantwofacilitatorsGEP3needstoconsiderhowknowledgecanbedisseminatedandembeddedbeyondthetwofacilitatorstargetedforGEP3training.Thisishighlightedbythequalitativecasestudies,whichindicatethatknowledgesharingaftertrainingdoesnotnecessarilyhappen.Sincefacilitatorsmostlyteachalimitednumberofsubjects,itwillbeimportantforGEP3totargetthosefacilitatorsfortrainingwhoactuallyteachthesubjectstowhichthetrainingrelates.
ThefacilitatorprofilesdifferslightlybetweenBauchiandNiger.Only9%ofthesurveyedfacilitatorswerefemale,althoughthiswashigherinNigercomparedtoBauchi,despitemoreIQSsinBauchibeinglocatedinurbanareas(seeTable5).FacilitatorsinBauchiaremoreexperiencedandprofessionallyqualified.ThismayberelatedtothefactthatinBauchitheSUBEBismanagingmostoftheGEP3schools,whileinNigertheSAMEisresponsible.Acrossthetwostates,lessthanhalfofthefacilitatorshavesomedegreeofprofessionaleducationalqualification(NCEorGrade2certificate).Thefacilitatortrainingneedstobeadaptedtothisfact.Thequalitativecasestudiesconfirmthattheleveloffacilitatorqualificationvariessignificantlyacrossthecases.Amongthecases,themoreprofessionallyqualifiedfacilitatorsarealsoteachinginpublicprimaryschools.Thefacilitatorswithahigherlevelofqualificationfeltmoreconfidentteachingtheintegratedcurriculum.Withregardstolanguage,90%ormoreofthefacilitatorsreportedspeakHausaandEnglish.ItisimportanttoacknowledgethisfactsincetheIQSSinterventionincludesthedistributionofHausateachingmaterials.However,aswillbediscussedbelow,thisdoesnotmeanthatthefacilitatorsareliterateinHausa,whichmeansthatcarefulattentionwillneedtobegiventothefacilitatortrainingmethodologyused.InNiger39%ofthefacilitatorsindicatedthattheyspeakNupe,whichisnoteworthysincealmosthalfofthepupilsinNigerindicatedthattheyspokeNupeathome.
Table5: Summaryoffacilitatorcharacteristics
Forsurveyedfacilitatorstheaverage… Ofteachers…
• …ageis32years;
• …teachingexperienceissevenyears(nineyearsinBauchiandfiveyearsinNiger);82%ofteachershaveatleasttwoyearsofteachingexperience;and
• …teachersarelikelytoteachoneortwonon-religioussubjects–thesebeingmostlymathematicsorEnglish.
• …96%speakHausa;90%speakEnglish;17%speakArabic;inNiger39%speakNupe,while13%inBauchispeakFulfulde.
• …only9%arefemale(2%inBauchi,16%inNiger));
• …14%alsoteachreligioussubjects;
• …35%holdanNCEqualification(51%inBauchi,20%inNiger)and66%havepassedtheSCCE(61%inBauchi,70%inNiger);33%inBauchihaveareligiouseducationqualification,comparedto11%inNiger;and
• …43%attendedtrainingduringthelasttwoyears(36%inBauchi,49%inNiger).
Source:GEP3Baselinesurvey2015
©EDOREN 51
Thereisroomforfurtherexpansionanddeepeningoffacilitatortraining.43%offacilitatorsreportedthattheyhavereceivedtraininginthepasttwoyears.Teacherswithaprofessionalqualificationaremorelikelytohaveattendedtraining.Amongstthosewhoreceivedsomeformoftraining,around80%reportedbeingtrainedonteachingmethodsandaround70%indicatedthatthetrainingwasimplementedbyGEP/UNICEF.Thesurveyalsoidentifiedthattheappropriatefacilitatorsarenotalwaystrained–notalltraineeswhoattendedGEP3’sfacilitatortraininginAugust–September2015wereinfactfacilitatorsteachingatanIQS.WithinthesampledIQSsforwhichthesurveyteamhadlistsoftrainedfacilitators22%offacilitatorswereunknownornotteachingattheIQS.Accordingtothefindingsofthequalitativecasestudiesfacilitatorsareinterestedinreceivingtraining:theyexpressaneedformoretraininginsubjectknowledgeinordertoeffectivelyusethetoolsthatseemtobethefocusofthetraining,togetherwithsensitisationaboutthevalueofinclusiveinstruction.ThecasestudiesalsosuggestthatfacilitatorswithinthesameIQSarenotnecessarilyawareofeachother’sparticipationintraining,whichmayhinderknowledgespillover.Hence,knowledgedisseminationstrategiesneedtobeconsideredtoreinforceimpact.33
Facilitators’knowledgeandskills
Facilitatorsdisplayedverylowlevelsofcompetenceacrossthesixdomainscoveredbytheteacher
assessments.Levelsofcompetencewereparticularlylowonthetwodomainsassociatedwithpedagogicalknowledge:only2%offacilitatorswerecompetentinidentifyinglowperformersandnoneofthefacilitatorswerecompetentinevidencingjudgementsanddiagnosingpupilperformance(seeFigure8).Thereisaverystrongtraditionineducationalresearchthatacknowledgesthatastudentlearnsbestwhenteachingistargetedtowhats/heisreadytolearn.Facilitators’weakskillsonthesefrontsthereforepresentmajorchallengestoimprovingpupillearning.
Facilitatorsalsodisplayalmostnoevidenceofcompetenceinwritingandinterpretingwordsandphrases.KnowledgeofGrade1and2-levelHausaappearsmoreprevalent,althoughalargeshare(66%)offacilitatorswerenotabletodisplaycompetenceinthis,whichraisesquestionsaboutfacilitators’abilitytoraisepupils’learninglevelsinthisarea.Similartothefindingsoftheearlylearningbaseline,writingskillsappeartobecorrelatedwithotherskills,whichsuggeststhattheliteracylevelsofthefacilitatorsmaybeakeyissue–ifthoselevelsarelowthislimitsperformanceacrossarangeoftheareasfacilitatorsneedtobecompetentininordertoimprovepupillearning.
Thequalitativecasestudyfindingsreinforcethispicture.Facilitatorswereoftenunabletoexplainwhichsubjectmattershouldbetaught,orwhytheyteachwhattheyteach.Manyfacilitatorswereteachingatthelevelthattheywerethemselvestaughtto.Whilstfacilitatorsdisplayedawarenessofteachingandpedagogicalmethods,inmostcasesfacilitatorsthemselvesdidnotseemtohavesufficientsubjectknowledgetoteacheffectively.
33Thefindingsofthemotivationcompositeindex(seesectiononfacilitatormotivationandattendance)indicatethatfacilitatorshavearelativelypositiveperceptionaboutteacher-to-teacherinteraction.Thereisthereforeopportunitytopromoteknowledgeexchange.
©EDOREN 52
Figure8: Percentageofteachersachievingwithinthelower,middleandupperbandsof
competenceacrosstheteacherknowledgeandskillssubscales
Facilitatorsdonotseemtoperceivetheirownlackofcompetencetobeacauseoflowpupil
performance.Facilitatorsgenerallyperceivepupilefforttobeboththecauseoflowpupilperformanceaswellasthebestresponsetolowperformance.Whenfacilitatorswereaskedwhatstepstheycouldtaketoimprovepupils’performance,theirresponsesfocusedonencouragingpupilstoincreasetheireffortandtopayattentioninclass.Therewaslittlediscussionoftheerrorsinpupils’work,andwhatfacilitatorscoulddotoimprovethis.Thefocusonpupileffortwithinthiscontextislikelytobeareflectionofthelowlevelsofcompetenceamongstfacilitatorsthemselves,andtheirlackofunderstandingofwhattheycoulddoto
improvepupils’performance.Arelatedfindingofnotefromthequalitativecasestudiesisthatcommunitymembersoftenperceivefacilitators’subjectknowledgetobehigh.Thisisinstarkcontrasttothefindingsoftheteacherassessments,andcanbeexplainedbyparents’ownlimitedknowledge.Thissuggeststhatparentsareunlikelytoholdfacilitatorstoaccountforlowqualityteaching.
Gender-sensitiveclasspractices
Facilitatorsstatedhavingpositiveattitudestowardsgirls’education,butgenderbiasespersist.Thefacilitatorsinthesixcasestudyschoolsexpressedpositiveattitudestowardsgirls’education.However,itisunclearwhetherthisaccuratelyrepresentsfacilitators’attitudesaswecannotruleoutsomedegreeofsocialdesirabilitybiasintheseresponses.34Indeed,bothqualitativeandquantitativedatapointtosomedeeplyingrainedgenderbiases.Somecasestudyfacilitators,whileperceivinggirls’educationtobeimportant,thinkthatgirlsarelessablethanboys,andthatgirlsdonotrequireasmucheducationasboysdo.Inallbutonecase,facilitatorsperceivedboystobemoreintelligentthangirls.Facilitators’attitudesseemtobeinfluencedbytheperceivedimportanceattachedtogirls’education,giventhegenderedroles
34Socialdesirabilitybiasreferstothetendencyofrespondentstoanswerquestionsinamannerthatwillbeviewedfavourablybyothers.Thisislikelytohavebeenthecaseinthequantitativesurvey,inwhichweobservedextremecomplianceeffectsacrossallitemsdevelopedtomeasureattitudestowardsgirls–thispointstosocialdesirabilitybiasandcallsintoquestionthevalidityandreliabilityofthemeasure.Therefore,changesinthegendersensitivityoffacilitators’classroompracticesisbestdetectedthroughthequalitativeresearch,althoughsomedesirabilitybiasmayalsobeobservedheretoo.
0%
20%
40%
60%
80%
100%
Abilitytoidentifylowperformers
Abilitytoevidence
judgementsanddiagnose
Teacherwritingskills
TeacherGrade2Hausa
knowledge
Teachercomprehensio
nskills
Interpretingwordsandphrases
Upperband 1.9 0 .9 34.0 25.4 2.8
Middleband 6.5 .9 .9 27.2 18.1 4.2
Lowerband 91.6 99.1 98.1 38.8 56.5 92.9
Note:Twocut-pointsweredefinedforeachsubscaletocreatethreeproficiencylevelsperscale.Theseproficiencylevelsare:LowBand–noevidenceofskill;MiddleBand–evidenceofrudimentaryskill;andUpperBand–evidenceofcompetenceSource:GEP3Baselinesurvey2015
©EDOREN 53
girlsareexpectedtotakeupinfuturelife.Thisisconfirmedbytheresponsesaspartoftheteacherassessments,whichindicatedthatgirlsareperceivedtobenaturallylesssuitedtothedemandsofeducationthanboys,ormoresuitedtootherroleswithinsociety.
Whilefacilitatorsdisplaygender-sensitivetechniques,gender-biasedclassroompracticesprevail.Bothqualitativeaswellasquantitativefindingsindicatethatfacilitatorsareawareof,andsometimespractice,gender-sensitivetechniques,suchasactivelyengagingbothgirlsandboys,andevenfocusingmoreongirls.However,thesepracticesseemtobeperformedinatokenisticway—inlinewithdevelopmentprojects’sensitisationefforts—andareunlikelytobeabletosurmountthegender-biasedclassroomarrangementswithintraditionalQur’anicschools,suchasthepracticeofsegregatingboysandgirlsintheclassroom.Furthermore,teachingpracticesareinfluencedbyfacilitators’andpupils’gender-biasedattitudesaboutgirls’abilitytolearnandtheimportanceofeducationintheirfuturelife.Itis,therefore,importantthatfacilitatortrainingnotonlyencouragestheuseofgender-sensitivetechniquesintheclassroom,butthatitalsoseekstocontributetothetransformationofthefacilitators’attitudestowardsgirls’abilitiesandtheimportanceofgirls’education.
Facilitators’motivationandattendance
WithintheIQSSToCfacilitatormotivationandattendanceareconsideredbothtobeintermediaryresultsofmentoringandbetterschoolmanagementaswellassupportingfactorstoachievemoreeffectiveteaching.Wemeasuremotivationthroughafive-scalemotivationindex(seeError!Referencesourcenotfound.intheearlylearningbaselinesection).
Overall,facilitatorsconsidertheirroletobeimportantandenjoyworkingasteachers,buttheyhavea
negativeperceptionoftheirteachingefficacy.Thequantitativefindingsofthemotivationscalescoresareverysimilartothoseoftheearlylearningsample.Onaverage,thesurveyedfacilitatorsscorehighestoneffortandlowestonperceivedteacherefficacy(seeFigure9).ThefacilitatorsinthecasestudyIQSsreportfeelingintrinsicallymotivatedbywhattheydo,andschoolstakeholders,includingpupils,considerfacilitatorsgenerallytobededicated.
Figure9: Comparisonofmotivationsubscales,IQSSsample
1.99
3.57
3.73
2.19
3.47
0 0.5 1 1.5 2 2.5 3 3.5 4
Perceivedteacherefficacy
Interestandenjoymentofteaching
Effortandimportance
Pressureandworkrelatedtension
Teacher-to-teacherinteraction
Source:GEP3Baselinesurvey2015
©EDOREN 54
Facilitators’motivationisshapedbyissuesofremuneration,socialstatusandcommunityrolesand
relations.Thequalitativefindingsindicatethatthefactthatmostfacilitatorsarenotpaidmakesitdifficulttoattractqualifiedfacilitators.Thequantitativefindingsconfirmthataverysmallshareoffacilitatorsarepaideitherasalaryorastipend(3%ofthesurveysampleinBauchiand33%inNiger)andpointtoapositiveassociationbetweenreceivingremunerationandteachermotivation.FacilitatorsinthecasestudyIQSsreportnotfeelingvaluedduetotheirlowpay,asitmakesthemstruggleto‘moveupalevel’inlifeascomparedtotheirpeers.Lackofremunerationislikelytoparticularlyinfluencemenduetothegenderedexpectationsofmenasbreadwinnerswithinsociety.StakeholdersofthecasestudyIQSsacknowledgethatthereisaninherentinconsistencyhere,withfacilitatorsexpectedtodeliveratthestandardofateacher,whilefendingforthemselvesfinancially.Asisevidentfromthequalitativefindings,bynotpayingfacilitators,theexpectationsplacedonafacilitator,andthedegreetowhichthecommunitycanholdfacilitatorstoaccount,decrease.Yet,remunerationclearlyisnotthemainmotivationforfacilitatorstoteachatanIQS.InallsixcasestudyIQSs,facilitatorsaremembersofthelocalcommunityandseveralhavepreviouslybeenpupilsundertheQur’anictutelageoftheMallamorhispredecessor.Stakeholdersperceivedthistobethemainreasonwhyfacilitatorsagreetoteachvoluntarily.Thequantitativefindingssuggestthatpedagogicalleadershipmayhaveaninfluenceonfacilitatormotivation,aspositedintheIQSSToC.Facilitatorswhohaveindividualmeetingswiththeheadteachersaremoremotivatedthanthosewhodonot.
Teacherattendanceisflexible.Around75%ofthefacilitatorsreportedhavingbeenabsentatleastonceduringthepreviousthreemonths,withfacilitatorsinBauchibeingmorelikelytobeabsent,andforlonger,thanfacilitatorsinNiger.Thereisnomarkeddifferenceintheabsenteeismratesforteachersbyremunerationstatus.ThecasestudyfindingshighlightthatfewofthecasestudyIQSsseemtohavesettimetables,andformalsubjectsaretaughtasandwhenafacilitatorisavailable.TeacherattendanceinthecontextofIQSsisthereforeaflexibleconcept.Furthermore,teacherattendancedoesnotnecessarilymeanthattheteacherisprovidinginstruction.Forexample,inseveralofthecasestudyIQSsitwascommonforpupilstorun(partof)theclasswiththehelpofaclassmonitor–thatis,apupilwhoteachesyoungerstudentsifthefacilitatorisnotpresent.Ingeneral,theinterviewedstakeholdersperceivethefacilitatorstobepunctualanddonotseefacilitatorabsenteeismasakeyreasonforlossofinstructionaltime–rather,theyattributethistoanIQSnothavingenoughfacilitators.Instructionaltimeisalsoinfluencedbyfacilitatorsandpupilshavingtoengageinincome-generatingactivities.Additionally,manypupilsinthecasestudyIQSsalsoattendpublicprimaryschool,thusrestrictingIQSinstructionaltimetotheafternoonandevening,whenQur’anicteachingalsooccurs.
Facilitators’teaching
Overall,levelsofcompetenceineffectiveteacherpracticeswerelowacrossallgroupsoffacilitators.ThefindingsofthebaselinequantitativesurveywereusedtodevelopacompositeindextomeasuretheToCintermediaryoutcome:toimproveteacherpracticesandgendersensitivity.Theindexisbasedontheextentofpupil-centredlearningactivitiesobservedintheclassroom35,observationsoftheteacherlinkingthelessontopreviouslearningandlearningobjectives36,andtimeontaskinclass37.Scoresontheindexwerelowforallgroupsoffacilitators,althoughwithsomevariations.Facilitatorsmeetingheadteachersregularly(onceaweekormore),olderfacilitators(aged>50)andthoseinBauchidemonstratedsomewhathigherlevelsofeffectiveteachingpractice.Thequalitativecasestudiesfurtherindicatethatfacilitatorshavesomeawarenessofpedagogicalpractices(forexample,theimportanceoflessonplanningandactivelyinvolvingpupils)butlackadeeperunderstandingofthevalueofcertainmethodsorwhentomakeuseofwhichmethodorpractice.
35Whichinvolved,forexample,observationsofteachersassistingingroupdiscussion,usingachild’sname,askingorrespondingtoanopenquestionetc.36Thisincluded,forexample,observationsoftheteachertalkingaboutthepreviouslesson,outliningtheobjectivesoftheobservedlessonetc.37Measuredaspercentageoftimethepupilswereengagedduringlessonobservations.
©EDOREN 55
Timeontaskwasgenerallygoodinlessonsobservedaspartofthequantitativesurvey—withinthemajorityoflessonobserved(64.5%),childrenspent100%ofthelessonontask—althoughthismaybeinfluencedbyobserversbeinginthelessons.Inabout10%ofclassroomsfacilitatorswerenotpresentatsomepointduringclass.Asmentionedabove,thequalitativecasestudieshighlightthatinstructionaltimeinIQSsisorganisedinaflexible,adhoc,wayandvariesacrossIQSs.Pupils,facilitatorsandotherstakeholdersreportedthatlessonsnormallylastaround20minutes,andthatabouttwosubjectsaretaughtperday,butwithstrongvariationacrossIQSs.
Hausaisthemostcommonlanguageofinstructionbutoftenvariouslanguagesareusedinteaching
duringthesameclass.InBauchi,themajorityoffacilitators(72%)usedjustonelanguageduringthelessonobservation.Afurther24%wereobservedusingtwolanguages.InNiger,25%offacilitatorsusedjustonelanguageduringthelesson,while60%usedtwolanguages.Asmallminorityoffacilitatorsinbothstatesusedthreeormorelanguagesduringalesson.ThemajorityoffacilitatorsinbothstateswereobservedusingHausaatleastonceduringthelesson(100%inBauchiand72%inNiger,seeFigure10).Englishwasalsowidelyused,particularlyinNiger.48%ofteachersobservedinNigeralsomadeuseofalanguageotherthanHausa,EnglishandArabic(theclassroomobservationtooldidnotrecordwhattheseotherlanguageswere,althoughitislikelythatNupefeaturesprominentlyheregiventhatitisspokenbyanotableshareoffacilitatorsandpupils).
Figure10: Facilitatorlanguageuse:Shareoffacilitatorswhousedthelanguageatleastonce
duringthelesson
Availabilityanduseofteachingandlearningresources
Theavailabilityofteachingandlearningresourcesisverylimited,ashighlightedbyFigure11.Themostcommonlyusedresourcesarethosemadebyhandbyfacilitators.Atextbookwasavailableandusedinonly9%ofclassesobserved.Whenthematerialswereavailable,theyweremostlyused.ThequalitativecasestudiespointagaintostrongdiversityamongIQSs:onecasestudyIQShadaccesstoawiderangeofresourcesincludingcomputers,whileothersdidnothaveanyformalmaterialsbutusedstones,sticksandbottletopstofacilitatemathematicsclasses.SomecasestudyIQSshadreceivedacurriculumforteachingHausa,andsomeflashcardsandotherteachingaids,whichfacilitatorsperceivedtoappropriate,althoughthereliabilityofthisfindingneedstobeinterpretedwithcautionasfacilitatorsseemedtoperceiveaccesstoanymaterialstobepositive.Thecasestudiesalsoindicatethatmanychildrenlackbasicmaterials,such
0
10
20
30
40
50
60
70
80
90
100
Hausa English Arabic Other
ShareofFacilitators(%
)
Bauchi Niger
Source:GEP3Baselinesurvey2015y
©EDOREN 56
asnotebooksandpens,theprocurementofwhichistheresponsibilityoftheparents.Severalrespondentsconsideredtheinabilityofparentstopurchaselearningmaterialstobeareasonwhychildrenwillnotattendschool.
Figure11: Availabilityanduseofteachingandlearningmaterials
AlmostnoHausamaterialsareavailablebutitshouldbenotedthatHausaisnotnecessarilyalwaysthe
mothertongue.Figure11showsthatmaterialsinHausawereavailableinalmostnoneoftheobservedlessons.ThisprovidesaclearbaselinefortheIQSSintervention,withmuchopportunityforimprovement.However,whilealmostallfacilitatorssurveyedreportedspeakingHausa,correspondinglyonly73%ofthesampledchildrenreportedspeakingHausaathome–withastrongdifferencebetweenBauchiandNiger:93%inBauchi,versus54%inNiger.InNiger43%ofpupilsreportedlyspeakNupeathome.Furthermore,theteacherassessmentdemonstratesthatfacilitatorshavelowliteracylevelsinHausa,henceHausamaterialswillneedtobefit-for-purpose,notonlyinrelationtowhatneedstobetaught,butalsowithrespecttotheskilllevelsoffacilitators.
Pedagogicalleadership
SchoolleadershipinIQSsiscomplex.IQSsareprivateinstitutionsthatcanbeownedbyindividualproprietorsorownedinamorecollectivewaybythecommunitywhilststillhavinganindividualincharge.ThereligiousteachingisgenerallyledbyaMallam,whointhemoretraditionalQur’anicschoolsisgenerallyalsotheproprietoroftheschool.38Withintegration,theheadteacherfunctionsofthenon-religiouseducationareaddedtothisexistingQur’anicschoolleadershiparrangement.Thecasestudiesshowthattheschoolleadershippositions(proprietor,Mallam,headteacherandCBMCmembers)arenotnecessarilyseparateroleswithintheIQScontext.InthecasestudyIQSs,themostcommonscenarioseemstobeonepersontakingonthethreerolesofproprietor,Mallamandheadteacher.Thequantitativedatawerenotabletoconfirmthis.Inone-thirdoftheIQSsurveysample,theheadteacherwasalsoconsideredtobetheproprietor,andthiswasmorecommonforschoolsinBauchi.However,thequantitativedataneedtobeinterpretedwithcautionbecauseofthevariouswaysinwhichthedifferentpositionsarereferredto.Ingeneral,thereisalackofclaritywithinmostcommunitiesregardingwhohaswhatrole,andwhatthatroleentails.Thelackofclarityabouttheserolescanmakeitdifficulttoidentifywhoseleadership
38AsAntoninis(2014)outlines,therearebroadlytwotypesofreligiouseducationinstitutioninnorthernNigeria:Qur’anicreligiouseducationinstitutionsandIslamiyyareligiouseducationinstitutions.Qur’anicinstitutionsencompassbothmakarantaallo(‘schooloftheslate’)andTsangaya.
0.0
5.0
10.0
15.0
20.0
25.0
Video Audio
Toolsorobjectsfromthelocal
environment
HandmadeMaterialsin
HausaPoster,chartorpictures
Textbook
Resourcesused 0.0 0 3.8 19.9 1.0 1.9 8.8
Presentbutnotused 0.0 0 4.8 4.7 .9 1.9 1.9
Percen
tofo
bservedlesson
s
Source:GEP3Baselinesurvey2015
©EDOREN 57
capacitiestobuild.Differentactorsmaybeinvolvedintheimplementationofsimilarrolesandthemostqualifiedpersonmaynotleadontherole,forexample,ofpedagogicalleadership.
Headteachersarealmostalwaysmaleandhavelowprofessionalandacademicqualificationswith
differencesbetweenBauchiandNiger.39AlloftheheadteachersinBauchiaremaleandonly7%ofheadsinNigerwerereportedtobefemale.TheheadteachersinBauchiareabout10yearsolder,onaverage,thanheadteachersinNiger,have10moreyearsofworkexperienceatthecurrentschool,andhavemoreyearsofteachingexperience(seeTable6).GiventhatintegrationisrelativelyrecentinBauchi,thissuggeststhat,particularlyinBauchi,theheadteachermayalsobetheMallam.Theprofessionalandacademicqualificationsofheadteachersaregenerallylow,and,onaverage,belowthoseoffacilitators,whichmaychallengetheheadteachers’technicalcapacitytoexercisepedagogicalleadershipoverfacilitators.HeadteachersinNigerhavehigheracademicqualifications:57%havepassedtheSSCE,versusonly23%havingdonesoinBauchi.Intermsofprofessionalqualifications,inbothstatesonlyaminorityofheadteachershaveanNCE,butamajorityofheadteachersinBauchihaveareligiouseducationqualification,indicatingagainthattheheadteacherroleisquiteoftentakenupbytheMallaminBauchi.Thequalitativecasestudiesfurtherhighlightthatthequalificationsoftheheadteacherdifferconsiderablyacrosscases,withsomehavingnoformaleducationatall.However,mostheadteachersappearedconfidentintheirabilitiestoinstructtheirfacilitators.However,pedagogicalleadershipseemstobesharedamongstmanystakeholders,withCBMCmembersguidingheadteachers,headteachersadvisingfacilitators,andproprietorsandcommunityleadersalsoofferingtheirperspectives.
Table6: Summaryofheadteachercharacteristics
Forsurveyedheadteacherstheaverage… Ofheadteachers…
• …ageis40years(44yearsinBauchi,36yearsinNiger);
• …yearsofworkexperienceatcurrentschoolis10years(15yearsinBauchiandfiveyearsinNiger);and
• …yearsofworkexperienceasateacheris14years(17yearsinBauchi,10yearsinNiger).
• …only3%arefemale(0%inBauchi,7%inNiger);
• …23%holdanNCEqualification(27%inBauchi,20%inNiger)and40%passedtheSSCE(23%inBauchi,57%inNiger);56%inBauchihaveareligiouseducationqualification,comparedto20%inNiger;and
• …43%attendedtrainingduringthelasttwoyears(37%inBauchi,50%inNiger).
Source:GEP3Baselinesurvey2015
Theappointmentofaheadteacherisnotnecessarilybasedonabilityandqualificationsbuthastodo
withsocialstatuswithinthecommunityandperceptionsaroundleadershipmoregenerally.InthecasestudyIQSs,theheadteacherroleismainlyconsidereda‘schoolleader’role–havingthesocialstatustosuperviseteachersandprovideadvice,andinfluencethecommunitywithregardstointegrationandgirls’education.Assuch,Mallamsmaybeassignedthepositionbasedonperceivedpoliticalandsocialstatus,ratherthanpedagogicalteachingknowledge.However,giventheiroftenlowacademicandprofessionalqualificationsitisquestionablewhethertheheadteachershavetheabilitytoprovidepedagogicalsupervisionandmentoring.Itwilllikelybeachallengetopinpointwhototrain,andforwhatleadershiprole,sincetheassignedheadteachersmaynotbethemostqualified,intermsofformaleducation,totakeupapedagogicalleadershiprole,buttheymayhavethesocialstatustoprovidemoralleadership.
MoreheadteachershavebeentrainedinNiger,whilemoreexternalmonitoringishappeninginBauchi.AlargershareofheadteachersinNiger(50%)havereceivedsomesortoftraininginthelasttwoyearsthaninBauchi(37%).Headteacherswithhigherprofessionaloracademicqualificationsweremorelikelytobe
39Wedefineheadteachersasthepersonswhoareresponsibleforday-to-daymanagementandsupervisionofthenon-religiousteaching.
©EDOREN 58
trained,andthisdifferenceislargerthantheBauchi/Nigerdifference.ThoughbothstatesreportedtrainingbeingmostlyorganisedbyGEPandUNICEF,moreheadteachersinBauchireportedtrainingbySUBEBortheLGEA,whichsuggestsgreaterinteractionwithgovernmentbodies.ThisisalsosupportedbythefindingthatagreatershareofschoolsinBauchireportedreceivinganymonitoringvisitsfromagovernmentbodyduringthelastterm.ThismaybepartlyexplainedbythefactthatmoreIQSsintheBauchisamplearelocatedinurbanareas.Ingeneral,though,only44%and33%ofIQSsinBauchiandNiger,respectively,reporthavingreceivedanymonitoringvisitfromgovernmentofficials,non-governmentalorganisations(NGOs)orexternalagenciesduringthelastschoolyear.
Headteachersdemonstratesomepedagogicalleadership,particularlyinNiger,andtheytakeactionsto
improvepupilandteacherattendance,butrecord-keepingispoor.ThereisamarkeddifferencebetweenNigerandBauchiintermsofheadteachersobservinglessonsoffacilitators.InBauchifewheadteachersreportedhavingsatinonanylessonstaughtbyfacilitatorsfortheentiredurationofthelesson,whilemorethanhalfofheadteachersinNigerreportedhavingdonesoduringthelastterm(seeFigure12).TherewerefewerdifferencesbetweenBauchiandNigerforotherpedagogicalleadershipandmanagementactions,suchasmeetingfacilitatorstodiscusstheirteachingoractionstoimproveteacherandpupilattendance,althoughheadteachersinNigerseemtobemeetingmorefrequentlywiththefacilitators,individuallyoringroups.Interestingly,trainedheadteachersweremorelikelytoobservelessons,haveobservationrecordsormeetteachers.Record-keepingbyheadteacherswasgenerallyfoundtobepoor,withonly30%ofheadsabletopresentP2pupilenrolmentorattendancerecords,andfarfewerinstancesofupdatedattendancerecords(10%).P2pupilattendancerecord-keepingwasmarkedlybetterinBauchi.
Figure12: HeadteacherpedagogicalleadershipandmanagementactionsinNigerand
Bauchi
5.3.2 ContributionClaim2:GEP3’sIQSScontributestoanimproved,girl-friendlyschool
environmentinIQSs
GEP3’sIQSSseekstocontributetoanimproved,girl-friendlyschoolenvironmentbystrengtheningschoolmanagement,promotingthemobilisationofmoreresourcesandincreasingtheinvolvementofthecommunityintheIQS.ThemaingatewaytoachievetheseintermediaryoutcomesistheempowermentofCBMCs,bytrainingthemandprovidingthemwithmini-grants.Inaddition,headteachertrainingisassumedtocontributetobetterschoolmanagementaswellasimprovedgovernmentmonitoringand
0 10 20 30 40 50 60 70 80 90 100
HTobservedalesson
HTtookactiontoimproveteacherattendance
HTtookactiontoimprovepupilattendance
HTwhomeetteachersindividually
HTwhomeetteachersingroup
PercentageofheadteachersNiger Bauchi
Source:GEP3Baselinesurvey
©EDOREN 59
supportsupervision.Inthissectionwefirstdiscussthebaselinefindingsinrespectofschoolmanagement,withspecificattentiongiventotheroleoftheCBMC,followedbyfindingsonresourcemobilisation,communityinvolvement,theschoolenvironmentandgovernmentmonitoringandsupport,drawingfrombothquantitativeandqualitativedata.
Summaryofevidencerelatedtoevaluationquestions
Q4:HowwellhasGEP3’sCBMC
capacitydevelopment
contributedtoimprovedschool
managementandincreased
mobilisationofresourcesfor
schoolinvestment?
CBMCsareactiveandfunctionaltovaryingdegrees.Thereisscopeforimprovementintheirrepresentativeness,theirschooldevelopmentplanningandtheirfinancialmanagement.CBMCsareabletomobiliseresourcesbutsignificantresourcegapsremain.
CBMCsexistin87%oftheIQSSschools.MostoftheestablishedCBMCsappeartobeactive,asindicatedbyregularmeetingsandmonitoringactivities.63%ofCBMCshadalreadyreceivedtrainingduringthelasttwoyears,althoughclearareasforimprovementinCBMCfunctionalityremain,including:childrepresentation,attendanceratesatmeetings(particularlyfemale),schooldevelopmentplanningandfinancialmanagement.
CBMCsareabletosuccessfullymobiliseresourcesfromthecommunitiesbutsignificantresourcegapsremain.TheperceptionofIQSstakeholdersislessthatparentsareunwillingtocontributefunds,butmorethatparentshavelimitedfundstogiveduetopoverty,whichisoneofthekeyreasonswhyparentssendtheirchildrentoIQS.CBMCtrainingonresourcemobilisationshouldconsiderthemobilisationofalternativeresources,giventhatthereappearstobelimitedscopetoraiseresourcesfromparents.
CommunitysupportofIQSsandCBMCsisevidencedbycommunitymembershipintheCBMC,andbycommunitymembersdonatingmoneyandotherresourcesforschoolimprovementactivities.HeadteachersandproprietorsareusuallypartoftheCBMC,andoftenchairtheCBMC.ThistestifiestotheiracceptanceoftheCBMC’sinvolvementinschoolmanagement.SinceIQSschoolmanagementisoftenseenascollectiveinthecommunity,GEP3willneedtoconsiderhowtoensurethatallrelevantstakeholdersareincluded,andthatconflictsofinterestareavoidedwhenproprietors/headteachershaveoverlappingroles.CBMCmembersseetheirrolesasbroad,coveringarangeofresponsibilities,butfeelthattheyhaveweakcapacityandlimitedresourcestofulfilthem.
Q5:HowwellhaveCBMCsbeen
abletoadequatelymanagemini-
grantsandinvesttheseresources
intheimprovementofagirl-
friendlyschoolenvironment?
IQSsareabletoinvestintheschoolenvironmentanddoconsidermakinggirl-friendlyinvestments,butthefinancialresourcestheyhaveaccesstoaregenerallysmall.Additionaltrainingandsupervisioninfinancialmanagementareneeded.
IQSsareuniversallydeficientin‘soft’and‘hard’infrastructure,andprovideapoorschoolenvironmentfortheattendingchildren,particularlygirls.IQSshavesevereshortagesofhygienefacilitiesforgirlsandofferfewspaceswheregirlsandboyscanparticipateinshapingeducationorvoicetheirneeds.CBMCsreportedutilisinganaverageof80%ofthetotalfundsraisedinthepastyearsonschoolimprovements.ThequalitativecasestudyfindingsindicatethatCBMCshaveconsideredtheattendanceandretentionofgirlswhenallocatingresources.However,almost50%ofCBMCsreportednothavingaccesstoanyfundsduringthelastyearandthemedianamountraisedislimited(NigerianNaira(NGN)20,000).Financialmanagementpracticesarepoorandnecessitatefurthertraining.GovernmentmonitoringandsupervisionislimitedandconsideredunreliablebyIQSstakeholders.
©EDOREN 60
SchoolmanagementandtheroleoftheCBMC
Schoolmanagementisseenascollective,withoutclearlydefinedanddelegatedroles.Aswasdiscussedintheprevioussection,schoolleadershipinIQSsiscomplex,whichisreflectedinschoolmanagement.Whilethequantitativesurveyshowsthat90%oftheIQSshaveasoleproprietor,thequalitativecasestudiessuggestthatthemanagementoftheIQSisnotanindividualresponsibilitywithclearlydefinedanddelegatedresponsibilities.Thiscreatesconfusionaroundwhoisinchargeofwhat.Severalactorsareperceived—andperceivethemselves—tobepartofthesuccessfulrunningoftheschool,inmanycasesmeaningthatmanagementbecomescollective.Theseactorsincludenotonlythosewhoareactivelyinvolvedwiththeschool,butalsostakeholderssuchascommunityleadersandimamswhooftenhaveinfluenceovertheschool’svisionandthecommunity’sacceptanceofintegration.Suchperceivedcollectiveresponsibilityovertheschoolprovidesanopportunityforacommunity-basedmanagementstructureliketheCBMC,whereresponsibilitiescanbeheldcollectively.TheimportanceofexistingsocialstructuresandnormswithintheIQScommunitiesmeanthatGEP3willneedtoconsiderhowtoensurethatallrelevantstakeholdersareincluded,andfortrainingtoespeciallytargetthosewhoholddecision-makingauthoritywithincommunities.
MostCBMCsareestablishedandactive,andalargesharehavereceivedtraining.TheCBMCcanonlycontributetotheimprovementoftheschoolenvironmentundertheassumptionthatitisactuallyestablishedandactive.In13%oftheIQSssurveyedtheCBMCwasnotestablished.Thiswasthecasein20%ofIQSsinBauchi,versus7%inNiger.WhilemostCBMCsappeartobeactive(asindicatedbyhavingmetduringthepastschoolyear),theaverageattendancerateatthelastmeetingwaslessthanhalfofthemembers(seeTable7).WhileCBMCsinBauchimetalmosttwiceasoftenasthoseinNiger,theattendanceratewaslower.Approximately60%ofCBMCshadreceivedsomesortoftrainingduringthelasttwoyears.
CBMCshaveconsiderablecommunityrepresentation,althoughwomenareaminorityandchildrenare
hardlyincluded.ParentsmakeupmostofthemembersinatypicalCBMC,althoughinthemajorityoftheIQSssurveyedtheCBMCisnotchairedbyaparentbutbytheproprietororheadteacher(seeTable7).InallIQSsincludedinthequalitativestudy,theMallam—oftenalsobeingtheproprietor—holdsakeypositionintheCBMC,suchaschairperson.ThereisthusaquestionaroundpotentialconflictsofinterestinregardtotheCBMC’smonitoringroleandabilitytoholdschoolleadershiptoaccount.Womenformaminorityamongthemembers,although87%ofallCBMCshadatleastonefemalemember.Theaveragefemaleattendancerateatthelastmeetingwas35%,whichisbelowtheattendancerateforallmembers.Almostnochildren(definedasanyoneundertheageof15years)areincludedinCBMCmembership.ThisindicatesthatatpresentCBMCsdonotprovideaforumforgirls’voicesandneedstobeheard,andforgirlstoinfluenceinvestmentsintheschoolandbroaderdecision-makingrelatedtoschoolmanagement.InterviewedCBMCmembersmostlyconsiderwomentobeanimportantresourceoftheCBMCastheyhavethepotentialtoconvincemotherstosendtheirchildrentoschool.However,genderrolesandresponsibilitiesunderminetheextentofwomen’sinvolvementinCBMCs.
Table7: SummaryofCBMCcharacteristics
ForsurveyedCBMCstheaverage… OfCBMCs…
• …yearssinceCBMCestablishmentisthreeyearsago;and
• …membershipis14members(16inBauchi,13inNiger),amongwhich:fourwomen,nochildren,10parentsandthreefacilitators;and
• …attendancerateatlastCBMCmeetingwas41%(36%inBauchi,46%inNiger).*
• …37%wereestablishedbeforeintegration,ofwhich70%wereestablishedtheyearbeforeintegration;and
• …55%havetheproprietororheadteacheraschair(66%inBauchi,45%inNiger);
• …90%hadameetingduringthelastschoolyear,withfourmeetingsonaverageperyear(sixinBauchi,threeinNiger);and
©EDOREN 61
• …63%receivedtraining(54%inBauchi,71%inNiger).
*AttendancerateisestimatedforthoseCBMCsthatkeptmeetingrecords,whichwastwo-thirdsofCBMCs.
Source:GEP3Baselinesurvey2015
CBMCmembersseetheirrolesasbroadandcoveringarangeofresponsibilities.CBMCsareactivein
termsofmonitoringbutthereisscopeforimprovementinschoolplanningandfinancialmanagement
practices.ThequalitativeresearchfoundthatCBMCmembersgenerallyunderstandtheirrolesandresponsibilitiesbuthaveweakcapacityandlimitedresourcestoactonthem.CBMCsseetheirroleasbroadandperceivetheirmainresponsibilitiestobeincreasingtheawarenessoftheimportanceofeducationforbothboysandgirls,identifyingwhychildrenarenotenrolledandlearning,andto‘solve’theseproblems.Whiletheyalsoperceivepromotingfacilitatorattendancetobepartoftheirresponsibilities,monitoringsometimesbecomessimplysymbolicasCBMCmembersmonitorfacilitatorperformancewithoutpriorteachingexperienceandwithalackofguidance.ThequantitativedataindicatethatCBMCsareactiveintermsofmonitoring.AlmostallCBMCchairsindicatedhavingvisitedtheschooltomonitoritsperformanceduringthelastterm,withtheaveragenumberofreportedvisitsbeingseven.Over80%ofCBMCmembersreportedlymonitoredpupilandteacherattendanceandtookactiontoimprovepupilandteacherattendanceduringthelastterm.Areasforimprovementareschoolplanningandfinancialmanagement.Only20%ofCBMCssurveyedhadcompletedaWCDP,ofwhichtwo-thirdscouldprovideevidenceabouttheWCDP.Intermsoffinancialmanagement,only27%ofCBMCsreportedusingacashoraccountbook,ofwhichhalfcouldpresentanupdatedcashbook.70%ofCBMCsdidreportusingabankaccounttostorefunds,butonly40%indicatedthattheykeptrecordsofbankdepositsandwithdrawals.
Resourcemobilisation
CBMCsareabletosuccessfullymobiliseresourcesfromthecommunities.ThemajorityofCBMCssurveyedundertaketheresponsibilityofmobilisingresources(65%ofCBMCsreportedcashmobilisingeffortsinthepreviousyear)andtheyarereasonablysuccessfulindoingso(over80%ofthoseundertakingtheseactivitiesreporteddoingsosuccessfully).Inallinstances,mostoftheresourcesweremobilisedfromthecommunity,withasmallsharereceivedfromthegovernment,NGOsandotherexternalagencies.40%ofCBMCsinBauchiand30%inNigerreportedhavingreceivedaGEPschoolgrantduringthelastschoolyear.Inaddition,about40%oftheCBMCsalsomobilisednon-cashresources,whichcameexclusivelyfromthecommunity.
SignificantresourcegapsexistinIQSsandCBMCs’perceivedroleisunderminedbyalackoffunding.ThemajorityofCBMCsurveyedhadmadeanefforttomobilisefundsduringthepreviousyear.Nevertheless,almosthalfreportedthattheydidnothaveaccesstoanyfundsduringthelastyeareitherbecausetheyhadnottriedtomobilisefundsorbecausetheireffortshadbeenunsuccessful,andbecausetheyhadnotreceivedanyfundsfromothersources.Also,themedianamountraisedwasonlyNGN20,000,whichissmallcomparedtothevalueoftheschoolgrants,whichmostlyamountedtoNGN120,000orNGN150,000.ThequalitativeresearchhighlightshowCBMCsperceivealackoffundstobehinderingtheirabilityto‘solve’theproblems.Atthesametime,CBMCsacknowledgethatparentshavelimitedcapacitytocontributefunds.Theperceptionislessthatparentsareunwillingtocontributefunds,butmorethatparentshavelimitedfundstogive.ThisisunderlinedbythefactthatoneofthekeyreasonswhyparentssendtheirchildrentoIQSsappearstobeduetopoverty,andaninabilitytoaffordtosendchildrentothepublicprimaryschool.AsIQSsareprivatelyrun,eitherbyaproprietororbyasmallcommunity,theyrelymoreonresourcemobilisationthanpublicschools.Byrelyingoncontributionsfromcommunitymembers,whomaynothavefundstogive,thereisariskthattheworkofCBMCscouldbediminishedas‘externalresources’runout.CBMCtrainingthatfocusesonbuildingcapacitytomobilisecommunityresourcesmaycreateunrealisticexpectations.Alternativeresourceswillneedtobeconsidered,althoughCBMCs’capacitytoadvocateforgovernmentresourcesmaybelimitedastheIQSsectorisalreadyunder-funded—asevidencedbythelackoffacilitatorremuneration—andCBMCs(unlikeSBMCs)donothaveLGA-levelandstate-levelrepresentativestovoicetheirconcerns.
©EDOREN 62
Communityinvolvement
ThereisgoodcommunitysupportforIQSs.Asmentionedabove,thereappearstobeconsiderablecommunitysupportforintegratedschools.Thisisreflectedinmanyways,suchascommunitymembersbeingpartoftheCBMCanddonatingresources.Supportbythecommunityisnotlimitedtofinancialcontributionsalonebutalsoencompassesmoralsupportandtrust,aswellasCBMCmembersbeinginvolvedinschoolmonitoring.ThequalitativecasestudiesindicatethatgiventheIQSs’embeddednesswithinthecommunity’ssocialstructurestheactivesupportofcommunityandreligiousleadersseemsvitalforsuccessfulintegrationandaidstremendouslyintermsofgainingparents’support.
However,morecommunityengagementwithCBMCsdoesnotalwaysmeancommunitydecision-making
power.Althoughparentsmaybeinvolvedindecisions,andactivelyengagewithdiscussions,thatdoesnotnecessarilymeanthattheyhavedecision-makingpower.SincetheMallamoftenhasahighstatus,throughbeingtheproprietor,theheadteacher,aCBMCmemberandareligiousauthority,hisdecision-makingauthoritytendstobestrong.Moreover,theinvolvementofmenandwomendifferssignificantlyintheIQScontext.MenmakeupthemajorityofCBMCmembers.Whenwomenarememberstheymaynotattend,asillustratedbysomeofthecasestudyschoolsandtherelativelylowerCBMCattendancerate.
RolesandresponsibilitiesmaybewellunderstoodbyCBMCmembers,buttheymaynotalwayshavethe
capacitytoenactthem.Thiscouldpartlyexplainwhyincreasedcommunityinvolvementdoesnotalwayscorrelatewithimprovedactivitieswithinschools.WhilstCBMCsgenerallyfeelthattheyaredoingagoodjob,theybelievethatiffacedwithfewercapacityconstraints—mainlyidentifiedasalackoffundstoimproveinfrastructureandtopayfacilitators—theycoulddobetter.WhilstCBMCmembersshowawillingnesstoadoptnewskillsandknowledge,theyfacenumerouspracticalchallengeswhenitcomestomobilisingresourceseffectively.Contextualconstraints—suchassocioeconomicfactors,theirownqualifications,andthechallengesofnavigatingengrainedsocialnorms—needtobetakenintoaccountwhendesigningcapacitybuildinginresourcemobilisationforCBMCs.ThecontextinwhichCBMCsoperatealsoneedstobetakenintoaccountwhenevaluatingtheirefficiencyandeffectiveness:thecaseofCBMCscannotbecomparedeasilytoSBMCsinpublicschools.Moreover,CBMCstendtobehighlyembeddedintheschoolstructure,dealingwithaspectsofreligiouseducationandgeneralcommunitywellbeing,inadditiontoaspectsofintegration.Asaresultofthisbroaderremit,CBMCswilllikelybeengagednotonlyindiscussingandmanagingIQSmatters,butalsodiscussingandmanagingablendofbothQur’anicandsecularlearning,aswellascommunityissuesexternaltoeducationmatters.
Theschoolenvironment
IQSsareuniversallydeficientin‘soft’and‘hard’infrastructure,andprovideapoorschoolenvironmentto
theattendingchildren.Intermsof‘soft’infrastructure,only3%ofIQSssurveyedhavealibraryandonlyathirdhaveaplayground.Only3%oftheschoolshavesomeformofmothergroup,teacher–studentassociationorpupilgroupwherestudentscancometogethertoshareconcerns.ThequalitativecasestudiesindicatethatsomecorporalpunishmentisusedinallsixIQSvisited,whichpupilsperceivetobethemainthingtheydislikeaboutschool,andwhichdetersthemfromattendingschool.However,thisappearstobealsothecaseinpublicprimaryschools,whichpupilsconsidertobelessofasafespace.Physicalinfrastructureislackinginthemajorityofschoolsinbothstates—withmoreschoolsinBauchiinneedofrepairs,andchildrennothavingaccesstofunctionaltoiletsanddrinkingwateratschool.90%oftheschoolsarereportedasbeing‘inneedofmajorrepairs’.Only40%oftheschoolshaveanelectricityconnectionandamongthoseconnectedtothegrid,only3%actuallyhadelectricityonthedayofthesurvey.Only3%oftheschoolshaveaccesstoawatersource,20%oftheIQSsdonothaveaphysicalclassroomstructureforthestudents,andonly25%oftheschoolshaveafunctioningtoiletfortheirpupils.Bothfacilitatorsandpupilsacknowledgediscomfortwiththepoorschoolconditions.
MostIQSsdonotcurrentlyofferagirl-friendlyschoolenvironment.IQSshavesevereshortagesofhygienefacilitiesforgirls—amongthelessthan25%schoolswithafunctioningtoiletforpupils,only8%have
©EDOREN 63
functioningtoiletsforgirls—whichcanleadtogirlsreturninghometousefacilitates,andsubsequentlymissingclasses.Whereschoolshaveplaygrounds,girlscouldbeseentobeusingthemduringobservations,butthequalitativecasestudiessuggestthatboyshavemorefreedomtoplayandgirlsexpressadesiretobeabletoplaylikeboys.Asmentionedabove,only3%schoolshaveanyinteractivegroupswherepupilscancometogethertoshareconcerns.NeithergirlsnorboysarerepresentedinCBMCsandthereisverylowfemalerepresentationamongfacilitators,headteachersandproprietorsofIQSs.WhilewomenaremostlyrepresentedontheCBMC,theyrepresentaminorityandattendlessthanmen.Thepresenceofastrongfemalerolemodelcanalterperceptionsandself-confidenceamongthegirlsandtheirmothers,asshowninoneofthecasestudyIQSsledbyafemaleMallama.However,thepresenceoffemalefacilitatorsdoesnotnecessarilymeanthatgirlswilldisplayahigherlevelofconfidence.Themostimportantfactorinimprovinggirls’agencyappearstobeapositiveattitudeonthepartofthefacilitatortowardsgirls,andwhetherhe/sheisempowered,ratherthanthesexofthefacilitatorhim/herself.Asmentionedabove,gender-biasedclassroompracticesandattitudesprevail,whichcanaffectgirls’learningopportunitiesandself-confidence,asreflectedintheirperceivedshynessinclass.However,thereappearstobeawarenessthatsuchinstitutionalculturesmayaffectgirls’education.andthereappearstobeawillingnessamongschoolleaderstoreflectonandconsidergenderequityinschoolplanning.Furthermore,itappearsthatgirlsarenolesslikelytobeattendinganIQScomparedtoboys.Thiswasindicatedduringthemanualcountingofpupilspresentonthedayofthesurvey,aswellasduringthelessonobservations,thoughpoorenrolmentandattendancedatamakesitdifficulttomakecomparisonsacrossallthesampledschools.
Mobilisedresourcesareinvestedintheschoolandgirl-friendlyinvestmentisconsidered,butamounts
appearinsubstantialgiventhepoorconditionsoftheschoolenvironment.CBMCsreportedutilisinganaverageof80%ofthetotalfundsraisedinthepastyearsonschoolimprovements,althoughinthemajorityofcasesnoevidencecouldbeprovidedoftheinvestment.Fundsaremostlyinvestedinconstructionworks,suchastheconstructionofnewbuildings,renovationofbuildingsorconstructionoftoiletsandwaterfacilities.Secondinorderistheprocurementofteachingandlearningmaterials.Itisinterestingtonotethatdespitefacilitatorsreceivinglittleremunerationand,often,fewfacilitatorsbeingavailable,fundsarerarelyinvestedinfacilitatorrecruitmentorremuneration.ThequalitativecasestudyfindingsindicatethatCBMCshaveconsideredtheattendanceandretentionofgirlswhenallocatingresources,withexamplesofinvestinginschooluniforms(hijabs),toiletsandwaterfacilities.However,itisnotcleartowhatextentCBMCsareawareoftheirroleinraisingadditionalresourcesforgirls’education,orensuringthatGEP3resourcesareusedforschoolinfrastructurethattargetsgirls’educationspecifically.Ingeneral,theamountofmobilisedresourcesappearstobeinsubstantialgiventhepoorconditionsoftheIQSschoolenvironment.
Governmentmonitoringandinteraction
ThereisavariationintheextentofinteractionbyIQSswithgovernmentbodies,althoughingeneralitis
limited.Respondentsinthequalitativestudyperceivegovernmentauthoritiesand‘externals’tobethemainactorsresponsibleforprovidingfurthersupporttotheIQSs.However,IQSsarereportedlyrarelyvisitedbyLGEAofficialsor‘government’,andthiswasconfirmedbyLGEAofficers.BauchiIQSappearstohavegreaterinteraction.AlargershareofsurveyedIQSsinBauchi(44%)reportedmonitoringvisitsduringthelastterm,comparedtoschoolsinNiger,whereonly33%ofIQSsreportedmonitoringvisits.ThisdoesnotseemtobeassociatedwiththefactthatBauchicountsmoreurbanIQSsbecauseurbanIQSsinthesampleareaslikelytohavereceivedamonitoringvisitasruralIQSs.Also,moreheadteachersinBauchireportedtrainingconductedbySUBEBandtheLGEA.ThisholdstruefortrainingreceivedbyCBMCmembersaswell.ThetwohigherperformingcasestudyIQSsperceivegovernmentofficialstobemorepresent.
Governmentofficialscitechallengesinreachingalltheschoolsonaregularbasis,whichundermines
theircontributiontoefficientlymonitoringandsupportingtheIQSs,andnegativelyaffectscommunities’
perceptionsaboutgovernmentsupport.Officialsperceivethechallengeofattemptingtovisit30schools
©EDOREN 64
andmoreonaregularbasistobeunmanageable,giventhedistancesconcerned,thelackoftransportationinmanycases,andtheirworkloads.Whenvisitsdooccur,localgovernmentofficers’engagementwithcommunitiesoccursthroughtheheadteacher/Mallam/proprietor,andtheyseetheirroleasprovidingsupervisionandadviceonteachingandmanagementpractices,andofferinginstructionandcorrectionbasedontheirobservations.Communitiesgenerallyviewsupportfromthegovernmentnegatively,consideringgovernmentactorstobeunreliable.SinceIQSslooktogovernmentforfinancialsupportorsupportoffacilitators,thelackofgovernmentinteractionandsupportmayhaveimplicationsforcommunityattitudestowardsintegration.
5.3.3 ContributionClaim3:Moreeffectiveteachingofformalsubjectsandanimproved,girl-
friendlyschoolenvironmentcontributetoimprovedlearninglevels,particularlyamong
girls
Thissectionfocusesonthebaselinedataofpupils’learningoutcomesinIQSs,themainfinaloutcomevariableintheToCofGEP3’sIQSS.TheIQSSinterventionisexpectedtoimprovelearningoutcomesinbasicliteracyandnumeracy,especiallyforgirls.Itisalsoexpectedtoimproveretentionand,toalesserextent,enrolmentofgirls.Tostartwith,weprovidebackgroundinformationonpupilcharacteristics,attitudesandschoolparticipation,whichcorrespondswithsomeoftheassumptionsunderlyingContributionClaim3.Next,wepresentthebaselinelearninglevelsofpupilsinHausaandEnglishliteracyandnumeracy.Finally,wediscussqualitativefindingsaboutfactorsthatinfluenceretention,asecondaryoutcomeintheToC.
Summaryofevidencerelatedtoevaluationquestions
Q7:Towhatextenthavepupil
literacyandnumeracyskills,
especiallyofgirls,improvedin
GEP3-supportedIQSs?Howhas
GEP3contributedtosuch
improvement?
TowhatextentdoestheIQSS
interventioncontributeto
reducingthegapbetween
learningoutcomesandexpected
learningoutcomes,asexpressed
inthecurriculum?
Pupilperformanceisverylow,particularlyinliteracy,whichmaybeassociatedwithfacilitators’lowteachingcompetenciesandadeficientschoolenvironment.Barrierswithregardstopupils’continuedparticipationintheeducationprocess,attitudestowardsgirls’education,andlanguagediversityarelikelytoalsoaffectlearning.IQSmobilityandpupilsattendingotherschoolsrequirefurtherinvestigation.
Pupils’literacylevelsareverylowinbothHausaandEnglish.ThevastmajorityofGrade2pupilsassesseddisplayedliteracylevelsassociatedwithpre-schoolpupils.Furthermore,mostofthesepupilsarealongwayfromthecut-offassociatedwithemergingliteracyskills(thoseassociatedwiththeP1curriculum).Numeracyoutcomesarebetter,with69%ofchildrendisplayingemergingnumeracyskills.However,asmallminority(11%)hadP2grade-levelnumeracyskills.Theverylowleveloflearningisgenerallyconsistentacrossgroups,althoughtheperformancegapbetweengirlsandboyswidensatthestageinthelifecycletypicallyassociatedwithpuberty.
GirlsandboysappeartobeattendingIQSsequally.However,bothfacechallengesinregardtoattendingandremainingfullyengagedwiththelearningprocessbecauseofout-of-schoolresponsibilities,whicharelinkedtohouseholdpovertyandtheneedforchildrentosupportthefamily’slivelihoodactivities.
AlthoughpupilslargelyreportedspeakingHausaathome,andclassroominstructionappearstobetakingplaceinHausaaswell,teachingisnotalwaystakingplaceinthemothertongueofthepupils,inparticularinNigerwherealmostofhalfofpupilsreportspeakingNupeathome.
TotheextentthatIQSsaremobile,girls’learningislikelytobeaffected.IQSmobilityneedstobeverifiedatmidlineandmonitoredregularly,withspecialattentiontoTsangayaIQSsinBauchi,whichhavetraditionallyhadanomadicculture.
AlargeshareofchildrenstudyinginIQSsattendanotherschoolaswell.Thismayaffectthetimethatpupilshaveavailabletoattendtheintegrated
©EDOREN 65
curriculumclasses.Itmayalsoconstituteanalternativeexplanationforchangesinlearningoutcomes.
Q8:Whataretheattitudesof,
andacceptanceby,community
membersandleaderstoward
integratingformalsubjectsin
Qur’aniceducation?Howhave
theseattitudesandhowhasthis
acceptancechangedduringthe
intervention?Why?Howhave
attitudeschangedinregardto
girlsreceivingformaleducation?
Thequalitativecasestudiesindicatethatintegrationhasgainedacceptancebutthatitneedstohappengradually,withoutaffectingreligiouseducation.Attitudestowardsgirls’educationareperceivedtobechangingbutbarriersremain.
Theintegratedcurriculumhaslargelygainedacceptanceinthecasestudycommunities.IntegratedQur’aniceducationnowseemstobeanintegralpartoftheschoolsinwhichitissituated,althoughthedepthofintegrationvariesamongIQSs.Communitiesperceivetheacceptanceofformaleducationtohaveincreased.Thisisowingtoacombinationof‘outsiderscomingandexplaining’(intheformofsensitisationcampaigns)andpeopleseeingthepositiveoutcomesofreceivingformaleducation.However,itisseenasimportantthatintegrationhappensgradually,andfortheIQSsnottotransformintopublicprimaryschools,whichcouldnegativelyaffectreligiousteaching.
Boysandgirlsviewthevalueofeducationfortheirfuturelifedifferently.However,IQSmanagementandfacilitatorssaythatgirls’interestinformalsubjectshasincreased.Barriersrelatedspecificallytoattitudestowardsgirls’educationpersist,althoughattitudestowardsgirls’educationareperceivedtobechanging,andparentsandcommunitiesdoappeartobewillingtoreflectonandreconsidertheroles,responsibilitiesandcapabilitiesofgirls.Earlymarriageofgirlsconstitutesanimportantbarriertocontinuededucationforgirlsovertheageof10.
Q9:Whatunintended
consequencesdoesIQSShavefor
teachers,headteachers,pupils
andproprietorswithintheIQSs,
aswellasforthebroaderschool
community?
TobeansweredafterIQSSintervention.
Pupilcharacteristics,attitudesandschoolparticipation
Hausaisthemostcommon,butnottheonly,languageoftheimmediateenvironment.AmajorityofthepupilsreportedspeakingHausaathome(75%),followedbyNupe(22%)andFulfude(2%),withnostudentreportingspeakingEnglishathome.AsdepictedinFigure13,thereareconsiderablecross-statedifferencesintermsoflanguagespokenathome.WhilealmostallpupilshaveHausaastheirmothertongueinBauchi,thisisthecaseforonly54%ofpupilsinNiger.InthisstateNupeisalsocommonlyspokenathome.
©EDOREN 66
Figure13: Languagespokenatpupils’homes,bystate
Teachingwithinschoolsisnotnecessarilytakingplaceinthemothertongueofthepupils.ClassroomobservationsandheadteacherinterviewsindicatethatHausaandEnglisharethetwomainlanguagesusedintheclassroom.Therefore,teachingattheIQSsisnotnecessarilytakingplaceinthepupils’mothertongue.InparticularinNiger,Nupeisunder-usedrelativetotheproportionofpupilswhospeakitathome.
BothboysandgirlsseemtoattendIQSsequally,andgirlsareonaverageolderthanboys.Thegenderratioappearsquitebalancedacrossbothstates,althoughduetothedifficultyofnothavingformalenrolmentrecords,thisfindingisnotrepresentativeofthesampleaswhole.40Schoolattendancedatacorroboratethisfinding:justasmanygirlsasboyswereattendingintegratedsubjectclassesinIQSsonthedayofthesurveyvisit.ThisfindingsupportstargetingIQSsasameansofeducatinggirls.Atanyparticularlevel,girlsareslightlyolderthanboys,whichsuggeststhattheyareenteringschoollateroraremorelikelytorepeatayear.
ThenumberofpupilsenrolledorattendingformalclassesvariesconsiderablyacrossIQSs.AmongIQSswithenrolmentrecords,enrolmentrangesfromaslittleas20pupilstoasmanyas400pupils,amajorityofwhichareattheP1–P3(earlygrade)level.Similarly,atthetimeofthelessonobservationsonefacilitatortaughtbetweenfourand147children,withamedianof40pupilsperfacilitator.
AlargeshareofchildrenstudyinginIQSsattendanotherschoolaswell.40%ofpupilsreportedcurrentlyattendinganotherschoolbesidetheIQS.Thisdidnotvarymuchbygender.However,thereweresignificantcross-statedifferences,with72%ofpupilsinBauchiascomparedtoonly13%pupilsinNigerreportingattendingotherschools.InBauchi,mostofthesechildrenwereattendingapublicprimaryschool(94%)ascomparedto55%inNiger.ThequalitativedatasupportthefindingofchildrenattendingIQSs,alongwithpublicschools.Lackoffundsisthekeyfactorwhychildren,andinparticulargirls,mayattendonlytheIQS,althoughthequalitativeresearchindicatesthatsomegirlsdonotattendpublicprimaryschoolsduetoparentsnot‘valuingeducationforgirls’.Incaseswhereparentsmaynotwishtosendgirlstothepublicprimaryschool(eitherforculturaloreconomicreasons)theIQSoffersanopportunityforgirlstoaccessformaleducation.However,thereissomeriskthatfollowingtheintroductionofsecularteachingattheIQS,parentsmayrefrainfromsendingtheirdaughterstopublicprimaryschoolsandsendthemtotheIQSonly,inordertosavemoney.Itisimportanttonotethefactthatpupilsareattendingotherschools,
40Only30%ofIQSssurveyedhadenrolmentrecords.
3%
93%
3% 0% 1%1%
54%
0%
43%
2%
Fulfulde Hausa Kanuri Nupe Others
Bauchi Niger
Source:GEP3Baselinesurvey
©EDOREN 67
particularlyinBauchi,becausethereissomelikelihoodofspillovereffectsofinterventionsatthelocalprimaryschoolsonoutcomesattheIQSs.Furthermore,this‘doubledipping’mayaffectthetimethatpupilshaveavailabletoattendtheintegratedcurriculumclasses.
Boysandgirlsviewthevalueofeducationfortheirfuturelifedifferently,althoughgirls’interestin
formalsubjectsisperceivedtohaveincreased.Inallschools,pupilsconsiderformalsubjectstobeimportantforbothboysandgirls.InseveralIQScommunities,stakeholders(includingboys)perceivegirlsasbeingtoofocusedonmarriage,andtheybelievethatthisaffectstheirattendanceandinterestinschool.However,IQSmanagementandfacilitatorsperceiveanincreasedinterestinformalsubjectsamonggirls.TheyattributethistotheactivitiesofCBMCsandfacilitatorsencouraginggirls.
Pupils’learningoutcomes
Pupils’literacylevelsareverylowinbothHausaandEnglish.Only2.5%ofthepupilsinBauchiandNigerwereabletocompletesomeofthetaskswithinthebasicHausaliteracyrange.Afurther1.8%ofpupilswereabletodemonstrateemergingliteracyskills,whilethevastmajorityofpupilswerefoundtohaveonlypre-literacyHausaskills.Figure14representsthedistributionofperformanceinHausaliteracy.TheaxisrepresentstheHausascalescorederivedfromthepsychometricanalysisofpupilperformanceontheassessment.Thepeakofthedistributionfallswellbelowthecut-offpointbetweenpre-literacyandemergingliteracy,indicatingthatmakinglargegainsinthepercentageofpupilsmovingfrompre-literacytoemergingliteracywillrequiresubstantialeffort.AsimilarpatternexistsforEnglishliteracy.Again,alargemajorityofpupilshaveyettoacquireanyknowledgeorskillsbeyondpre-literacy,while3.4%ofpupilswereabletodemonstratesomeoftheknowledgeandskillsthatfallwithintheemergingliteracyrangeand2.4%wereabletodemonstratesomeoftheskillsthatfallwithinthebasicliteracyrange.ThissuggeststhatpupilproficiencyinEnglishafterayearofschoolingisslightlyhigherthanpupilproficiencyinHausa.
Manychildrenareonthecuspofachievingemergingnumeracythoughgainsinthepercentageofpupils
achievingbasicnumeracywillbemorechallengingandwillrequiremoreeffort.AscanbeseenfromFigure15,thecuspofthedistributionfornumeracyachievementfallsjustabovethecut-offpointbetweenpre-numeracyandemergingnumeracy.Thisindicatesthatlargegainsinthepercentageofpupilsfallingwithintheemergingnumeracyrangecouldbeobservedasmanychildrenareonthecuspofachievingemergingnumeracy.Achievinggainsinthepercentageofpupilsachievingbasicnumeracywillbemorechallengingandrequiremoreeffort.
©EDOREN 68
Figure14: DistributionofHausaliteracyproficiency
©EDOREN 69
Figure15: Distributionofnumeracyproficiency
Theverylowleveloflearningisgenerallyconsistentacrossgroups,althoughtheperformancegap
betweengirlsandboyswidensatthestageinthelifecycletypicallyassociatedwithpuberty.AllagegroupstypicallyachievewithintheHausapre-literacyproficiencyrange,andtherearenosubstantialdifferencesbetweentheachievementofboysandgirlswithineachagelevel(withtheexceptionoffour-andfive-year-olds).ThistrendissomewhatdifferentforEnglishliteracyandnumeracy.ForEnglishliteracymeanscalescoresforallagecategoriesareagainbelowthecut-offpointbetweenpre-literacyandemergingliteracy.However,girls’performancedeclines,ascomparedtoboys,aroundpuberty(aged12orso).Similarly,thenumeracyperformancegapbetweenboysandgirlswidensaroundpuberty,withgirls’performancedecliningascomparedtotheirmalecounterparts.ItisalsonoteworthythatacrossHausa,Englishandnumeracyassessments,pupilsinBauchiperformedslightlybetterthanpupilsinNiger,whichmayberelatedtothegreaterlanguagediversityinNigerormorepupilsinBauchiattendingpublicprimaryschoolsatthesametimeasattendingtheIQS.
Barrierstoattendanceandretention
Continuedbarrierstogirls’educationalaccessandretentionexist–amongothersrelatedtoattitudes
towardsgirls’education(althoughattitudesareperceivedtobechanging).Thequalitativeresearchhighlightsthatgirls’attendanceandretentioninIQSscontinuestoremainachallenge,withpersistentbarriersrelatedspecificallytoattitudestowardsgirls’education.Lackofawarenessoftheimportanceofeducation,inparticularforgirls,waswidelycitedasareasonforlowattendance.However,stakeholdersperceivedattitudestowardsgirls’educationtobechanging,andparentsandcommunitiesdoappeartobewillingtoreflectonandreconsidertheroles,responsibilitiesandcapabilitiesofgirls.
Financialcapacityconstraintshaveacriticalbearingonattendanceandretention.Changingattitudesalone,saythroughCBMCactivities,maynotbesufficienttobringaboutchangeinbehaviouraspovertyisoftencitedasacriticalreasonwhyparentsdonotsendanyoftheirchildrentoanyschool,publicprimary
©EDOREN 70
orIQS.WhileIQSsmaynotchargeafeeorrequireinvestmentinschooluniforms,povertyremainsabarriertoaccessforsomechildren,owingtotheopportunitycostofschooling,whichislinkedtotherolethatchildren(andparticularlygirls)playinsupportingthefamily’slivelihoodactivities(seebelow).CBMCandschoolleadersinallsixcasestudyschools,forexample,feltthattheycouldnotinsistonpaymentfromparentsasthiswouldturnparentsaway.TheybelievethatresourceconstraintsofthecommunitylimitthecapacityofschoolmanagementtoinvestinIQSs,whichmay,inturn,affectthequalityofeducationprovidedandthecapacitytoretaingirls.
Out-of-schoolresponsibilitiesaffectchildren’sschoolattendance.Acrossthesixcasestudyschools,boysandgirlswereallresponsibleforincome-generatingactivities,orforhelpingwithhouseholdchores.Whilstschoolstakeholdersattributedthistoaperceivedlackofprioritisationofchildren’seducation,parentstendedtoviewthisasafinancialandpracticalnecessity.Mothersareperceivedtobelesspositiveaboutgirls’educationandmoreoftenhindertheirattendingschoolbecausegirls’schoolattendancemeansthemothercannotgenerateasmuchincomeasisneeded.Unmarrieddaughtersfulfilakeyroleintherunningofhome-basedbusinessesinnorthernNigeria(eitherbydirectlyengaginginincome-generatingactivities,suchashawking,ortakingcareofhouseholdchoressoastoallowthemotherstoengageinincome-generatingactivities),wherewomenarekeyincomegeneratorsforthefamily.ThegenderedpoliticaleconomyofthisrealityisonethattheToCofGEP3’sIQSSdoesnotaddresswhenitcomestoensuringthatgirlsareabletonotonlyaccessIQSs,butalsotoremainfullyengagedconsistentlyandlongenoughforlearningtooccur.
Oneofthemainreasonsstatedforlowenrolmentandretentionofgirlsisearlymarriage.Whilsttheacquisitionofformaleducation,includingforgirls,maybegainingacceptanceinthesixIQScasestudyschools,earlymarriageisstillprevalent.Thisisparticularlyproblematicforgirlsovertheageof10whoareacquiringabasiceducationforthefirsttime.Whileprovisionsareoftenmadeforgirlstocontinueeducationaftermarriage,thisisdependentonthewillingnessandattitudeofthehusbandinregardtoallowinghiswifetoattendschool.Thus,thoughattitudechangesbyparentsareimportantintheyearsbeforemarriage(andinnegotiatingamarriage),changingtheattitudesofmenlikelytomarrygirlsinthecommunityislikelytobeequallyessential.
ThenomadiccultureofsomeIQSscouldhavedirectimplicationsforlearningoutcomes,especiallyfor
girls.IfanIQSweretomove,therecouldbedirectimpactsonlearningoutcomesastheboysmovingwiththeMallammayresultinthemnotreceivinganyformaleducationformonths,unlesstheMallamwerealsoteachingformalsubjects.Forgirls,similarly,therewouldbeuncertaintyas,iftheydonotmovewiththeschool,theiraccesstoformaleducationwoulddependonwhetherformalsubjectteachingcontinuedintheabsenceoftheMallamandthemalepupils.Itisimportanttoexplorethisfurtheratmidline,and,aspartoftheimplementationofGEP3goingforward,itmaybenecessarytoconsiderhowthisinformationcanbeverifiedandtrackedaspartofthemonitoringprocessattheLGAlevel.
©EDOREN 71
6 Conclusionsandrecommendations
Thisreporthassynthesisedavarietyoffindingsfromthebaselinedatacollectionofthemulti-yearevaluationofGEP3.ItisaccompaniedbytheBaselineTechnicalReport.Thefindings—somedrawingattentiontotheweaknessesof,andotherssupportiveof,thekeyaspectsunderpinningGEP3’sToC;animpactevaluationofGEP3’searlylearningintervention;andaperformanceevaluationofGEP3’sIQSS—aimtocapturetheeducationsituation,answerspecificevaluationquestionsandtoprovideevidencetoinformtheToCandprojectimplementation.Herewereiterateanddiscusssomeofthekeyfindings,andofferrecommendationsforimprovementsthatreflecttheimplicationsofthebaselinefindingsforGEP3.
6.1 Conclusions
Keyconclusions:GEP3ToC
Theplausibilityofthere-designedGEP3ToCisbackedbyitspotentiallysynergetic,coherentlogicand
stakeholders’commonunderstandingofitsmainobjectivesandoverarchingstrategy.GEP3seekstoholisticallyimproveeducationaccess,qualityandgovernance,witheachoutputhavingthepotentialtostrengthentheachievementofeachotheroutput.Therearealsosynergiesacrossinterventions:inparticular,SBMCcapacitydevelopmentcanreinforceinterventionsrelatedtogirls’enrolment,girls’retentionandeffectiveteaching.StakeholderscommonlyrecognisethatGEP3isseekingtoimprovegirls’educationaloutcomes,particularlyincreasedgirls’enrolment–representedbyitstargetofgetting1millionadditionalgirlsintoschool.StakeholdershaverelativelygoodknowledgeofGEP3’soverallstrategyandacknowledgethatthere-designhasimprovedtheproject’sfocus.
However,criticalassumptionsremainhighlyuncertain.Criticalassumptionsrelatedtothemanagementandresourcingoftheeducationsystemarehighlyuncertain–inparticular,thereleaseofgovernmentfunding,schoolinvestmentandhumanresourcecapacityremainingonaparwithincreasedenrolment,effectivemonitoringattheschoolandinterventionlevel,andSBMC’sabilitytodeliveronitsmultipleprojectresponsibilities.Thefactthatsomeoftheseassumptionsaretargetedwithstrengthenedeffortaspartofthere-designenhancesplausibility.However,theeffectivenessofsucheffortsdependsonoperationalisationandeffectivedelivery,thescopeandstrategyofwhicharenotalwayscleartostakeholders,andwhicharesubjecttogovernmentcapacityandcontextualconstraints(suchassecurityconcerns).
SBMCsandMAshaveapivotalroletoplayinthesuccessofmanyinterventions.SBMC/CBMC/MAfunctionalityandcapacitytoundertakethemultiplerolesassignedaspartofGEP3interventionsisacriticalassumptionofGEP3’sToCandrequiresclosemonitoring.TheexpectationsplacedonSBMCsarehighandthecapacitybuildingandempowermentprocessisstartingfromalowbase,particularlyinIQSs.TheyfacegreaterchallengeswithinIQSs,astheirempowermentisbasedonalargernumberofassumptions(suchastheassumptionthattheyareactuallyestablished,inthefirstinstance,andtheassumptionthatthereisacceptanceofintegration).
Improvedcapacityofteacherstodelivereffectivelearningforgirlsisunderstoodtobevitaltoimproving
learningoutcomes,butstakeholderswerenotwellawareofthetrainingapproachandoutcomes.Teachingqualityisgenerallyconsideredtobelowandarangeoffactorsarelikelytoplayanimportantroleinimprovingit.Theseareoftennot–oronlyindirectly–underthecontroloftheproject,suchasteachers’initialcompetencylevel,theirdeploymentandtheirremuneration.IQSspresentgreaterchallenges,whichrequireattentionfromGEP3andneedtobetakenintoaccountinthedesignofinterventions.
TheplausibilityofGEP3’sscale-upapproachisdependentonuncertaingovernmentfundingandvaries
acrossinterventions.Scalingupinterventionsismoreplausiblewheretheirobjectivesarewellrecognisedbystatepartnersasbeingimportant,asupportinginstitutionalframeworkisinplaceandfundingsourcesarerelativelywelldefined.Plausibilityofscale-upisunderminedbythefactthatstakeholdersdidnothave
©EDOREN 72
aclearunderstandingabouthowscale-upwouldhappen.Thescale-upoftheIQSSisparticularlyuncertainbecausetheinstitutionalmandateoverIQSsisunclear,andduetotheSAMEs’limitedanduncertainaccesstoresources,thelimitednumberofwell-establishedIQSsavailableforscale-up,andsupply-sideconstraints—particularlywithregardstofacilitators.
Whilsttheprojectmakesaconcertedefforttoinvolveavarietyofstakeholderswithandthroughstateeducationpartners,theirinvolvementinoperationalplanningremainsunevenacrossinterventions.Governmentinvolvementremainsexternallydrivenandinfluencedbychangesinstaffingandotherexternalconditions.Nevertheless,theinvolvementofabroadrangeofstakeholdersisaperceived
strengthofGEP’simplementationstrategy.Identifyingandaddressingwheregovernmentcapacitybuildingneedsarethehighest,especiallyatthelocallevel,isessentialforGEP3’ssuccessfulimplementation.MonitoringcapacityisrightlyemphasisedinGEP3’sre-designasitisconsideredakeyoperationalprocessforproject,butitsoperationalisationhasyettoadvance.SeveralGEP3interventionsaredesignedtobespecificallyequityenhancing,empiricalverificationisrequiredastowhetherthemostvulnerablehouseholdsandgroupsareabletobenefitandactivelyparticipateinGEP3interventions.GEP3isnolongeroperatinginLGAswiththehighestgendergapinallstates.
Keyconclusions:Earlylearningintervention
TheevidenceonlearningoutcomespointstoverylowlevelsofHausaandEnglishliteracy.AlargemajorityofpupilshavenotacquiredtheknowledgeandskillsexpectedbytheNigeriancurriculuminHausaliteracy,norhavetheyacquiredeitheremergingorbasicEnglishliteracyskills.Mostpupilsscorewellbelowthecut-offpointbetweenpre-literacyandemergingliteracy,indicatingthatachievinglargegainsinthepercentageofpupilsmovingfrompre-literacytoemergingliteracywillrequiresubstantialeffort.KnowledgeofphonicsinparticularisfoundtobemostchallenginginregardtobothHausaandEnglishliteracy.Theearlylearninginterventionwillneedtotakethesecaveatsintoaccountwhenpromotingaphonicsapproachtoteaching.
Learningoutcomesaredifferentiatedbyage,genderandtypeofschool,witholdermalestudentsin
particularperformingbetterthanoldergirls.Regressionanalysisindicatesthatwhilstteachercharacteristicsdonotappeartobeassociatedwithlearningoutcomes,pupils’individualcharacteristicsandsocioeconomicbackground,aswellasschooltype,areassociatedwithpupilachievement.Thelackofassociationbetweenteacher-levelfactorsandlearningoutcomescouldreflectthefactthatcompetencylevelsareverylowacrossallteachers.Itislikelythatastrongercorrelationwouldbedetectedifthereweregreatervariationincompetencylevelsacrossteachers,andthattheirabilitytoinfluencepupils’learningoutcomeswouldbehigher.Thefindingthatwealthierchildrenmayperformbetterthantheirpoorercounterpartsisanindicationthatpovertycouldbeacriticalfactornegativelyinfluencingchildlearning.ThefindingthatstudentsinIQSsperformbetterthanthoseinpublicprimaryschoolsaftercontrollingforseveralfactorsisinterestingbutdifficulttoexplain.
IQSsandpublicprimaryschoolsoffergreatlydifferentcontexts.Teachercharacteristicsvaryacrossthetwoschooltypes,asdoschoolsize,schoolmanagement,externalschoolmonitoring,accesstoinfrastructure,andpupilcharacteristics.Teacherandheadteachercapacitydevelopmentwillneedtobeappropriateforthesedifferentcontexts.AconsiderablenumberofIQSsonlyhaveasinglefacilitatoroftheintegratedcurriculumsubjects,whichhasimplicationsforimplementingRANA’sschool-basedpeermentoring.
Teachersdemonstrateverylimitedknowledgeandskillsinthemajorityoftheareasrequiredtofunction
effectivelyasateacher.Almostnoneoftheteachersdisplayanypedagogicalknowledgeandskills,andmostappearunabletoascertainwhichactivitiesarebestsuitedtoimprovepupilperformanceorsupportgirlsintheirlearningwithinclassrooms.Over60%ofteachersareunabletodisplaycompetenceinGrade1and2-levelHausa.Hardlyanyteachersarecompetentinwritingandinterpretingwordsandphrases.ThelowHausaknowledgeissignificantforaninterventionthatfocusesonteachinginHausabecauseitislikelytonegativelyaffectteachers’abilitytopasstheirknowledgeontopupils.Thereareveryfewdifferencesinthelevelsofknowledgedemonstratedbydifferentgroupsofteachers.Forexample,nodifferenceswere
©EDOREN 73
notedbetweenteachersinpublicprimaryschoolsversusIQSs,orbetweenteachersthatattendedpreviousGEP/UNICEFtrainingandthosethatdidnot.
Hausaisusedextensivelyasthemainlanguageinteaching,buttheuseofHausateachingandlearning
materialsisveryrare.AllteachersreportspeakingHausaandamajorityreportspeakingEnglish.AllpupilsreportspeakingHausaathome.Hausawasusedinallclassesobservedduringthesurvey,whileEnglishwasalsoused,particularlyinpublicprimaryschools.ThisvalidatestheassumptionthatHausaisthelanguageoftheimmediateenvironmentforthepupilsandthemostcommonlanguageusedinclass.TheavailabilityanduseofHausamaterialsinclasswasrarelyobserved.TheemphasisonHausa-basedteachingandthedistributionofHausateachingandlearningmaterialsisthereforerelevant.AkeypointofattentionisthattheteachertrainingandteachingmaterialsneedtobecarefullyadaptedtothelowlevelofHausaliteracyskillsamongtheteachers.
Overall,teachersconsidertheirroletobeimportantandenjoyworkingasteachers,whilstbeing
consciousofthefactthattheyarelimitedintheirabilitytocontributetopupils’learning.Theevidencepointstoteachermotivationbeingrelativelyhomogenousacrossdifferentdisaggregations(rural/urban,schooltypeetc.).Highteacher-to-teacherinteractionscoressuggestarelativelyhighlevelofcollaborationamongteachers,whichcouldhelpthedevelopmentofspillovereffectswithinschoolsofanyteacher-specificinterventions.
Keyconclusions:IQSS
1. ContributionClaim1:GEP3’sIQSScontributestomoreeffectiveteachingofformalsubjectsinIQSs
GEP3’scontributiontomoreeffectiveteachingassumesthatIQSfacilitators’knowledgeandskillsandgender-sensitiveclasspracticescanbeimprovedthroughtrainingandmentoring.Baselineevidencesuggeststhatthereisaneedforfurtherexpansionanddeepeningoffacilitatortraining.Facilitatorsdemonstrateverylowlevelsofpedagogicalknowledge,somethingthatwillnotonlyhindereffectiveteachingbutthatmustalsobeborneinmindindesigningeffectivetrainingprogrammes.Subjectknowledgelevelsarealsolow,with66%offacilitatorsfailingtodisplaycompetenceinGrade1and2-levelHausa.GEP3alsoneedstotakeaccountofthefactthatfacilitatorsthemselves,aswellascommunitymembers,maynotperceivetheirlackofcompetencetobeacauseoflowpupilperformance.
Themodalitiesofthistrainingcanalsobeimproved.Forexample,itwasfoundthatappropriatefacilitatorsdonotalwaysgettrained.Knowledgeanddisseminationstrategiesatschoollevelalsoneedtobeconsidered,toreinforceimpact,althoughitalsoneedstobeconsideredthataquarterofIQSsonlyhaveonefacilitatorofnon-religioussubjects,whichwillconstrainamentoringprocessthatisbasedonschool-basedpeerinteraction.
Moreover,evidenceatbaselinesuggestsdeeplyingrainedgenderbiasesamongfacilitators,whoaremostlymale,towardsgirls’education.Andwhilstfacilitatorsdisplaygender-sensitivetechniques,theclassroomspacecontinuestoimposeagenderbias.Itis,therefore,importanttodelivergender-sensitivetechniquesintheclassroom,withthesupportofincreasinglytransformedgender-sensitiveattitudesamongstthefacilitatorsthemselves.
AnotherassumptionunderpinningGEP3’scontributiontoeffectiveteachingisthatmentoringoffacilitatorscanincreasetheirmotivation,leadingtomoreeffectiveteaching.Thisassumptionisborneouttosomeextent—thebaselinesurveyfoundthatpedagogicalleadershipinfluencesmotivation,asfacilitatorswhomeetheadteachersindividuallyreportedbeingmoremotivatedthanothers.
Facilitators’motivationisshapedbyissuesofremuneration,socialstatusandcommunityrolesandrelations.Motivation,anditstranslationintoteachingefforts,isinfluencedbytheverybasicandpracticalunderpinningofanIQSfacilitator’seconomicneedsandresponsibilities,andhow,intheabsenceofadequateremuneration(andthestatusthatthismakesmorelikely),thesemayconflictwithfacilitators’
©EDOREN 74
well-intentionedcommitmenttobeingengagedinteachingtheintegratedcurriculum.Thelackofremunerationislikelytoaffectfacilitators’motivationtoteach,andmakesitdifficulttoattractqualified
facilitatorsandholdthemaccountable.
IntegratededucationinIQSsappearstohavegainedacceptancebutitisimplementedtovarying
degrees.AsubstantialportionofIQSscontactedforthesurveywerefoundtobenon-integratedandfewIQSsprovideallcoresubjectsoftheintegratedcurriculum.Thismayberelatedtothefactthatintegrationhasoccurredrelativelyrecent,andthatonlyalimitednumberofhoursarededicatedtonon-religioussubjects.Also,facilitatorattendanceseemstobeflexible,withnon-religioussubjectsbeingtaughtasandwhenafacilitatorisavailable.Amorestructured,intensifiedteachingprogrammewillbehardtoenforcebecausebaselineresultssuggesttheneedforintegrationtobegradualandfortheschooltoretainitsQur’aniccharacter.Moreover,itisdifficulttoholdunremuneratedfacilitatorstoaccount.Nonetheless,thediverseIQScontextmayoffertheopportunitytoidentifyIQSsthatpositivelydeviateandcanactasrolemodelforothers.
GEP3assumesthatthedistributionofHausateachingandlearningmaterialscansupporteffectiveteaching.ThebaselinefoundthattheavailabilityofteachingandlearningmaterialsisverylowinmostIQSs,whichsuggeststhatthereisaneedforsuchmaterials.However,thesematerialswillneedtobe
carefullytailoredtotheverylowlevelsofskillsandknowledgeamongstbothfacilitatorsandpupils,in
orderforthemtoplayausefulrole.Furthermore,itisimportanttoconsiderthatforaconsiderableshareofpupilsinNigerNupeisthelanguageoftheimmediateenvironment.
Theappointmentofaheadteacherappearstodependonsocialstatusratherthanqualifications,which
raiseschallengesrelatedtowhototargetforheadteachertraining.Thischallengearisesbecausetheassignedheadsmaynotbethemostqualifiedtotakeupthepedagogicalleadership,buttheymayhavethesocialstatusandcloutneededtoundertakemoralleadershipduties.Pedagogicalleadershipseemstobesharedamongstmanystakeholders,withCBMCmembersguidingheadteachers,headteachersadvisingfacilitators,andproprietorsandcommunityleadersofferingtheirperspective.Alackofclarityaboutleadershiprolesmakesitdifficulttoidentifywhoseleadershipcapacitiestobuild.Furthermore,itmaybethatthereisnoclear‘candidate’forheadteacher/pedagogicalleader,asinsomecasesnooneinthecommunityhasanyformaleducationbeyondsecondaryschool.Headteachers’record-keepingabilitieswerenotablypoorandthiscouldbeanareatofocusongoingforward.
2. ContributionClaim2:GEP3’sIQSScontributestoanimproved,girl-friendlyschoolenvironmentin
IQSs
ThemainpathwaytoachieveincreasedcommunityinvolvementinIQSsandtomobilisegreaterresourcesisthestrengtheningofCBMCsandschoolmanagement.BaselineevidencesuggeststhatschoolmanagementinIQSsisseenasacollectiveeffort,withoutclearlydefinedanddelegatedroles.Itwill,therefore,beimportantfortheCBMCtotakeintoaccountandincorporatethevoicesofkeycommunityfigures,giventheinfluencetheymayhaveonthesuccessofintegration.TheCBMCcanonlycontributetoimprovementsifitisestablishedandfunctional,andthisisnotcurrentlythecaseforallGEP3IQSs.Itwillalsobeimportantforwomenandchildren’svoicestobeheardinCBMCs,toensuregirl-friendlyschoolinvestments.Thebaselinefindingspointtodeficienciesonthisfrontatpresent,partlybecausegendernormsseemtoaffectwomen’sparticipationinCBMCs.
OneofthecriticalrolesofCBMCsisinmobilisingfunds.WhileCBMCsareabletomobiliseresourcesfrom
thecommunity,significantresourcegapsremaininIQSs,andCBMCs’perceivedroleisundermineddue
toalackoffunding.Gapsinresourcemobilisationcanbetracedtoparents’inability,ratherthanunwillingness,tocontributefunds.TheselimitationsinthescopeforraisingresourcesfromthecommunityneedtobetakenintoaccountbyGEP3whendesigningitstrainingmoduleonresourcemobilisationforCBMCs.
©EDOREN 75
WhileCBMCtrainingcanbeausefultooltofurtherguideCBMCsintheimplementationoftheirrolesandresponsibilities,CBMCmembersmaynotalwayshavethecapacitytofulfiltheserolesandresponsibilitiesduetoresourceconstraints.FurthertrainingoftheCBMCsshouldgobeyondtheirdutiesinregardto
sensitisation,whichseemtobewellunderstood,andshoulddevelopmoretheircapacityinmonitoring,
formalschoolmanagementpracticesandensuringthatgirls’educationisactivelysupportedintheIQSs.Thiscanincludetheabilitytomorerigorouslyengagewiththefacilitatorsontheirattitudesandteachingbehaviours,andagreatersenseofaccountabilityonthepartoftheCBMCstoensureagirl-friendlylearningenvironment.
IQSsareuniversallydeficientin‘soft’and‘hard’infrastructure,andprovideapoorschoolenvironment
fortheattendingchildren.Theschoolconditionsarenotalwaysconducivetoagirl-friendlyenvironment,despitethefactthatgirlsappearnolesslikelytobeattendinganIQScomparedtoboys.EvidencesuggeststhatschoolleadersandCBMCmembersarewillingtoreflectonwaystoimprovegenderequityinschool
planning,andthisshouldbenotedinanyplanninggoingforward.Moreover,thebaselinestudyrevealsthatthepresenceofastrongfemalerolemodelcanalterperceptionsamongmothersandgirls.Themost
importantfactorimprovingagirl’sagency,however,appearstobetheattitudeofthefacilitator,andwhetherhe/sheisempowered.
ChallengesremaininthemonitoringofIQSsbylocalgovernment,withofficialsindicatingthattheystruggletoregularlyvisitthelargenumberofschoolsallocatedtothem.However,thismeansthatfollow-throughonCBMCeffectivenessisdifficult,witheachIQSoperatinginwhatappeartobelargelyunstructuredandcontextuallydrivencircumstances.Communityperceptionsregardingtheroleofthegovernmentalsotendtobenegative.
3. ContributionClaim3:Moreeffectiveteachingofformalsubjectsandanimproved,girl-friendly
learningenvironmentcontributetoimprovedlearninglevels,particularlyamonggirls
GirlsandboysappeartobeattendingIQSsequally.Atanyparticularlevel,girlsareslightlyolderthanboys,whichsuggeststhattheyareenteringschoollaterorarebeingheldbackmorethanboys.Amajorityofthe
pupilshaveyettoacquirebasicliteracyinHausaortheknowledgeandskillsexpectedinEnglishliteracy.
Manychildrenareonthecuspofachievingemergingnumeracythoughgainsinthepercentageofpupils
achievingbasicnumeracywillbemorechallengingandwillrequiregreatereffort.Girls’performanceinEnglishliteracydeclines,ascomparedtoboys,aroundpuberty(aged12orso),andperformancegapsinnumeracyalsoappeartowidenaroundthisagegroupaswell.
Girls’interestinimprovingtheirlearningoutcomesandboysrespectinggirlsbeingeducatedareassumptionsunderlyingContributionClaim3.ThebaselinefindingssuggestthatCBMCs’activitieshavecontributedtoanincreaseingirls’interestinformalsubjects.However,boysandgirlscontinuetoperceivethevalueofeducationdifferently.Stakeholders(includingboys)perceivegirlsasbeingtoofocusedonmarriage,andbelievethatthisaffectstheirattendanceandinterestinschool.
Alargeshareofchildrenattendmorethanoneschoolatthesametime,particularlyinBauchi.Thismeansthatchangesinthequalityofeducationintheotherschoolmayformanalternativeexplanationforimprovementsinlearningoutcomes.Theresearchidentifiespovertyandparentalattitudesaskeyfactorsrelatingtowhychildren,andinparticulargirls,mayonlyattendanIQS.Whenfacedwithbindingeconomicorculturalconstraints,IQSsofferanopportunityforchildren’s,particularlygirls’,exposuretoformaleducation.However,thereissomeriskthatfollowingtheintroductionofsecularteachingattheIQS,parentsmayrefrainfromsendingtheirdaughterstoapublicprimaryschoolandsendthemtotheIQSonly,inordertosavemoney.
Evidencepointstotwodiscrete,albeitinteracting,issuesthatunderminegirls’accessandretentionin
school–thecostofschooling,andsomepersistingresistanceamongstparentstosecularschoolingfor
girls.However,attitudestowardsgirls’educationareperceivedtobechanging,andparentsandcommunitiesappeartobewillingtoreflecton,andreconsider,theroles,responsibilitiesandcapabilitiesof
©EDOREN 76
girls.Changingattitudesalone,saythroughCBMCactivities,may,however,notbesufficienttobringaboutbehaviourchangeasthefinancialconstraintsofparentsarehighlightedashavingacriticalbearingoneducationalaccess.Givensuchconstraints,financialconstraintswilllimitthecapacityofschoolmanagementtoinvestinIQSs,whichmayultimatelyimpactthequalityoflearning,aswellasthecapacitytoretaingirls.
Theroleofwomenaseconomicagentsandaskeyincomegeneratorsforthefamily,withresultantresponsibilitiesforgirlstocontributetorunninghome-basedbusinessesinnorthernNigeria,affectsgirls’
schoolattendanceandlearningprocess.Whilstsensitisingwomeniscriticaltoimprovinggirls’involvementineducation,onitsownitisunlikelytobesufficientgiventhegenderedpoliticaleconomyofthecontextinwhichGEP3operates.
Menalsoplayacriticalroleingirls’lives,particularlyinrelationtoearlymarriage.Thequalitativecasestudiesindicatethatearlymarriageisprevalent,andisstatedasoneofthemainreasonsforlowenrolmentandretentionofgirls.Whenfacedwithearlymarriage,provisionsmaybemadeforthegirltocontinuehereducationpost-marriage,butthisisheavilydependentonherhusband’sdiscretion.Thus,thoughattitudechangesbyparentsareimportantintheyearsbeforemarriage(andinnegotiatingamarriage),changingtheattitudesofmenlikelytomarrygirlsinthecommunityislikelytobeequallyessential.
6.2 Recommendations
GEP3-widerecommendations
1. Theprojectpartnersshouldcontinuetheirpoliticalengagementtoensuregovernmentfundingis
leveraged,withanincreasedemphasisontheprojectscale-upandIQSS.Inlinewiththerecommendationsofthe2015AnnualReviewandtheEDORENpoliticaleconomyanalysis,theprojectpartnersneedtoputpoliticalengagementforeffectiveresourceprovisionforgirls’educationatthecentreofGEP3’sToC.Theprojectteamshouldsystematicallytrackandreportongovernmentprojectexpendituretofacilitatethisengagementandaccountability.Projectpartnersshouldcometoacommonunderstandingofhowprojectscale-upcanhappen,planforagreedtargetsaspartofinstitutionalisedplanningprocesses,andmonitorandaccountfortheirprogress.AnagreementonhowIQSScanbescaledupandfundedisparticularlyneeded.TheGESCshouldbeusedasaregularplatformtocoordinateadvocacyforgovernmentfundingandtofacilitatethemonitoringofkeyprojectassumptions.Aspartofgovernmentengagement,learningandretentionneedtoreceivestrongerattentionascoreprojectobjectives,togetherwiththecriticalassumptionthatsupply-sideinvestmentneedstorespondtoincreasedgirls’enrolment.
2. Theprojectshouldspecifyandcommunicateoperationalobjectivesandstrategies,strengthentheir
monitoringandstrengthenthemonitoringofunderlyingassumptions,andfeedbackmonitoring
informationtofacilitatelearningandaccountability.Governmentandotherimplementingpartnersthatareinvolvedin,oraresupervising,interventionexecutionshouldbemademoreawareof,andneedtoagreeon,whotheinterventionsintendtoreach,how,whenandwithwhatspecificobjectives.Themonitoringofinterventiondelivery,ofitsoperationalassumptions(e.g.theassumptionthattheinterventiontargetgroupwillbereached)andofitssupportsupervision,needstobestrengthened.Themonitoringdataneedtobesystematicallydiscussedamongprojectpartners,toenablelearningandaccountability.Theuseofdataingeneral–includingASCdata,amongotherkinds–shouldnotbetakenforgranted.ActivitiestopromotedatauseshouldbeembeddedinGEP3’sstrategy.
3. SBMC/CBMCmonitoringandsupportsupervisionneedcomprehensiveandsystematicemphasis.WesupportGEP3’semphasisonSBMC/CBMCcapacitybuildingandeffectivenessmonitoringbecauseoftheirpivotalroleintheproject’sToC.ThemonitoringandsupportsupervisionneedstobecomprehensivewithregardstothemultiplerolesthatSBMC/CBMCsplayinGEP3-supportedactivitiesandtheschool-levelresultsthatSBMC/CBMCsareexpectedtoachieve.Themonitoringshouldalso
©EDOREN 77
collectfeedbackaboutthesupportsupervisionitself,particularlyinIQSs,sincethisisanimportantcomponentofthecapacitybuildingapproach.CBMCeffectivenessmonitoringneedstoreceiveparticularattention,giventheirrecentinstitutionalisationandthecomplexcontextinwhichtheyexist.Themonitoringandsupportsupervisionneedstoberegular,backedbyadequateresourcesandreportedonsystematically.MonitoringfindingsneedtobediscussedwithstateandresponsiblelocalgovernmentofficersinvolvingSBMClocalandstaterepresentationstructures.TheempowermentoftheseSBMCstructuresisapromisingmechanismbywhichtofacilitatecontinuousandgraduallearningamongpeersandtoprovidevoiceandasenseofself-efficacytotheSBMCsandMAs.
Recommendationsfortheearlylearningintervention
1. Theteachertrainingandteachingmaterialsneedtobedesignedtakingintoaccountthelow
competencylevelsoftheteachersandteachers’subjectspecialisations.Thebaselinefindingshighlightthatteachers’knowledgeandskilllevelsareverylow,includinginbasicHausaliteracyandinassessingpupils’performance.TheRANAinterventionwillneedtobecarefullytailoredtoexistingknowledgeandskilllevelstoensurethattrainingcontentandmaterialsarepitchedattherightlevel,andthattrainingissufficientlyintensivetofillthelargegapsinteachers’subject,pedagogicalandcurriculumknowledge.Inaddition,theinterventionteamneedstocarefullyconsideritsteachertargetingapproach,asitcannotbeassumedthatallteachersareinvolvedinHausaliteracyteaching.Thebaselinefindingsindicatethatalargeshareofteachersonlyteachonesubject,andthatmathematicsisoneofthetwomainsubjectstaught.
2. TheRANAteamneedstoassesswhetheritsapproachtopupilliteracyimprovementisappropriate
fortheverylowpupillearningoutcomelevelsandteachercompetencylevels,andtoadaptteacher
trainingandlearningmaterialsifneeded.Pupillearningoutcomelevelsarecurrentlyverylowandlevelsfallwellbelowthecut-offpointbetweenpre-literacyandemergingliteracy.AssessmentitemsthatrequireknowledgeofphonicsrankasthemostdifficultitemsinboththeHausaandEnglishassessments.Theinterventionteamneedstotakethisintoconsiderationinitsapproachtopupilliteracyimprovement.
3. Teachercapacitydevelopmentneedstoincorporateactionstochangeteachers’awarenessand
perceptionsofteachers’self-efficacy.Teachersperceivepupilefforttobethecauseoflowpupilperformance,ratherthantheirowncompetenciesandteachingpractices.However,itislikelythattheirownlowlevelofcompetencycontributestolowpupilperformance.Thecapacitybuildingshouldsupportteachers’reflectionaboutthelinkbetweentheirowncompetenciesandpupilperformance;andshouldbuildteachermotivationandaccountabilityforimprovingpupillearningoutcomes.
4. TheRANAcommunityengagementcomponentandtheteachertrainingshould,totheextent
possible,addresspupilandhouseholdfactorsthatinfluencelearning.Theregressionmodelindicatesthatpupilandhouseholdcharacteristicsareassociatedwithlearningoutcomelevels.RANAiswellplacedtooffsetsomeoftheeffectsofsocioeconomicbackgroundonlearningoutcomesbypromotingamoresupportivehomelearningenvironmentforallchildrenthroughitscommunityengagementcomponent.
5. Thepeermentoringapproachneedstobeadaptedtoschoolswithonlyonetargetedteacher.Thebaselinefindingsindicatethat40%ofIQSshaveonlyasinglefacilitatoroftheintegratedcurriculumsubjects.Therefore,theschool-levelpeermentoringinitscurrentformisnotapplicabletoalargeproportionofIQSs.Itisrecommendedthattheprojectidentifiesalternativewaysofensuringthattrainedteacherscanprovidepeersupport,possiblybyengagingwithteachersfromdifferentbutnearbyschools.
6. RANAneedstosufficientlystrengthenthecapacityofIQSgovernmentstakeholderstoensurethat
theyprovideeffectivemonitoringandsupportsupervisiontoIQSs.Thebaselinedataindicatethat
©EDOREN 78
IQSsreceiverelativelylimitedexternalmonitoringandsupervisionvisits.Giventheircomplexcontext,theyarelikelytoneedcontinuoussupportandfeedbackevenmorethanpublicprimaryschools.RANAshouldnotassumethatIQSgovernmentstakeholdershavethecapacitytoconductmonitoringandsupportsupervision,andtheyshouldactivelysupportthis.Theroles,responsibilitiesandprocessformonitoringandsupportsupervisionshouldbewelldefinedandexplainedtoIQSstakeholders,inorderforittobeacceptedandaccountedfor.
7. RANAshouldtrytoattractthemostsuitablecandidatesatIQSsforitspedagogicalleadership
training.Thebaselinefindingsindicatethatheadteachersupportivepedagogicalactionsarerelativelylimited,particularlyinIQSs.ThecomplexityofIQSleadershipmaycontributetothis.ItwillbeimportantforRANAtotargettheIQSleadershipthatismostrelevantforthechangesinteachingpracticeswhichareaspiredto.
RecommendationsfortheIQSS
1. Selectionoffacilitators,headteachersandCBMCmembersrequirescloseattention,verificationand
monitoring.GEP3needstotargetfacilitatorsfortrainingwhoregularlyteachtheintegratedsubjects,whichrequiresverificationandsystematicmonitoring.Giventhat,ofthefiveintegratedcoresubjects,mathematicsandlanguagesarethemostfrequentlytaught,GEP3maywanttotargetfacilitatorsteachingthesesubjectsinthefirstinstance.SincetherolesofpedagogicalleadershipandschoolmanagementmaybesharedamongIQSstakeholders,andheadteachersmaybechosenforreasonsotherthantechnicalqualifications,theselectionofheadteachersfortrainingneedstobebasedonanexplanationoftheexpectationoftheroleintermsoftechnicalcapacity,inordertoattracttheappropriatecandidatefortraining.SimplyaskingtheIQStonominateandsendsomeonebasedonthetitleofheadteacher/Mallammaynotyieldpositiveresults.InthecaseofCBMCs,thereisacaseforGEP3totargetmemberswhocanmosteffectivelyfulfiltheresponsibilitiesrequiredfromtheCBMCs,whilebalancingthiswithequityconsiderations.ThesemembersmaynotnecessarilybethoseCBMCmemberswhoholdkeypositions,asrolesarenotalwaysclearlydefinedanddelegated.CBMCsappeartoneedfurthertrainingthatgoesbeyondtheirdutiesinsensitisation,anddevelopsmorecapacitiesinmonitoring,formalschoolmanagementpracticesandensuringthatgirls’educationisactivelysupportedintheIQS.Thiscanincludetrainingonhowtoengagewithfacilitatorsontheirattitudesandteachingbehaviours,andagreatersenseofaccountabilityonthepartoftheCBMCtoensureagirl-friendlylearningenvironment.
2. Monitoringandsupportsupervision,particularlyinthecaseofIQSs,needsclosefollow-upand
capacitysupport.TheIQScontextisdiverse,flexibleandstillunderdevelopment.InordertoadapttheIQSSinterventiontosuchacontext,quicklearningandfeedbackbasedonmonitoringdataisneeded.However,externalmonitoringandsupportsupervisionofIQSshasbeenlimitedinthepast.Hence,thisneedsclosefollow-upandcapacitysupportbyGEP3.Specialattentionisrequired,tomonitorschoolmobility,facilitatortransferandremuneration,pupilenrolmentrecords,pupilattendanceinotherschools,investmentinagirl-friendlyschoolenvironmentandtheschedulingofintegratedsubjectswithintheIQSclassschedule.
3. Facilitatortrainingandmentoringneedstobewelladaptedtotheverylowlevelsoffacilitator
competencyandtotheIQScontext.Facilitatortrainingwillneedtoaddressboththelowlevelsofpedagogicalknowledgeaswellasthelackofsubjectknowledge,particularlyinregardtoHausaliteracy,requiredtoeffectivelyusepedagogicaltechniquesandmaterials.Whilegenerallylow,facilitatorqualificationsvaryacrossIQSs,whichwillrequirethetrainingprogrammetobeabletodealwiththisvariation.Itisworthincreasingfacilitators’awarenessaboutthelinkbetweenteachercompetencies(orlackthereof)andpupilperformanceinorderforfacilitatorstobeabletobetterself-assesstheirrequiredcompetencies.Peermentoringwillrequiresupportingpeer-to-peerinteractionbetweenschoolssinceaquarterofIQSsonlyhaveonefacilitator.Knowledgedisseminationstrategies
©EDOREN 79
needtobeconsideredtoreinforceimpact,becausefacilitatorswithinthesameIQSarenotnecessarilyawareofeachother’sparticipationintraining.SincetheIQSteachingcontextisdiverse—withadequateinfrastructureoftenlacking,differentclassorganisations,andvaryingpupil–teacherratios—thetrainingneedstobecontextuallysensitive,totakeintoaccountthisteachingreality.
4. Trainingandmentoringongender-sensitiveclasspracticesneedstogobeyondthedisplayof
gender-sensitivetechniques,andneedstoaddressentrenchedgenderedattitudesaboutgirls’abilitytolearnandtheimportanceofeducationintheirfuturelife.
5. GEP3needstoadvocateforadequatefacilitatorremunerationandtopromotethemobilisationof
resourcesbeyondthecommunity.AverysmallshareofIQSfacilitatorsareremunerated,whichisagainsttherecommendationofpolicydocumentssuchastheIQTEBenchmark.Thelackofremunerationislikelytoaffectfacilitators’motivationtoteach,andmakesitdifficulttoattractqualifiedfacilitatorsandholdthemaccountable.WhileCBMCsareabletomobiliseresourcesfromthecommunity,significantresourcegapsremaininIQSsandCBMCs’perceivedroleisunderminedduetoalackoffunding.TheprojectshouldnotassumethatCBMCswillbeabletomobiliseconsiderableresourcesfromwithinthecommunity,andthereforeitshouldlookinto,andpromoteaccessto,otherresources.
6. Femaleandgirls’participationinshapingeducationandtheschoolenvironmentneedsfurther
investigation.CBMCsstruggletoattractwomenandensurewomenparticipateintheCBMC.Girls’voicesarehardlyrepresented.Increasingwomen’sinvolvementmayproveachallengesincetheirabsenceseemstoberelatedtoentrenchedgenderrolesandresponsibilitiesratherthanalackofrecognitionabouttheirpotentialcontributiontoschoolmanagement.Therefore,GEP3willneedtofurtherinvestigatemechanismsforensuringwomen’sandgirls’engagementinschoolmanagementthataresensitivetosuchrolesandresponsibilities.
7. Learningandteachingmaterialsneedtobeadaptedtotheverylowlevelsofskillsandknowledge
amongstbothfacilitatorsandpupils,andtothelanguageoftheusers.WhilealmostallfacilitatorssurveyedreportedspeakingHausa,Hausaliteracylevelsamongfacilitatorsaregenerallylow.Pupils’HausaliteracyproficiencyisextremelylowandinNiger43%ofpupilsspeakNupeathome.Learningandteachingmaterialsneedtobeadaptedtothislanguageandliteracyreality.
©EDOREN 80
Bibliography
Antoninis,M. (2012) Tackling the largest global education challenge? Secular and religious education inNorthern Nigeria. CSEA Working Paper WPS/2012-17. Oxford: Centre for the Study of AfricanEconomies.
DFID.(2011)DFIDEthicsPrinciplesforResearchandEvaluation.DFID.
Delahais,T.andToulemonde,J.(2012)Applyingcontributionanalysis:Lessonsfromfiveyearsofpractice.Evaluation,18(3),pp.281–293.
EDOREN.(2015)EvaluationFrameworkandPlan,UNICEF’sGirls’EducationProjectPhase3(GEP3).Oxford,UK:EDOREN.
Graham, A., Powell, M., Taylor, N., Anderson, D. and Fitzgerald, R. (2013) Ethical Research InvolvingChildren.Florence:UNICEFOfficeofResearch–Innocenti.
Mayne,J.(2012)Contributionanalysis:Comingofage?Evaluation,18(3),pp.270–280.
©EDOREN 81
AnnexA ToCoftheearlylearningintervention
Figure16visualisestheToCoftheearlylearningintervention.ThemaincausalassumptionunderlyingtheToCisthatliteracylearningoutcomesofgirlsandboys,particularlyinHausaasamothertongue,willimproveinearlygradesifteachingpracticeimprovesthroughtheuseofimprovedteachingandlearningmaterials(useoftheRANALiteracyPackage)andthepresenceofmoreknowledgeable,skilledandgender-sensitiveteachers.Numeracyisalsoexpectedtoimprove,buttoalesserdegree.Numeracywillbeinfluencedindirectlybyincorporatingnumeracythemesintothereadingcontent,exercisesandmaterials.
Theabovecentralassumptionisconditionalonschoolstakeholders(suchasparents,communityleaders,IQSproprietors,headteachers,teachersandgovernmentstaff)beingsupportiveofanincreasedemphasisonmothertongueinstructionandliteracyacquisition;andonteachersadheringtothemothertongueduringinstruction.Tothisend,theprojectwillengagethecommunity,championliteracyinthecommunities,trainheadteachersandadvocateforgovernmenttoimplementenablingpublicpolicies.OtherfactorsthatwillinfluenceimprovedHausa-basedteachingpracticeareregularpupilattendanceandtheclassandschoolenvironmentsbeingconducivetolearning.ThefocusonHausaassumesitisthemothertongueandthelanguagethatpupilsknowandunderstandbest.
Improvementofteacherknowledgeandskillsthroughin-serviceteacherdevelopmentisassumedtobecentraltomoreeffectiveteachingintheearlygrades.41ThisassumesthattheimprovedHausaliteracycurriculum,thematerials,thepedagogicalmethodologyandthein-servicetrainingandmentoringapproacharewelltargetedtotheneedsandcapacitylevelsofteachers,aswellastotheirteachingenvironmentinpublicschoolsandIQSs.Tooptimiseteacherlearning,theprojectenvisionsateacherdevelopmentapproachbasedonprogressiveprofessionaldevelopmentandschool-basedpeermentoringcomplementedbysupportivesupervisionandmonitoringbyMasterTrainersandgovernmentstaff.ThesuccessofthisapproachdependsonteachersbeingliterateinHausa,onweeklypeermeetingsbeingfeasiblegiventhecontextoftheschool(forexample,teachersbeingabletomeetregularlywithotherteachersgiventhesizeoftheschooloritslocation),LeadTeachershavingthecapacity,motivationandavailabilitytoleadthemeetings,andMasterTrainersandgovernmentstaffhavingtheresources,competenciesandincentivestoconducteffectivesupportivesupervision.Throughgender-sensitivetrainingthataddressesequityintheclassroom,teachersareexpectedtotargetgirlsmorespecificallyinclass,whichismeanttoincreasethelikelihoodofgirlsbenefitingfromtheeducationprovided.
Inorderforteachertrainingtoresultinimprovedknowledge,skillsandteachingpracticesteachersareassumedtobemotivatedtolearnandtotranslatenewknowledgeandskillsintopractice.Peer-to-peerinteraction,supportivesupervisionandotherincentives(e.g.trainingcertificates)aremeanttocontributetothis.Theextenttowhichteachermotivationtranslatesintoimprovedteachingmayalsobeinfluencedbyteachers’remunerationandtheirworkingconditions.Furthermore,actualimprovedteacherefficacyandlearningoutcomesmayimproveteachers’perceivedefficacy,whichinturnmayincreasetheirmotivation.
ImprovedteachingandlearningisassumedtobefacilitatedbythedistributionandusageofrelevantteachingandlearningmaterialsinHausa.Thisrequiresthattheteachershavethepedagogicalknowledgeandskillsneededtoeffectivelyusethematerialsduringteachingpractice.Furthermore,itassumesthatthematerialsarewellalignedwiththecurriculumandwiththecompetencylevelsoftheteachersandpupils.
Whilethecentralobjectiveoftheearlylearninginterventionistoimprovepupils’literacyand,toalesserdegree,numeracyskills,itisassumedthatthiswillfacilitatetheacquisitionofEnglishasasecondlanguage,andthetransitiontoEnglishinlatergrades.Furthermore,improvedlearningintheearlygradesisexpectedtocontributetohigherretentionratesbecausechildrenwhoperformwellearlyonareassumedtobemore
41Wedistinguishbetweenthreetypesofknowledge:subjectknowledge,curriculumknowledgeandpedagogicalknowledge.SeeSection3.2.9.2intheBaselineTechnicalReport.
©EDOREN 2
motivatedandsupportedbytheirparentstoremaininschoollonger,alinkthatmaybeparticularlyimportantforgirls,whofaceahigherriskofdroppingout(UNICEF,2014d).Apotentialfeedbackloopexistsbetweenincreasedretentionandteachingqualityinearlygrades.Thisfeedbackloopcanbenegativeorpositive.Ifhigherretentionisnotaccompaniedbyincreasedteachingandschoolresources,itcanleadtohigherpupil–teacherratios,whichmaynegativelyaffectteachingquality.Ontheotherhand,teachersmaybeincreasinglymotivatedbythehigherretentionoftheirstudents,whichmaypositivelyinfluenceteachingquality.
©EDOREN 3
Figure16: DiagramdepictingToCoftheearlylearningintervention
©EDOREN 4
AnnexB Evaluationquestions
B.1 Evaluationquestionsfortheearlylearninginterventionevaluation
Box6.EvaluationquestionsforGEP3’searlylearningintervention
1. TowhatextentdoestheearlylearninginterventionimproveHausaliteracyandEnglishlanguagelearningoutcomesamonggirlsandboysintheearlygradesinprimaryschoolsandIQSs?Towhatextentdoestheearlylearninginterventionreducethegapbetweenthelearningoutcomesofthelowestperformingpupilsandtheexpectedlearningoutcomes,asexpressedinthecurriculum?
2. Towhatextentdoesteachers’knowledgeinliteracyandlanguageacquisitioninearlygradesimproveasaresultoftheintervention?
3. Towhatextentdoteacherskillsinearlygrade,gender-sensitiveinstructionimproveasaresultoftheintervention?
4. Towhatextentandhowdoteachersadjustandchangetheirclassroompracticesasaresultoftheintervention?
5. AremorereadingandlearningmaterialsinHausausedintheclassroomduetotheintervention?Dotheycontributetomoreeffectiveteachingandlearning?
6. Towhatextentdoestheearlylearninginterventionimprovepupilretention,especiallyretentionofgirls?
B.2 EvaluationquestionsfortheIQSSevaluation
Box7.EvaluationquestionsforGEP3’sIQSS
1. HowwellhasGEP3’steachercapacitydevelopment(trainingandmentoring)contributedtoimprovedteacherknowledgeandskills,andmoreeffectiveteachingintheclassroom?
2. HowwellhasGEP3’steachercapacitydevelopmentcontributedtoanimprovementingender-sensitiveteaching?
3. HowwellhasGEP3’sheadteachercapacitydevelopment(trainingandmentoring)contributedtoimprovedpedagogicalleadershipandschoolmanagement?
4. HowwellhasGEP3’sCBMCcapacitydevelopmentcontributedtoimprovedschoolmanagementandincreasedmobilisationofresourcesforschoolinvestment?
5. HowwellhaveCBMCsbeenabletoadequatelymanagemini-grantsandtoinvesttheseresourcesintheimprovementofagirl-friendlyschoolenvironment?
6. Howwellhaveteachingandlearningmaterialssuppliedthroughtheinterventionbeenperceivedbyteachersandheadteachersasappropriateandwelltargeted?Havetheybeenusedformoreeffectiveteaching?
7. Towhatextenthavepupilliteracyandnumeracyskills,especiallyofgirls,improvedinGEP3-supportedIQSs?HowhasGEP3contributedtosuchimprovement?TowhatextentdoestheIQSSinterventioncontributetoreducingthegapbetweenlearningoutcomesandexpectedlearningoutcomes,asexpressedinthecurriculum?
8. Whataretheattitudesof,andwhatisthelevelofacceptanceby,communitymembersand
©EDOREN 5
leadersinregardtointegratingformalsubjectsinQur’aniceducation?Howhavetheseattitudesandhowhasthisacceptancechangedduringtheintervention?Why?Howhaveattitudeschangedinregardtogirlsreceivingformaleducation?
9. WhatunintendedconsequencesdoesIQSShaveforteachers,headteachers,pupilsandproprietorswithintheIQSs,aswellasforthebroaderschoolcommunity?
©EDOREN 6
AnnexC Samplesizeofbaselinesurvey
C.1 Samplesizeofearlylearninginterventionbaseline
Table8presentsthetargetedandfinalsamplesizesofthebaselinesurveyoftheearlylearninginterventionevaluation.Intotal,240schoolsweretargetedandsurveyedacrossthetwoschooltypesandthetwostates.Thesamplewasstratifiedaccordingtostateandschooltype,withequalsamplesizesof60schoolsineachstratum(60publicprimaryschoolsand60IQSsineachstate).
Ineachschool,thetargetsamplesizeforpupilsissixgirlsandsixboys,resultinginatotaltargetedsamplesizeof2,880pupils(1,440girlsand1,440boys).ThetargetedsamplesizeforteachersisthreeteachersperpublicprimaryschoolsandtwoteachersinIQSs.ThenumberofteachersislowerinIQSsbecauseweanticipatedonlytwofacilitatorsonaverageperIQS.Hence,thetotaltargetedteachersamplesizeis360primaryschoolteachersand240IQSfacilitators.Thefinalpupilsamplesizeis96%and88%ofthetargetinpublicprimaryschoolsandIQSs,respectively,whilethefinalteachersamplesizeequals83%and74%inpublicprimaryschoolsandIQSs,respectively.Therelativelylowerpercentageofteacherssurveyedcomparedtothetargetedsamplesizewasduetothefactthatsomeschoolsdidnothavethetargetednumberofteachers,whichwasparticularlythecaseintheIQSsinKatsina.
Table8: Targetedandfinalsamplesizesoftheearlylearninginterventionbaselinesurvey
Population
Publicprimaryschools IQSs
Targetsample Finalsample % Targetsample Finalsample %
Schools 120 120 100% 120 120 100%
Pupils 1,440 1,389 96% 1,440 1,264 88%
Teachers 360 299 83% 240 178 74%
C.2 SamplesizeofIQSSbaseline
Table9presentsthetargetedandfinalsamplesizesofthebaselinesurveyoftheIQSSevaluation.Intotal,60schoolsweretargetedandsurveyedacrossthetwoschooltypesandthetwostates.ThesamplewasstratifiedaccordingtoLGAs,withequalsamplesizesoffiveschoolsineachLGA,andtherefore30perstate.
IneachIQS,thetargetsamplesizeforpupilsissixgirlsandsixboys,resultinginatotaltargetedsamplesizeof720pupils(360girlsand360boys).ThetargetedsamplesizeforfacilitatorsistwofacilitatorsperIQS.Hence,thetotaltargetedfacilitatorsamplesizeis120IQSfacilitators.Thefinalpupilandfacilitatorsamplesizesbothequal80%ofthetarget.Inaround20%oftheIQSsonlyhalforlessofthetargetnumberof12pupilscouldbesurveyedbecauseonlyalimitednumberofpupilswaseligibleforthesurvey.Thebelowtargetachievementoffacilitatorssurveyedwasduetothefactthattheschooldidnothavethetargetednumberoffacilitators.
©EDOREN 7
Table9: TargetedandfinalsamplesizesoftheIQSSbaselinesurvey
Population
IQSs
Targetsample Finalsample %
Schools 60 60 100%
Pupils 720 576 80%
Facilitators 120 96 80%
CBMCs 60 52 87%
©EDOREN 8
AnnexD Randomisationchecksfortreatmentandcontrolgroup
Table10: School-levelbalance(allheadteachersinpublicprimaryschoolsandIQSs)
Variable Treatmentmean
Controlmean
Totalmean TotalN P-value
Headteacherisfemale 0.0 0.0 0.0 240 0.653
Whetherheadteacherhadbeenabsentinthelastterm? 0.6 0.7 0.6 225 0.114
Numberofdaystheheadteacherwasabsentinthelastterm 7.7 8.0 7.9 137 0.911
Shareofschoolsthathaveseparatefunctioningtoiletforgirls 0.3 0.2 0.3 239 0.583
Numberofyearssinceintegration 1.8 2.0 1.9 119 0.652
TotalnumberofboysenrolledinintegratedsubjectsfromP1toP6 173.8 201.1 187.6 115 0.522
TotalnumberofgirlsenrolledinintegratedsubjectsfromP1toP6 121.4 145.2 133.4 115 0.487
Ratiooftotalnumberofchildrenstudyingintegratedsubjectstoteachers
42.4 42.2 42.3 114 0.978
RatiooftotalnumberofchildrenstudyingintegratedsubjectstoteachersinP1–P3
39.6 42.2 40.9 115 0.694
Ratioofgirlstoboysenrolledintotal 0.7 0.7 0.7 111 0.347
Headteachersatinonanylessonfortheentireduration 0.5 0.5 0.5 225 0.846
Schoolrepairs 0.9** 1.0 0.95 239 0.047
RatioofgirlstoboysinZamfarainP1–P3 0.6 0.6 0.6 55 0.841
RatioofgirlstoboysinKatsinainP1–P3 0.8 0.9 0.8 56 0.354
Ratiooftotalpupilsenrolledtototalroomsusedforteachingontheday 49.9 77.7 64.4 107 0.182
Fractionoffemaleteachersofintegratedsubjects 0.1 0.1 0.09 240 0.172
OverallF-test,F-stat 1.875 P-Value: 0.1722
Note:Statisticalsignificanceofmeandifferencesidentifiedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
©EDOREN 9
Table11: Teacher-levelbalance(allteachersinpublicprimaryschoolsandIQSs)
Variable Treatmentmean
Controlmean
Totalmean TotalN P-value
Teacherisfemale 0.1 0.1 0.1 477 0.613
DoestheteacherhaveanSSCE? 0.7 0.7 0.7 477 0.515
DoestheteacherhaveanNCE? 0.6 0.5 0.5 477 0.330
DoestheteacherhaveaGrade2Certificate? 0.1 0.1 0.1 477 0.344
Hastheteacherbeenabsentinthelastterm? 0.7 0.7 0.7 450 0.862
Numberofdaystheteacherwasabsentinthelastterm 6.1 5.0 5.5 449 0.341
DoestheteacherspeakHausa? 1.0 1.0 1.0 356
Hastheteacherattendedtraininginthelasttwoyears? 0.4 0.5 0.5 473 0.238
Istheteacherabletoidentifylowperformers? 0.4 0.5 0.5 463 0.430
Istheteacherabletogiveevidenceforjudgementsanddiagnose? 0.1 0.1 0.1 463 0.556
Doestheteacherhavewritingskills? 0.2 0.2 0.2 463 0.578
DoestheteacherhaveHausaknowledge? 2.9 2.8 2.9 463 0.763
Doestheteacherhavecomprehensionskills? 2.1* 2.3 2.2 463 0.091
Istheteacherabletointerpretwordsandphrases? 0.2 0.1 0.2 463 0.652
OverallF-test,F-stat 0.2037 P-Value: 0.6519
Note:Statisticalsignificanceofmeandifferencesidentifiedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
©EDOREN 10
Table12: Pupil-levelbalance(allpupilsinpublicprimaryschoolsandIQSs)
Variable Treatmentmean
Controlmean
Totalmean TotalN P-value
TotalscoreinEnglishassessment 354.47 356.95 355.66 2649 0.429
TotalscoreinHausaassessment 507.34 509.67 508.46 2649 0.600
SpeaksHausaathome 0.99 0.99 0.99 2623 0.381
Femalepupil 0.46 0.44 0.45 2651 0.498
Ageofpupil 9.61* 9.35 9.48 1677 0.077
Hasachair 0.9*** 0.90 0.88 2578 0.001
Hasamotorcycle 0.67 0.66 0.66 2581 0.352
Hasacar 0.18 0.19 0.19 2578 0.256
HasaTV 0.38 0.39 0.38 2579 0.587
Hasacomputer 0.05 0.05 0.05 2574 0.92
Hasacamera 0.05 0.04 0.05 2576 0.194
Hasamobilephone 0.9** 0.95 0.94 2580 0.025
Hascattle 0.56 0.53 0.54 2579 0.187
Hasagoat 0.87 0.87 0.87 2579 0.973
Hasahorse,donkeyormule 0.18 0.16 0.17 2579 0.352
Hasasheep 0.7** 0.68 0.70 2580 0.018
Hasachicken 0.92 0.91 0.92 2581 0.356
OverallF-test,F-stat 0.8539 P-Value: 0.3555
Note:Statisticalsignificanceofmeandifferencesidentifiedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
©EDOREN 11
AnnexE Regressionoflearningoutcomesonkeyinfluencingfactors
ThetablesbelowreporttheresultsobtainedwithourmainregressionmodelsonfactorsinfluencingHausascaledscoresandEnglishscaledscores,respectively,intheearlylearningpupilsample.Thetablesreportthecoefficientestimatedforeachexplanatoryvariable,theirassociatedstandarderror,t-statisticsandp-values,whichprovideanindicationofthelevelofsignificanceinthecorrelationbetweeneachexplanatoryvariableandtheoutcomevariableofinterest,eitherHausaorEnglishscaledscores.
Table13: RegressioncoefficientsandtheirstatisticalsignificanceforthemainfactorsinfluencingHausascaledscores
Variable Coefficient Stderror t-stat p-value
Gender(pupilisfemale) -9.212 6.294 -1.464 0.145
Agedfrom7to10 23.376 6.828 3.423 0.001***
Agedfrom11to15 114.695 8.856 12.951 0.000***
Agedover16 178.553 20.394 8.755 0.000***
PupilinsecondHWItertile 13.769 7.181 1.917 0.056*
Pupilinthird(top)HWItertile 37.849 6.713 5.638 0.000***
Publicprimaryschool(notIQS) -57.353 11.026 -5.201 0.000***
Schoolhasgirls’toilets 8.479 7.486 1.133 0.259
Schoolhaswater 8.971 7.026 1.277 0.203
Pupilattendsotherschool -5.628 6.903 -0.815 0.416
Schoolinruralareas -32.710 11.559 -2.830 0.005**
SchoolinKatsina 3.438 6.683 0.514 0.607
Teachermotivation 12.749 16.729 0.762 0.447
Teacherknowledge1 25.998 14.323 1.815 0.071*
Teacherknowledge2 2.246 4.721 0.476 0.635
Teacherpedagogy1 -3.103 24.650 -0.126 0.900
Teacherpedagogy2 0.414 0.364 1.137 0.257
constant 446.139 57.053 7.820 0.000***
Numberofobservations:1,613;R-squared0.339
Note:Statisticalsignificanceofmeandifferencesidentifiedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
©EDOREN 12
Table14: RegressioncoefficientsandtheirstatisticalsignificanceforthemainfactorsinfluencingEnglishscaledscores
Variable Coefficient Std.error t-stat p-value
Gender(pupilisfemale) -6.752 5.706 -1.183 0.238
Agedfrom7to10 18.687 6.733 2.775 0.006**
Agedfrom11to15 82.448 9.185 8.977 0.000***
Agedover16 126.046 14.208 8.871 0.000***
PupilinsecondHWItertile 16.665 5.759 2.894 0.004**
Pupilinthird(top)HWItertile 27.566 5.902 4.671 0.000***
Publicprimaryschool(notIQS) -45.609 8.715 -5.234 0.000***
Schoolhasgirls’toilets 13.246 6.512 2.034 0.043**
Schoolhaswater 7.182 5.873 1.223 0.223
Pupilattendsotherschool 3.371 6.704 0.503 0.616
Schoolinruralareas -28.869 9.372 -3.080 0.002**
SchoolinKatsina 4.834 5.710 0.847 0.398
Teachermotivation 21.505 15.426 1.394 0.165
Teacherknowledge1 15.479 12.668 1.222 0.223
Teacherknowledge2 -0.142 3.710 -0.038 0.970
Teacherpedagogy1 -18.557 20.066 -0.925 0.356
Teacherpedagogy2 0.192 0.350 0.550 0.583
constant 278.692 53.062 5.252 0.000***
Numberofobservations:1,613;R-squared:0.332
Note:Statisticalsignificanceofmeandifferencesidentifiedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
Inadditiontothemainregressionmodel,theanalysiswasextendedbyusingaclusterfixedeffectsmodel,withtheschoolbeingthecluster.Thisenablesustocontrolforallschool-levelinfluencingfactorsatonce.
©EDOREN 13
Table15: Hausaschoolfixedeffectsmodel
Variable Coefficient Std.error t-stat p-value
Gender(pupilisfemale) -13.161 6.266 -2.100 0.037**
Agedfrom7to10 15.989 5.891 2.714 0.007**
Agedfrom11to15 83.130 10.829 7.677 0.000***
Agedover16 133.901 23.743 5.640 0.000***
PupilinHWItertile -0.598 7.513 -0.080 0.937
PupilinthirdHWItertile 20.247 6.027 3.360 0.001***
Pupilattendsotherschool 3.783 7.179 0.527 0.599
Numberofobservations:1,639;R-squared:0.606
Note:Statisticalsignificanceofmeandifferencesidentifiedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
Table16: Englishschoolfixedeffectsmodel
Variable Coefficient Std.error t-stat p-value
Gender(pupilisfemale) -10.366 5.698 -1.819 0.070*
Agedfrom7to10 15.998 7.250 2.207 0.028**
Agedfrom11to15 61.565 10.618 5.798 0.000***
Agedover16 96.922 17.944 5.401 0.000***
PupilinHWItertile 9.296 5.917 1.571 0.118
PupilinthirdHWItertile 14.186 5.770 2.458 0.015**
Pupilattendsotherschool 11.474 7.783 1.474 0.142
Numberofobservations:1,639;R-squared:0.606
Note:Statisticalsignificanceofmeandifferencesidentifiedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
©EDOREN 14
AnnexF UpdatedToCofIQSSintervention
TheinterventionToChasanimportantmethodologicalroleintheevaluationofGEP3’sIQSS.Whilecausalinferenceinthecaseoftheevaluationoftheearlylearninginterventionisderivedfromcounterfactualcomparisonusingacontrolgroup,contributionanalysisinferscausalityfromareasonedToC.Therefore,inthissectionwepresentanin-depthdiscussionoftheToCoftheIQSS,startingfromtheToCincludedintheGEP3EvaluationFrameworkandupdatedbasedoninsightsfromthebaselinedatacollection.SincetheoverallToCresultchainpresentedintheEvaluationFrameworkwaswell-founded,littleupdatinginthisregardhasbeenrequired.Thereviewmostlyfocusesonfurtherexplainingthemechanismandassumptionsunderpinningtheresultchain.
Error!Referencesourcenotfound.visualisestheToCofGEP3’sIQSS.InthefollowingsubsectionswewilldiscusstheToCaccordingtothreecausalpackagesthatmakeupthisToC.Eachcausalpackageconsistsofcausalfactorsinterlinkedviacausalmechanisms,plustheirunderlyingassumptions.Acausalpackageisassumedtobesufficientforgeneratingkeyintermediaryorfinaloutcomesontheconditionthatthecausalchainandassumptionshold.ForeachcausalpackagerelatedtoGEP3’sIQSStheremaybealternative,non-GEP3relatedexplanationsfortheoutcomeoccurring,whichwillrequireexaminationaspartoftheevaluation.
Eachcausalpackagepresentedbelowcorrespondstothecontributionclaimspresentedinthemethodologicalsection.42
42(Mayne,2012).Thecausalpackageisdefinedasbeingsufficienttocauseanoutcome,whileitsindividualelements,inparticulartheinterventionelements,arenecessarypartsforthepackagetomakeadifferencetotheoutcome.
©EDOREN 15
Figure17: DiagramdepictingToCoftheIQSS
©EDOREN 16
ContributionClaim1GEP3’sIQSScontributestomoreeffectiveteachingofformalsubjectsinIQSs
Causallinkagesandmechanisms
GEP3’sstrategyforintegratedQur’aniceducationaimstoprovideaccesstoqualityeducationinIQSs,particularlyforgirls,byimprovingeffective,gender-sensitiveteachingoftheformalsubjectsincludedintheharmonisedintegratedcurriculum.MoreeffectiveteachingisconsideredakeyintermediaryoutcomeoftheIQSS.Weconceptualiseeffectiveteachingasacombinationofthefollowingattributes:1)demonstratedteachingcompetency,drawingonthreetypesofknowledge(subjectknowledge,pedagogicalknowledgeandcurriculumknowledge);2)useofapupil-centredlearningapproach;3)timeontask;4)effectiveuseofteachingandlearningmaterials;and5)noobservablegenderbiasduringteachingpractice.
ThecorecausalpathwaytoachievemoreeffectiveteachingisbyimprovingIQSfacilitators’knowledgeandskillsthroughtrainingandmentoring.Thetrainingismeanttoimprovetheirknowledgeandskillsintermsofthecoresubjectsoftheharmonisedintegratedcurriculum,pedagogy,andtheintegrationprocessinQur’aniceducation.Thetrainingisalsomeanttoimpartabetterunderstandingofgender,equityandgender-sensitiveteachingmethods..Thementoringprocessisassumedtoplayanimportantroleinfacilitatorseffectivelyacquiringthisknowledgeandskills,andputtingitintopractice,byactivatingthefollowingmechanisms:
• peerinteractionthatencouragesfacilitatorstodiscussacquiredknowledgeandskills,shareexperiencesandprovidefeedbackbasedonacollaborativerelationshipamongpeers;
• school-basedsupportthatallowsfacilitatorstoaccessandrefreshtheirknowledgeandunderstandingatanytimewhilstusingtheprovidedmaterials;and
• continuousface-to-face,practicalsupportbyanexperiencedmentorbasedonalong-term,personalrelationshipthatreinforcesthelearninganddeliveryofnewlyacquiredknowledgeandskillswithconfidence.
Mentoringisalsoassumedtoincreaseteachermotivation.Whileincreasedteachermotivationisnotadirectintendedobjectiveoftheintervention,itisassumedtobeakeycontributingfactortomoreeffectiveteaching.Theunderlyingtheoryisthatpeerinteractionandongoingsupportincreasethefacilitators’perceivedteachingefficacy,self-confidenceandself-esteem,whichresultsinincreasedmotivation,asexpressedin(i)moreinterestandenjoymentintheirwork,(ii)moreeffortbeingmade,(iii)moreimportanceattachedtotheirwork,and(iv)lesspressureandtensionexperiencedinrelationtotheirwork.43Thisinturnsenhancesthelikelihoodoffacilitatorsactuallyapplyingtheirimprovedknowledgeandskills,andspendingincreasedtimeontaskonactivitiesthatbenefitstudentlearning.
ThedistributionofHausateachingandlearningmaterialscontributestomoreeffectiveteachingasmaterialsprovideguidancefortheteacherduringtheteachingprocessandfacilitatetheknowledgetransferbetweenteachersandpupils.Thisrequiresteacherstohavethepedagogicalknowledgeandskillstoeffectivelyusethematerialsduringteachingpractice.Pupilworkbooksandnotebooksalsoallowpupilstopracticeandallowteacherstoassesspupilsinwriting.TheprovisionoflearningmaterialsmayalsoensuretheinclusivenessofintegratedQur’aniceducation.However,itmustbenotedthatpovertyhasbeenidentifiedasoneofthekeybarrierstoaccessingpublicprimaryschool43ThistheoryisbasedonthetheoryofmotivationusedbytheTDPinNigeria.
©EDOREN 17
andtotheextentthattheintegrationprocessforcesparentstoinvestinlearningmaterials,comparedtothesituationwithoutintegration,parents’inabilitytodosomaycausepupilstodropout.
Headteachertrainingseekstosupportamoreeffectiveteachingprocessinseveralways.First,thetrainingisassumedtoincreasepedagogicalleadershipintheIQSbyprovidingtheheadteacherwithknowledgeandskillsintermsofpedagogyandleadership.This,inturn,willprovidethefacilitatorswithschool-based,continuousguidanceinhowtobetterdeliveryinstruction.Second,leadershipcaninspireandmotivatethefacilitator,whichcontributestoimprovedattendanceanddemonstratesitselfinincreasedeffortandenjoymentofwork.Third,thetrainingalsoaimstoimprovetheheadteacher’sschoolmanagementpractices,suchasteacherattendancemonitoringandclassscheduling,whichinturncancontributetoincreasedfacilitatorattendanceandtimeontask.Improvedschoolmanagementcanalsoincreasetheresourcesavailableforfacilitatorremuneration,whichcanaffecttheirattendance,andcansubsequentlycontributetomoreeffectiveteaching.
Animproved,moregirl-friendlyschoolenvironmentandimprovedmonitoringandsupportsupervisionbylocalgovernmenteducationstaffisassumedtocontributetomoreeffectiveteaching.Abetterschoolenvironment,intermsofinfrastructure,classroomconditions,safetyandrelationshipsbetweendifferentactorsintheschool,canimproveteachers’andpupils’comfortandfeelingofwellbeingintheschool.Itcanalsoleadtoincreasedinstructionaltimebycreatingthephysicalconditionsrequiredforteaching(e.g.light).Girl-friendlyschoolfeatures,suchasthepresenceoffemaleteachersorfemaleparticipationinschoolmanagement,cancontributetoanincreaseingender-sensitiveteachingpracticesbycreatingaclimatethatismoresupportiveofgirls.Improvedgovernmentmonitoringandsupportsupervisioncancontributetobetterschoolmanagementandcanmotivatefacilitatorsandproprietorstocontinueprovidingtheintegratedcurriculumasitsignalsgovernmentsupportforintegration.Furthermore,sustainedfinancialgovernmentsupportcansupportfacilitatorremuneration,whichwillcontributetofacilitators’motivationandattendance.
Whilemoreeffectiveteachingisassumedtocontributetoimprovedlearningoutcomes,improvementinlearningoutcomescaninturncontributetoanincreaseinfacilitators’perceivedteachingefficacy,whichisexpectedtostrengthentheirmotivation.Anincreaseingirls’enrolmentandretentionmayalsoaffectteachingpractices:first,itmaycontributepositivelybycreatingamoregirl-friendlyschoolenvironment,whichinturncansupportgender-sensitiveteachingpractices;second,itmayhaveanegativeeffectbyincreasingpupil–classroomandpupil–teacherratios.
Assumptions
Assumptionsaredefinedhereasinfluencingconditionsthatmightenablecausallinkagesandmechanismstowork,ormayimpedethoselinkagesandmechanism.KeyassumptionsunderlyingthecausalchainofContributionClaim1arepresentedinFigure18.
Implementationassumptionsarenotincludedinthediagram.Itisassumedthatthetrainingandmentoringareimplementedasplannedandcoverallthecomponentsofthetrainingcurriculuminaqualitymanner,andthattheteachingandlearningmaterialsaredistributed.Theseareimportantassumptions,inparticularwithregardstothementoringsinceGEP3’scapacitydevelopmentapproachforfacilitatorsstronglyemphasisescontinuous,school-basedfollow-upbyexperiencedmentorscomplementedwithpeerinteraction.
©EDOREN 18
Figure18: Causalpackageandunderlyingassumptionsof‘moreeffectiveteaching’
©EDOREN 19
Alternativeexplanations
Weconsidertwoalternative,non-GEP3-relatedexplanationsthatcouldalsobesufficientforthe
intendedchangeinteachingintheIQSstobeobserved:
• Facilitatorsbuildcapacityinotherways.IffacilitatorsintheGEP3IQSsparticipateinothertrainingorcapacitybuildingprocessesduringtheGEP3projectperiod,teachingcouldbecome
moreeffectiveregardlessofGEP3.Suchcapacitybuildingmaytakeplacewhenfacilitatorsare
alsoteachersinpublicprimaryschools.Furthermore,ifnewfacilitatorswhohaveahigher
teachingcompetencylevelareemployed,teachingmayalsobecomemoreeffective.
• Facilitatorsaremotivatedinotherways.Iffacilitatorremunerationorworkingconditions
improvewithoutanyinfluencebyGEP3interventions,facilitatorsmaybecomemoremotivated
andteachmoreeffectivelyregardlessofGEP3.
ContributionClaim2:GEP3’sIQSScontributestoanimproved,girl-friendlyschoolenvironmentinIQSs
Causallinkagesandmechanisms
MainlythroughitssupporttoCBMCsGEP3seekstoimprovetheschoolenvironmentoftheIQSandmakeitmoregirl-friendly.Weconsidertheschoolenvironmenttoencompassthephysical
environment,intermsofinfrastructureandclassroomconditions,itssecurityandperceivedsafety,
andtheschool’sorganisationandinstitutionalculture,intermsofleadership,organisational
processes,relationshipsbetweenschoolstakeholders,andtraditions.Theschoolenvironment
affectsthelearningexperienceofthepupils,aswellastheteachingexperienceofthefacilitators.
GEP3seekstocontributetothegirl-friendlinessoftheschoolenvironmentintheIQS,promoting
investmentininfrastructureandresourcesthatimprovetheschoolexperienceofgirlsandaschool
organisationthatsupportsgenderequity.
Thecorecausalpathwaytoimprovingtheschoolenvironmentisbyimprovingschoolmanagement
andsupportingthemobilisationofmoreresources.Additionalresourcesthataremobilisedcanbe
investedincriticalinputsthataddressbarrierstogirls’educationandthatareprioritisedinschool
developmentplans.Betterschoolmanagementisassumedtoimprovetheschoolenvironment
because,throughthedevelopmentofWCDPs,strengthsandweaknessesintheschoolenvironment
areassessed,prioritiesforimprovementidentified,andresourcemobilisationactionsoutlined.
ThroughtheactiveparticipationoftheCBMC—whichismeanttorepresentthedifferentschool
stakeholders—theprioritiesandpreferencesofvariousstakeholdersaretakenintoaccount:in
particularthepreferencesofwomenandgirls.Itisassumedthatwomenandgirls’participationin
theschoolmanagementbettersignalsgirls’needsandstrengthensthedemandforgirl-friendly
schoolinputs.Furthermore,betterschoolmanagemententailsmoreadequatefinancial
managementandrecord-keeping,whichsupportsplanningandtransparency,andhencemore
effectiveuseofresourcesprovidedbythecommunityorotherexternaldonors.Giventheoftenlow-
resourcesettingoftheIQSsresourcemobilisationisanecessaryconditiontotranslatedevelopment
plansintoinvestments.
ThroughafunctionalCBMCcommunityrepresentativesareinvolvedintheIQS.Thisisassumedto
increasecommunitysupportforintegratedQur’aniceducation,particularlyforgirls,becauseitmakescommunitymembersmoreconsciousoftheintegrationandempowersthemtohaveasayin
itsimplementation.Thiscontributestoincreaseddemandforgirls’education.Inaddition,ifthe
communitybecomesmoreinvolvedinthemanagementanddecision-makingoftheIQSthenitis
assumedthattheywillbemorelikelytosupportitfinanciallyorwithotherresourcesbecausethere
©EDOREN 20
willbecommunityownership,oraperceptionofownership,overtheIQS.Finally,community
membersareabletobetterholdschoolleadersandteachersaccountableforthequalityofthe
schoolenvironmentandattendancesince,throughtheirinvolvement,theyaremoreawareofwhat
occursintheschool,andagainbecauseoftheirstrongersenseofschoolownership.
Themainmeanstoachievetheaboveintermediaryoutcomesisthetraining,mentoringandmonitoringoftheCBMCmembers.Thetrainingaimstoimprovetheirunderstandingoftheirroles
andresponsibilities,whichcanenableschoolmanagementandresourcemobilisationbecause
managementandsupervisorytasksandfunctionswillbebetterdefined(e.g.teacherattendance
monitoring,planning,andfundraising).BytrainingCBMCmembersonwholecentredevelopment
planningtheirskillsinschoolplanningaremeanttoimprove.Theinclusionofgenderinthetraining
curriculumaimstocreateawarenessoftheissue,whichissubsequentlyassumedtoresultinmore
gender-responsivewholecentredevelopmentplanning.Trainingonresourcemobilisationand
financialmanagementismeanttomakeCBMCmembersmoreabletomobiliseandmanage
resources.Theperiodicmentoringvisitscanreinforceandrefreshthelearningfromthetraining
workshops,providingpracticalfeedbacktailoredtothecontextofeachIQS.Itislesscleartowhat
extentthementoringvisitswillre-enforcegendersensitisationtoensurethatCBMCmembersfollow
uponandmonitorthegirl-friendlinessoftheschoolenvironment.Monitoringcansignal
deficiencies,basedonwhichactioncanbetaken,andcanincreaseaccountabilitybydirecting
attentiontotheachievementofresults.GEP3willsupportgovernmentstafftocarryouttermly
CBMCeffectivenessmonitoring,providingatoolthatismeanttoimproveCBMCs’monitoring
capacity.
BeyondtheroleoftheCBMC,theheadteachertrainingseekstoincreasetheheadteachers’skillsinschoolmanagement,whichisalsoassumedtocontributetomoreeffectiveschoolmanagement.
Themini-grantsdirectlycontributetoanincreaseinschoolresourcesand,giventhattheyaretargetedatgirl-friendlyinvestments,canimprovethegirl-friendlinessoftheschoolenvironment.
ThegrantscanalsoworkasanincentiveforCBMCmemberstodeveloptheircapacity,asmini-
grantsareprovidedontheconditionthatCBMCmembershaveparticipatedinthetraining.
Furthermore,themanagementofthegrantcanimproveoverallcommunity-basedschool
managementthrougha‘learning-by-doing’capacitybuildingapproachfortheCBMC,whichismeant
tomanagetheresources.
Gender-sensitiveclasspracticesandtheincreasedpresenceofgirlsintheIQS,duetoincreased
enrolmentandretention,canstrengthenaschoolclimatethatsignalsanopennesstogirlpupilsand
cancreateasocialenvironmentwheregirlsfeelwelcomeamongtheirpeers.
Assumptions
Figure19presentstheassumptionsabouttheconditionsthatneedtobeinplaceforIQSS
interventionstoactuallyresultinanimproved,moregirl-friendlyschoolenvironment.
©EDOREN 21
Figure19: Causalpackageandunderlyingassumptionsof‘improved,girl-friendlyschoolenvironment’
©EDOREN 22
Alternativeexplanations
Weconsiderthefollowingalternative,non-GEP3-relatedexplanationsthatcouldimprovetheschoolenvironmentandmakeitmoregirl-friendly:
• CBMCmembersandschoolleadershipimprovetheircapacitythroughothermeans.SchoolmanagementandresourcemobilisationcanimprovebynewstakeholdersbecominginvolvedintheIQS,regardlessofGEP3support.Forexample,aphilanthropistmaytakeaninterestintheIQSortheIQSmaystartbenefittingfromtheinfluenceorcompetenciesofahighlyeducatedor/andrespectedcommunitymember.ItisnotonlydifficulttoidentifyallsuchinfluencesbutitwillbealsohardtoseparateoutGEP3’sinfluenceinsuchinstancessinceCBMCtrainingpromotessuchinvolvement.SomeCBMCmembersmayalsobemembersoftheSBMCintheneighbouringpublicprimaryschool,andmaybuildtheircapacitythroughtheirparticipationinpublicprimaryschoolmanagement.
ContributionClaim3:Moreeffectiveteachingofformalsubjectsandanimproved,girl-friendlylearningenvironmentcontributetoimprovedlearninglevels,particularlyamonggirls
Causallinkagesandmechanisms
TheprincipalfinaloutcomethatGEP3aimstoachievewithitsIQSSisforpupilslearningoutcomestoimprove,particularlyforgirls.Moreeffectiveteachingisconsideredacriticalcontributingfactor,whichwillbesupportedbyanimproved,moregirl-friendlyschoolenvironment.Asthesetwofactorsimproveandresultinbetterqualityeducationgirls’retentionisalsoassumedtoincrease.Inaddition,girls’enrolmentisexpectedtoincreasebecauseofanincreaseddemandforgirls’educationasaresultofmorecommunityinvolvementandsupportforIQSs.Girls’retentionandenrolmentarenotfurtherdiscussedbelow,astheyarenotthefocusofthisevaluation.However,itisimportanttonotethatincreasedretentionandenrolmentcaninturninfluenceeffectiveteachingandtheschoolenvironment,aswasdiscussedintheprevioussections.
Internationalstudiesindicatethatwhatteachersknow,whattheydoandhowmuchtheycareaccountsformorevarianceinpupilachievementthananyotherpolicy-amenablevariable(Hattie2003).Severalmechanismsareatplaythatmaycausemoreeffectiveteachingtoleadtoimprovedlearningoutcomesforgirls.First,thecombinationofimprovedsubjectknowledgeandpedagogicalskills,supportedbyfit-for-purposeteachingandlearningmaterials,canenablemorecompetentandequippedfacilitatorstobettersupportpupils’learningachievementintheclassroom.Byteachinginamoregender-sensitivemannertheteacher’sinteractionwiththegirlpupilimproves,whichcanimprovethequalityofthegirl’seducationinparticularandprovideadditionalmotivationforhertolearn.Second,student-centredteachingmethods—acompetencythatGEP3trainingintendstostrengthen—encourageschildren,especiallygirls,toattendschools,whichincreasesthepupil’sactualinstructionaltimeandcan,therefore,increaselearning.Third,effectiveteachingalsoentailsfacilitatorsattendingandteachingwheninschool(asaresultofbetterschoolmanagement).This,again,canincreasetheactualinstructionaltime,andhencepositivelyinfluencelearningoutcomes.
Totheextentthattheschoolenvironmentismoreconducivetogirlslearning,learningachievementisfurtherstrengthened.Again,thecausalmechanismshereareseveral.First,improvedphysicalconditionsunderwhichteachingandlearningtakeplace(e.g.lessovercrowdedclassroomsorteachinginashadedlocation)enhancethedegreeofsupportthatteacherscanprovideandtheattentionthatpupilscangivetolearning.Second,thephysicalconditionsinfluencetheinstructionaltimeasprotectionagainstrain,orlightduringeveningclasses,enableclassestotakeplace.Third,asafe,comfortableandenjoyableschoolenvironmentaffectspupils’experienceofschooling,whichislikelytoleadtomorefrequentattendanceandgreaterengagementintheschoolingenvironment.Totheextentthattheschoolenvironmentismoregirl-friendly,girls’schoolexperienceisparticularlyenhanced.
©EDOREN 23
Assumptions
Figure20presentstheassumptionswithregardstotheconditionsthatneedtobeinplaceformoreeffectiveteachingandanimproved,girl-friendlyschoolenvironmenttoactuallyresultinimprovedlearningoutcomes,particularlyforgirls.Wedonotfocushereonassumptionsrelatedtoretentionbutmanyofthepresentedassumptionsarerelevanttoachievingtheretentionofgirls.
Therearemanyfactorsthat,inturn,influencewhethertheconditionshold.Thedifferentassumptionsarealsointerdependent.Pupils–inparticular,girls–actuallyattendingclassisinfluencedbyexternalfactors,suchastheiroutsideschoolactivitiesandresponsibilities,publicprimaryschoolattendance,economicconstraints,schoolaccessibility,andtheattitudesofthepupilsandtheirsocialenvironment.Girlsareassumedtobeinterestedinacquiringaformaleducation,whichwilldependontheiraspirationsandthevaluetheyseeinformaleducation–given,inparticular,theexpectationofearlymarriage,whichisanimportantbarriertogirls’retention.
Figure20: Causalpackageandunderlyingassumptionsof‘improvedlearningoutcomes’
Alternativeexplanations
Weconsiderthefollowingalternative,non-GEP3-relatedexplanationsthatcouldalsobesufficienttoexplainimprovedlearningoutcomes:
• Improvedaccessandqualityeducationinotherschools.ThebaselinedatademonstratethatIQSsarenottheonlyalternativetoaccessingformaleducation.PupilsareenrolledinbothIQSsandpublicprimaryschools.Theextentthatpupilsattendthetwoschoolsdependsonmanyinteractingfactors,suchasaccessibilityofthepublicprimaryschool,therelativecostofattendingpublicprimaryschoolsandpovertyoftheparents,aswellasculturalbeliefsandsocialnormswithregardstoformaleducationandgirls’accesstothis.Iftheseunderlyingfactorschangepublicprimaryschoolattendancemay
©EDOREN 24
increaseandlearningoutcomesmayimprove,regardlessoftheintegrationofformaleducationintheQur’anicschools.Thequalityofeducationalsoplaysarole.Accesstopublicprimaryschooldoesnotnecessarilyresultinimprovedlearningoutcomes.TotheextentthatthequalityofeducationinthepublicprimaryschoolsthatIQSpupilsareattendingbecomesbetter,theirlearningoutcomescanimprove.Furthermore,inlessremoteareaspupilsmayattendvocationalorothertraining,whichmayprovidesomeformofbasiceducationormotivatepupilstolearn.Finally,inmoreurbanareasformalprivateschoolsmayalsoattractpupils,whichcanaffecttheirlearningoutcomes.
• Changesinthecompositionofthepupilpopulation.Overtime,theIQSmayattractadifferentpupilpopulation:forexample,youngerpupilsstarttheintegratedcurriculumormorepupilswithpreviouseducationexperienceareattractedtotheIQS.TotheextentthattheirlearningoutcomelevelsaredifferentfromthelearninglevelsofthebaselinepopulationchangesintheaveragelearningoutcomeintheIQSmaybeobserved,regardlessofimprovedteachingoranimprovedschoolenvironment.
EDOREN is a consortium of leading organisations in international development and education: Oxford Policy Management (OPM), and the Institute of Development Studies (IDS) at the University of Sussex, and is supported by UK Aid. EDOREN cannot be held responsible for errors or any consequences arising from the use of information contained in this report. Any views and opinions expressed do not necessarily reflect those of OPM, IDS and EDOREN or any other contributing organisation.