Impact Evaluation of the QIDS UP:Impact Evaluation of the QIDS UP:Learning and Teaching Support MaterialLearning and Teaching Support Material

Department of Education, South Africa

AFRICA IMPACT EVALUATION INITIATIVE, AFTRL

Africa Program for Education Impact Evaluation

QIDS UP

• QIDS UP aims to raise school achievement of South African pupils towards international standards by raising primary schools above a set standard of infrastructure and school inputs within the next 5 years in the poorest 40% of learners.

• QIDS UP will provide– Teaching and learning material– Qualified teachers– Infrastructure– Reduction of class size

Evaluation Questions

• What is the impact of the provision of the minimum resource package on teaching and learning outcomes in the quintile 1 and 2 schools?

• What is the additional impact of teacher and management training on utilization of learning material on teaching and learning outcomes?

Indicators

• Learning achievement: – literacy and numeracy for grade 3

• Quality of teaching: – learner perception, – participation, – use of material (exercises from the textbooks,

frequency of use of material) • Availability of material:

– learner textbook ratio per learning program (literacy, numeracy and life skills)

• Management: – textbook retrieval rate

Evaluation Design(Gauteng: Q1-2; Western Cape Q2, option Q1)

• Question 1: Regression Discontinuity design 250 participating schools representing 100% of eligible schools– Impact measured as the difference in outcome

indicators for schools below (125) and above (125) the 21.9% cutoff.

• Question 2: Experimental design– Training will be randomly assigned to half of the

schools in the program (125 out of the 250 participating schools) to all principals and grade 3 teachers in each school.

– The additional impact will be measured as the difference in outcomes between trained schools and not trained schools. (w. double dif)

Sample and Data

EXISTING• March 2007 Auditing census of School Infrastructure

(30,000 schools).• Sept 2007 National Grade 3 Assessment. Random sample of

schools and learners in the country (1 day in each school). • Contextual questionnaires: principals, teachers and student

and parents (1 additional day per school).• Provincial administrative system/monitoring system of

QIDS UP: financial and in kind resources delivered to each school.

ADDITIONAL• Gauteng. Expand sample to the 375 bottom schools. Test

schools in Sept 2007 and Sept 2008• Western Cape. Expand sample to the 144 bottom schools.

Test schools in Sept 2007 and Sept 2008• One additional province to be considered

Staffing Plan

TeamNational project managerProvincial project managersResearch team (Lead researcher WB, JPAL, local researcher(s), Quality Assurance (DoE))Data collectorsSteering Committee National project managerProvincial project managersLead researcherQuality Assurance

TimelineSubmit Concept Note to DDG and DG for

approval.6 or 13 Jul 2007

Feedback from management 10 or 17 Jul 2007

Provide comments on contextual questionnaires Jun-07

Consultative meeting with QIDSUP and Provinces

End Jul 2007

Data collection (baseline) Sep-07

School training Oct-Nov-07

QIDS UP roll out Starting Jan 2008

Data collection (follow up) Sep-08

Impact evaluation analysis Dec-08

Budget