wfp bangladesh school feeding programme mid term...

WFP Bangladesh School Feeding Programme Mid Term Evaluation Final Report

Submitted by:

Beatrice Lorge Rogers, Jennifer Coates, Akoto Kwame Osei

Friedman School of Nutrition Science and Policy Tufts University

150 Harrison Avenue Boston, MA 02111 USA

December 30, 2004

Acknowledgements: The authors would like to thank Edith Heines, Mahjabeen Masood, and Ezaz Nabi at WFP/Bangladesh, Akhter Ahmed at IFPRI, and Parke Wilde and Robert Houser at Tufts FSNSP for facilitating this study and report through their comments, suggestions, and shared expertise. Our deep gratitude is extended to the Managing Directors of Data Analysis and Technical Assistance, Inc. (DATA), Zahidul Hassan and Md Zobair, and to the rest of the DATA staff for the key role they played in all stages of the survey planning, implementation, and analysis. Many thanks as well to the talented and dedicated enumerators and to the schoolteachers and other respondents, without whom the survey could not have happened.

Table of Contents

Executive Summary I. Introduction 1 II. Study Design and Sample Methods 2

Baseline Survey 2 Resurvey Sampling Methods 2 Questionnaire Development 4 Field Testing and Training 5 Survey Implementation 5 Data Entry and Analysis 6

III. Results (Descriptive) 7

Presence of WFP School Feeding Programme 11 Presence of Other On Site Feeding 11 Presence of Take Home Food and Cash and other Programmes 12 School Feeding Programme Implementation 13 Reliability of Delivery and Distribution 13 Acceptability and Consumption of the Biscuits 14 Attitudes Toward the School Feeding Programme 16

IV. Results: Programme Impact (Descriptive Bivariate Analysis) 18 Impact Indicators: Enrollment 18 Impact Indicators: Attendance 18 Impact Indicators: Dropout Rate 19 Impact Indicators: Repetition Rates 20 Impact Indicators: Achievement Test Scores 22 Impact Indicators: Classroom Behavior 22 School Management Committees 24 Water and Sanitation Facilities 25 Other Programme Effects: Substitution of Meals 25 Other Programme Effects: Classroom Crowding and Student/Teacher Ratio 26 Other Programme Effects: SFP Effect on Teaching 27

4

V. Results: Programme Impact (Multivariate Analysis) 29

V.1 Fixed Effects Estimates 29 Impact Indicators: Enrollment 30 Impact Indicators: Attendance 32 Impact Indicators: Drop-Out 33 Impact Indicators: Repetition 33

V.2. Pooled Analysis 33

Impact Indicators: Enrollment (Pooled Analysis) 34 Impact Indicators: Attendance (Pooled Analysis) 35 Impact Indicators: Achievement Test Score 35

VI. Discussion and Conclusions 38 References 41

5

List of Tables

Table 1. Key School-level Characteristics by Intervention and Control Areas at Baseline 7 Table 2. Comparison of Rural and Urban School Characteristics, 2004 Survey 9 Table 3 Comparison of Government and Non-Government Registered School Characteristics,

2004 Survey 10

Table 4. Current Participation in On-Site Feeding Programmes 11Table 5. Participation in Take Home Food or Cash Stipend or Other Programmes: 12Table 6. Frequency and Timeliness of SFP Food Delivery to School, Headmaster Report 13Table 7. Frequency and Timeliness of Food Delivery to Beneficiary, Child Report 14Table 8. Child Perceptions of School Feeding Biscuits, reported by children and teachers 15Table 9. Children’s Consumption of the Biscuits 16Table 10. Enrollment by Gender, 2002 and 2004 School Years 18Table 11. Seasonal Attendance Rates by Gender, 2001 and 2003 School Years 19Table 12. Annual Drop-out Rates by Year by Grade and by Gender 20Table 13. Repetition Rates by Gender, Aggregated for All Grades 21Table 13a. Repetition by Grade and Gender, 2000-2001 and 2003-2004 School Years: 21Table 14. Average Achievement Test Scores by School at Follow up for Intervention and Control

Schools 22

Table 15. Teacher Reported Rate of Hunger-related Classroom Behaviors, by Gender 23Table 16. School Management Committees: Gender Representation and Participation 24Table 17. SFP Impact on School-based Water and Sanitation Facilities 25Table 18. Programme Side-Effects: Meal Substitution 26Table 19. Programme Side Effects: Classroom Crowding and Student/Teacher Ratio 27Table 20. Programme Side Effects: Teacher Burden and Teaching Time 28Table 21. Impact of SFP Programme on Total Enrollment: Fixed Effects Model 31Table 22. Impact of SFP Programme on Average Annual Attendance: Fixed Effects Model 32Table 23: Impact of SFP Programme on Enrollment: Pooled Model 34Table 24: Impact of SFP Programme on Attendance: Pooled Model 35Table 25 Impact of SFP Programme on Standardized Test Scores 36Table 26 Impact of SFP Programme on Standardized Test Scores: Pooled Model 37

i

Executive Summary In July 2002, the World Food Programme (WFP) and the Government of Bangladesh (GOB) initiated a School Feeding Programme (SFP) targeted to schools in chronically food insecure rural areas and in urban slums in the capital, Dhaka. The programme was intended to distribute fortified biscuits to elementary school children in the targeted schools six days a week during the school year. The biscuits provide 300 kilocalories (about 15% of daily calorie requirements), and a range of micronutrients, contributing about 75% of the daily requirements of vitamin A, zinc, folate, and iron. The goals of the programme were to increase school enrollment and attendance; reduce school repetition and drop out rates, improve attention and learning capacity by reducing short-term hunger, and thereby improve school achievement. The programme also aimed to improve nutritional status and health by reducing micronutrient deficiencies and providing a protein/calorie supplement. The WFP also intended to use the SFP as a mechanism to encourage schools to provide water from tested, arsenic-free tube wells, and to provide separate toilet facilities for girls, as the lack of such facilities was viewed as a potential barrier to girls’ school enrollment. The WFP also hoped to encourage greater participation by women in the School Management Committees (SMC) and more active involvement of the SMC in school activities in general. A baseline survey was conducted in June-August 2002, prior to the implementation of the SFP (see Coates and Hassan, 2002). A representative sample of schools serving low-income communities in both urban and rural areas was chosen; the sample included schools that were to be included in the SFP as well as comparable schools not slated to be included. Data were collected on school characteristics and on key outcome variables (enrollment, attendance, repetition, drop out, teachers’ perception of students’ behavior). In July-September 2004 a follow up survey was conducted in the same schools. One urban area that originally was included among the control schools turned out, at the time of follow up, to have been given the SFP; this area was therefore excluded from the follow up survey, and a replacement upazila selected (but the schools in this replacement turned out to be non-comparable to the other sampled schools). In addition to replicating the data collection from baseline, the follow up survey included information on SFP implementation, teachers’ and students’ perceptions of the programme and use of the food, and qualitative information from parents, government officials, and SMC members about their attitudes toward the programme. The SFP appears to be well implemented. Over 93% of children in intervention schools had received biscuits on the day previous to the interview. Only 17% of schools reported ever having a supply shortfall, and this happened only about once a year. Acceptability of the biscuits was high. Ninety-eight percent of children said the biscuit was ‘tasty’ or ‘very tasty’; 65% said there was nothing they disliked about it, and only 10% said they didn’t like it. Of those who received the biscuits, 98% ate at least some of them in school, and 70% ate them all. Teachers, school officials, and parents had some criticism of the biscuits: that they are too hard, they are the same all the time; the quality is variable. But children’s own reports were more positive. There is no evidence of any gender-based differences in children’s receipt of the biscuits nor in their use of them. There is also no evidence that the SFP is a substitute for children eating

ii

breakfast at home. There was actually an increase in the proportion of children reporting eating a substantial meal before coming to school in the intervention compared with the control schools. Some parents reported that the children eat less at home, and several parents and SMC members said they no longer had to provide money for snacks in school as a result of the SFP. Members of the SMC reported that they increased their involvement in the school as a result of the programme, and that they did not mind doing extra work, because they felt they were doing something valuable. Women reported increasing their activity in the SMC because of their responsibility for monitoring distribution of the biscuits. In bivariate analysis looking at the change from baseline to follow up in intervention compared with control schools, the SFP showed a positive, significant impact on enrollment and attendance, but no impact on repetition and drop out. Presence of the SFP did not affect the availability of arsenic-free tube wells (availability increased in both groups) nor of girls’ toilets. A small increase in classroom crowding was associated with the programme. Teachers reported a significant improvement in classroom behavior: less disruptive behavior, better attention and motivation, and less sleepiness among children receiving the biscuit (but of course these are subjective impressions, and the teachers were aware of whether or not the children were receiving the SFP). Most teachers reported that the SFP increased their work burden and took away from class time, but only about 8% of teachers viewed this as a problem. In a multivariate, fixed-effects model controlling for urban location, whether the school was a government school, crowding, availability of girls’ toilets, and whether or not the school offered a cash stipend, the SFP was associated with a significant increase in attendance of about 5 percentage points. There was no effect on enrollment once urban location was accounted for, and there was no effect on repetition or drop out rates. The pooled analysis (using all schools whether or not they were included in both baseline and follow up surveys) gave comparable results. The SFP showed a consistent, small, positive association with test score, but this association was not statistically significant in any of the models tested. Test score was most affected by urban location, with urban children scoring significantly higher on the achievement tests. Measures of school and teacher quality, including crowding, teacher absenteeism, and days of school closure showed no effect on test score, but schools with higher attendance rates scored very slightly but significantly higher. There were no gender differences observed in the SFP’s effect on enrollment, attendance, repetition or dropout, nor on test score. Overall, the programme is very positively regarded by parents, teachers, SMC members, and government officials concerned with the schools. Suggestions from respondents for improving the programme included extending it to children below grade I, improving biscuit quality control across suppliers, expanding it to more regions of the country, and increasing the quantity of biscuits for older children. These seem to be reasonable recommendations, especially the extension of the programme to younger children attending preschool. In future studies, we would recommend collecting information on dietary and nutrition/health effects of the SFP.

1

I. Introduction In July 2002, the World Food Programme (WFP) and the Government of Bangladesh (GOB) initiated a School Feeding Programme (SFP) targeted to schools in chronically food insecure rural areas and in urban slums in the capital, Dhaka. The programme was intended to distribute fortified biscuits to elementary school children in the targeted schools six days a week during the school year. The goals of the programme were to:

• Increase school enrollment by using the biscuits as an incentive • Increase daily attendance, also because of this incentive • Improve dietary adequacy and thus nutritional status among participants • Reduce short-term hunger during the school day and thereby improve classroom

behavior • Improve cognitive performance and school achievement by reducing short-term

hunger during the school day and by alleviating micronutrient deficiencies that can interfere with cognitive ability.

• Reduce school drop out rate (i.e. students dropping out or repeating a grade). It was felt that drop out rates would fall due to two mechanisms. First, students would be motivated to continue to attend school because of the incentive effect of receiving biscuits in school. Second, students would improve their school achievement because of more regular attendance and because of improved ability to concentrate and learn; this would increase their likelihood of being promoted to the next grade.

The WFP and GOB also intended to use the SFP as leverage to increase the effectiveness of School Management Committees (SMCs) and to increase the participation of women in these committees. The feeding programme was initiated at a time when several national educational initiatives were also under way. The Food for Education (FFE) Programme provided a take-home ration to children from the most disadvantaged families within targeted schools, a transfer conditional on the child maintaining 85% attendance. In 2002, this programme was phased out in favor of a conditional cash transfer programme structured the same way (Ahmed 2004). The FFE and subsequent cash transfer programmes both operate as income transfer programmes providing an incentive to families to send their children to school, but they do not necessarily address the nutritional and short-term hunger constraints to children’s school performance as in-school feeding can. The SFP provides a packet of biscuits daily to each child in a programme school. The biscuits provide 300 kilocalories (about 15% of daily calorie requirements), and a range of micronutrients contributing about 75% of the daily requirements of vitamin A, zinc, folate, and iron. A study conducted in 2003 found that the biscuits accounted for 16.3% of urban children’s caloric intake, and 14.6% of rural children’s intake, offering the second most important source of calories after rice (Ahmed, 2004 p16).

2

II. Study Design and Sample Methods Baseline Survey During May-June 2002, in preparation for the programme’s launch, the WFP and Tufts University undertook a baseline survey (details of the baseline survey design and sample methods are found in Coates and Hassan, 2002). In brief, the study adheres to a classic quasi-experimental survey design, where pre- and post- data were collected on project and non-project schools. Collecting data on the same schools at both baseline and midterm follow up automatically controls for unobservable school-level variables that are fixed over time as well as for secular changes unrelated to the programme that might affect the outcomes. By collecting data from schools that did not receive the biscuit ration as well as from those that did, the midterm evaluation is able to attribute any improvements, beyond those seen in control sites, to the intervention itself. The baseline sample was comprised of 192 schools. In rural areas, intervention schools were randomly selected from districts slated to receive the SFP while control schools were randomly selected from comparable districts (matched on key characteristics such as level of food insecurity) that were not expected to receive biscuits during the first few years of the programme. A similar procedure was followed in urban areas to select intervention and control thanas. Due to resource constraints, the control sample size was limited to one-half of the intervention group, for a total of 64 control and 128 intervention schools. Seventy-five percent of the schools were in rural areas and 25 percent were in urban areas. Two years later, in July-August 2004, WFP and Tufts University returned to the same schools to collect follow-up data for a pre-post evaluation of the impact of the SFP on key programme impacts. In addition to impact indicators, information was also obtained on the implementation of the programme, and children and teachers in programme schools were interviewed to find whether they liked the biscuits and what they did with them (that is, did they eat them all, share them with siblings, save them for later, etc.). Focus group discussions were also conducted with teachers, implementing NGOs, School Management Committees, local government officials, and parents to gain qualitative impressions of people’s perceptions of the programme. Resurvey Sampling Methods

1. Schools In line with the original study design, the intention at the mid-term was to revisit the same schools surveyed at baseline. In order to confirm that schools in the baseline “intervention” group were in fact in the SFP and that all control schools were not receiving WFP biscuits, the baseline sample frame was carefully compared against WFP’s programme records. According to this documentation, all of the schools that were planned for intervention at baseline were indeed receiving WFP biscuits. However, schools from the Gulshan control thana had also joined the intervention. Hence, all schools from this thana were dropped from the follow up survey, and the entire thana was replaced with schools from Motijheel thana. A complete list

3

of government and non-government registered schools in this new thana was obtained from the Directorate of Primary Education and a random sampling procedure was used to draw 16 new control schools to replace the Gulshan control schools that were dropped. This had the unfortunate result that there are no urban control schools that have both pre-implementation and post-implementation data. Furthermore, the follow up sample in the urban control thana of Motijheel was drawn according to different criteria from those used at baseline. Before the programme began, WFP had intended to distribute biscuits in urban areas to only schools serving a low-income population, so the sample of intervention schools included only those serving low-income children. Before the sample was drawn for the new control area, WFP informed researchers that the programme was, in fact, serving all schools in the selected thanas. Therefore, schools were drawn randomly from a complete list of all schools in the thana. As a result, the urban control schools of Motijheel are in general of higher quality and serve a better-off population than the rest of the sample. In the present report, we perform multivariate analysis primarily using the paired data (ie. schools with measures from both time periods, which does not include urban control schools) in order to control for unchanging but unobservable school characteristics and to take advantage of the more powerful paired design. We present, for comparison, a pooled analysis that makes use of all the schools, paired and unpaired, but caution that these results may be misleading because of the non-comparability of the urban control schools in the unpaired sample.

2. Children As one of the objectives of the mid-term evaluation was to assess children’s perceptions of the SFP biscuits, a sample of children was selected from each school. The total number of students to be selected for interview was estimated based on the outcome variables of primary interest. For key indicators such as “short term hunger during school hours”, where comparisons would be made between children in intervention and control schools, the sample size for proportions was calculated using a computerized programme approved by The American Statistical Society for sample size calculation (Dallal, 1986). In this case, calculations were based on the smallest expected difference in proportions of these key indicators between the two independent populations of children in both project and control schools. For this evaluation, this difference was expected to be around 15%, assuming that the proportion of children who experience short-term hunger in project schools is approximately 40% versus 55% in control schools. To be very certain of detecting such a difference if one actually occurred, a power of 80% was specified along with a 95% confidence level to be sure that any differences detected would not be due to chance. The sample size for each of the two groups was estimated to be 186. To account for the loss in sampling efficiency that results from two-stage cluster sampling (the design effect), this figure was multiplied by a factor of 2, leading to a total sample size of 744 (372 children from project and 372 children from control schools). For key indicators that are relevant only to intervention schools (eg. those that assess children’s perceptions and impressions of the programme), the sample size was calculated based on the ability of the study to detect, within a certain level of precision, the proportion of children with these characteristics in the population of children in intervention schools. Because the PC-size

4

programme was not modeled to perform sample size calculation for a single population proportion, the sample size was estimated by using the formula for calculating a 95% confidence interval for a single population proportion, as given below: P ± 2 sqrt [P (1- P)/N] Where: P=population proportion of the indicator of interest N = Sample size Assuming that 50% of the children in intervention schools say they like the biscuits and with the hope of estimating this proportion with a +/- 5% precision, then the sample size is estimated as: 0.05 = 2 sqrt [0.5 (1- 0.5)/N] Which yields N=400. This figure was multiplied by 2 to account for the design effect, giving a total of 800 children to be sampled from intervention schools only. Since the 372 child sample specified by the PC-size calculations, above, are also interviewed about whether they like or dislike the biscuits, the sample of children taken from intervention schools was 800, and the number from control schools was 372, bringing the overall total to 1172 children from both intervention and control schools. Approximately 8 children were randomly selected from different grades for interview until the desired sample size was reached in both intervention and control schools.

Questionnaire Development Structured questionnaires and qualitative guides were developed in a participatory manner with input from the Tufts University FSNSP Principal Investigator, Field Coordinator, and Technical Advisor; from WFP staff involved in the School Feeding Programme; and from Data Analysis and Technical Assistance Inc. (DATA), the local survey research consulting firm that conducted the study and had also conducted the baseline study. The baseline study relied on structured interviews with headmasters and teachers as well as data abstraction from school records to collect information on key impact indicators, school characteristics and other potential confounders. The same structured questionnaire used for the baseline survey was adapted for the midterm evaluation. Additional components were added to assess programme implementation and to capture other factors that might possibly confound the results and make comparison between intervention and control areas difficult. A set of questions was administered directly to children to investigate, among other things, their perception of the SFP and their degree of satisfaction with the quantity, quality, and taste of the biscuits. In order to assess cognitive achievement, as part of the mid-term survey we also administered a standardized grade IV achievement test originally designed by IFPRI (see Ahmed 2002 p. 36) to students of grade V. The test enables post-only comparisons between

5

intervention and control groups. Additionally, qualitative guides for focus group discussions and key informant interviews were developed at mid-term to assess teachers, parents, school management committees, NGOs and upazila education officials’ perception of the programme, constraints to programme implementation and their suggestions for programme improvement.

Field Testing and Training The questionnaire, which was developed in English, was first pre-tested through a three-day formative research trip to seven schools similar to, but not part of, the survey sample. The schools used for the formative research were spread over five upazilas in three districts (Gulshan, Uttara and Shingair upazilas in Dhaka district; Gangni upazila in Meherpur district; and Chuadanga upazila in Chuadanga district). The objective of this trip was not only to inform the development of appropriate survey instruments but also to confirm that information to be abstracted from school records was generally available and complete. Following the formative research trip, the instruments were modified and used in the enumerator training sessions, during which lively and rigorous discussions of each question ensured that any unclear wording was identified and modified. The enumerators recruited to carry out the follow-up study were highly experienced in field survey work. Many were educated to the post-graduate level and all had previously worked on similar studies in Bangladesh. The two field supervisors had also supervised the baseline study and had worked in a recent evaluation comparing the school-feeding programme with the government stipend and food for education programmes. In order to minimize any data collection errors, ample training time ensured that enumerators and supervisors understood the overall objectives of the survey and the specific information that each question was aiming to capture. Practice sessions ensured that all enumerators asked questions correctly and consistently, and that data abstraction was done reliably from the same records. The instruments were translated into Bangla (the local language spoken by respondents) and then back-translated into English, compared to the original version, and corrected for inconsistencies. The questionnaire was then pilot-tested in a different district with similar characteristics to the sample districts to further identify any inadequacies in the questions and minimize unexpected problems that could occur during the actual survey. It was also used as a means of identifying the terms that respondents understood best, determining how long an interview could last, verifying and finding out additional options to the closed ended questions, and eliminating questions that were not considered useful. After each field-test, field coordinators and supervisors sat down with enumerators to debrief, and the resulting information was used to refine the questionnaire further. Fourteen enumerators and two supervisors participated in six days of training and three days of pilot testing during the period of June 29-July 7.

6

Survey Implementation Male and female enumerator pairs carried out the data collection in each school between mid-July and the end of August, 2004. Unexpected early rains that resulted in severe flooding in the Northwest of Bangladesh and in Dhaka, together with a national strike by the teachers of non-government registered schools disrupted the data collection in certain areas and extended the data collection by one month. Enumerators worked under logistically challenging circumstances and, in some instances, were required to return twice to certain schools in order to collect complete information. In spite of these complications, we do not anticipate any effect of the floods or strikes on the quality or accuracy of the data. Enumerators were asked to wait at least a week after schools reopened before returning for an interview so as to allow the resumption of ‘normal’ activity and to avoid the potential influence of school closure on responses related to biscuit receipt etc. Also, the bulk of the data were abstracted from records referring to previous school years that were not affected by these events. Data Entry and Analysis The data were entered and cleaned using Microsoft Access program. All statistical analyses were performed with SPSS for Windows, version 12.0.

7

III. Results (Descriptive) Table 1 shows characteristics of schools in the intervention and control groups at the time the baseline data were collected (2002). Table 1. Key School-level Characteristics by Intervention and Control Areas at Baseline Intervention

(n=128) Control (n=64)

P(t)1

School Type (%) Government 63.3 70.3Non-government registered 35.2 26.6Non-government non-registered 1.6 3.1

Government Rating (%) A-B 85.2 82.8C-D 13.3 14.1None 1.6 3.1

Location

Rural 75 75.3Urban 25 24.7

School Size Total Enrollment (2002) 361 407

Quality of Teaching Staff: Student/Teacher Ratio 34.5 38.6Teacher Schooling (yrs) 17.7 17.0Absenteeism (days per yr.) 10 9.5Time to school (minutes) 23.8 26.3

School Facilities: Students per Classroom 40.7 46.5Toilets (% w/) 88.3 88.9Arsenic-free tubewell on site or nearby (%)

16.4 29.7 *

Tubewell on-site, untested (%) 63.3 46.9 *Flood Vulnerability:

Days closed due to flood last year? 11.7 11.8Days closed due to flood in 1998? 16.9 26.8 *Days closed for flood shelter in 1998? 16.7 27.8

Source: Tufts University/DATA/WFP survey, 2002. 1Independent t-test, *p < .05, **p< .01, ***p< .001

Intervention and control schools were statistically similar on most characteristics, except that control schools were significantly more likely to have an arsenic free tubewell available, and less likely to have an untested tubewell. Control schools were also closed due to flooding in 1998 for significantly more days. Notably, though, there were no differences in measures of teacher quality nor in other facilities (students per classroom, availability of toilets). There was no difference between intervention and control schools in the percent that are located in urban

8

areas, nor in the percent that are government (Gov) versus non-government registered (NGR) schools. There were some striking differences between urban and rural schools, and between Gov and NGR schools, that might affect the impact of the SFP on key outcomes of enrollment, attendance, dropout, repetition, and achievement test score. Table 2 compares the characteristics of urban and rural schools at the time of the follow up survey. Urban schools are much larger in terms of total enrollment, and have a slightly higher percentage of females enrolled. Consistent with their larger size, urban schools have more classrooms (though on average the same number of grades). They have many more students per classroom, even when adjusted for the number of shifts, and are far more likely to provide separate toilet facilities for female students. The student/teacher ratio, which might be expected to affect educational quality, is much higher (that is, worse) in urban schools, but rates of absenteeism are significantly lower, and a much higher percentage of teachers are women. The urban areas have a slightly, but not significantly higher, proportion of Gov schools. For these reasons we adjust for urban/rural location in the multivariate analysis. Table 3 compares Gov and NGR schools. Independent t-tests for differences of means again show distinctions that may affect the impact of the SFP on key outcomes. NGR schools are much smaller in total enrollment, with fewer classrooms (but no difference in students per classroom when adjusted for shifts). NGR schools have teachers with slightly less schooling, and have over twice the rate of teacher absenteeism than Gov schools. NGR schools also show a smaller proportion of female teachers. These are differences that could have important effects on educational quality. Fortunately for our analysis, the intervention and control groups had equal proportions of Gov/NGR and urban/rural schools.

9

Table 2. Comparison of Rural and Urban School Characteristics, 2004 Survey

All schools (n=190)

Rural schools (n=143)

Urban schools (n=47)

P(t)1

School Type and Rating (%) Government 67.4 63.6 78.7 Non-government registered 32.6 36.4 21.3 A-B 90.0 88.8 93.6 C-D 10.0 11.2 6.4

Student Population Total enrollment (#) 357.4 240.4 713.3 *** Female enrollment (%) 48.8 48.2 50.7 *

School Facilities Classrooms (#) 4.4 3.6 6.7 *** Grades (#) 5 5 5 Students per classroom 81.1 72.1 108.5 *** Students per classroom by shift

44.9 40.7 57.7 ***

Toilets (% w/) 90.5 88.8 95.7 Separate girl toilets %w/) 38.4 33.6 53.2 * Arsenic-free tubewell (on site or nearby)

39.5 51.7 2.1 ***

Tubewell on-site, untested (%)

18.9 22.4 8.5 *

Piped water supply 19.5 0.7 76.6 *** Teacher Quality:

Student/Teacher ratio 61.5 56.0 78.1 *** Years of schooling 12.4 12.3 12.5 Absenteeism (days per yr.) 24.0 28.9 16.8 *** Gender (% Female) 53.1 40.5 71.3 ***

Achievement Test Score Average test score, all students

59 54 74 ***

School Management Committee Have SMC (%) 98.9 99.3 97.9 Av.members (#) 10.9 11.0 10.8 * Female members (%) 15.1 13.5 19.8 *** Av. meetings per year (#) 9.9 10.2 9.0 *

Source: Tufts University/DATA/WFP surveys, 2004 1Independent t-test, *p < .05, **p< .01, ***p< .001

10

Table 3 Comparison of Government and Non-Government Registered School Characteristics, 2004 Survey

Characteristic1 All schools (n=190)

Government schools (n=128)

Non-gov. registered

schools (n=62)

P(t)1

Location Rural (%) 75.3 71.1 83.9Urban (%) 24.7 28.9 16.1

Government Rating (%) A-B 90.0 93.8 82.3 *C-D 10.0 6.3 17.7 *

Student Population Total enrollment ’04 (#) 357.4 406.8 255.3 **Female enrollment (%) 48.8 48.9 48.6

School Facilities Classrooms (#) 4.4 5.0 3.1 ***Grades (#) 5 5 5Students per classroom 81.1 79.5 84.6 *Students per classroom by shift

44.9 46.3 42.1

Latrine (% w/) 90.5 92.2 87.1Separate girl Latrine (%w/) 38.4 40.6 33.9Arsenic-free tubewell (on site or nearby)

39.5 40.6 37.1

Tubewell on-site, untested (%)

18.9 16.4 24.2

Piped water supply 19.5 25.0 8.1 **Teacher Quality:

Student/Teacher ratio 61.5 62.3 59.9Years of schooling 12.4 12.6 11.5 ***Absenteeism (days per yr.) 24.0 17.4 44.0 ***Gender (% Female) 53.1 59.5 33.5 ***

School Management Committee Have SMC (%) 98.9 98.4 100.0Av.members (#) 10.9 10.9 11.0Female members (%) 15.1 15.8 13.5 *Av. meetings per year (#) 9.9 9.8 10.3

Source: Tufts University/DATA/WFP surveys, 2004 1Significance of difference between government and non-government registered school characteristics assessed by independent t-test, *p < .05, **p< .01, ***p< .001

11

Presence of WFP School Feeding Programme One of the first priorities was to assess whether the WFP SFP was actually taking place in schools designated to receive the programme, as the first piece of evidence that any impact we observed was likely to be due to the programme (and that any lack of impact was not simply due to the fact that the programme was not actually implemented). In this regard, we were reassured: among intervention schools, 100% reported that they were currently receiving SFP biscuits and that they had been receiving them for an average of 24.2 months (ie. since the programme’s planned inception), while no control schools reported receiving the WFP SFP biscuits (Tufts/DATA/WFP surveys 2004). Presence of Other On Site Feeding We were also concerned that the control schools, having no WFP SFP programme, might have some other in-school feeding programmes that would produce similar effects to the SFP, thereby reducing the measured impact of the WFP programme. We found that about 24% of the control schools did have in-school feeding during the two year period between studies, but in every case the other programme started after the time of the baseline and ended before the follow up survey, so no on site feeding programme was operating during either of our data collection periods. Table 4: Current Participation in On-Site Feeding Programmes

Intervention schools (n=128)

Control Schools (n=62)

All schools (n=190)

(%) Food during school day currently? 100.0 0 67.4Food during the past five years? 100.0 24.2 75.3If yes, programme type:

WFP/SFP Programme 100.0 0.0 67.4Average Duration (months) 24.2 -Other 1.6 24.2 27.4

Type of Food: Fortified Biscuits 100.0 100.0 100.0Milk 1.6 0.0 1.0

Food Recipients Only grades I-V children 94.6 100.0 95.2Both grade I-V and pre-school 5.4 0.0 4.8

Schools with SFP and at least one other programme 1.6 0.0 1.0

Schools receiving fortified biscuits plus at least one other food type 1.6 0.0 1.0

Source: Tufts University/DATA/WFP surveys, 2004

12

We learned from our enumerators that the other programme was a biscuit distribution provided in Fulchhari and Gaibandha by the Relief and Rehabilitation Ministry for a period of six months. In both intervention and control schools, the programme distributed food to all the children in all grades, and in 100% of the schools that had a programme, the food provided was biscuits. Informants reported that they were “special” biscuits, and we infer that this refers to the fact that they are fortified. In just 1.6% of intervention schools, but none of the control schools, milk was also provided as part of on-site feeding. Presence of Take Home Food and Cash and other Programmes Table 5 shows the distribution of schools in the intervention and control groups offering take home food, cash, or other in-school health services. A significant number of schools in both groups provided take-home food as part of the Food for Education programme prior to June 2002, the time of the baseline survey. But the Food for Education Programme was terminated before the WFP SFP began. Table 5, below, includes this programme even though it did not operate during the course of our study. More widespread was the practice of providing a cash stipend to students under the Cash for Education program. Among intervention schools, 70.3% provided a cash stipend of 100 taka per month at the time of the follow up survey, and in control schools, the figure was 75.8%. Many schools offered other health related programs that did not involve the provision of food. There are differences between the two groups of schools in the percentage that offer these services, but these are in the “other” category, and we do not know what services are included. Table 5: Participation in Take Home Food or Cash Stipend or Other Programmes:



All schools (n=190)

Any take home food currently? (%) 0.0 0.0 0.0Any take home food in past 5 yrs (%) 17.2 24.2 19.5Programme type (%):

Food for Education 100.0 100.0 100.0Type of Food:

Wheat 100.0 100.0 100.0Rice 95.5 100.0 97.3

Food Recipients Only grades I-V children 77.3 100.0 86.5Only low income (boys and girls) 95.5 100.0 97.3

Since 2002, any take-home food? (%) 0.0 0.0 0.0Any cash stipend currently? 64.9 70.8 66.8

Average monthly amount (Tk) 100.0 100.0 100.0Non-food health program offered (%)

Health and Nutrition Education 25.0 27.4 25.8Deworming 3.9 6.5 4.7Vitamin A distribution 3.1 1.6 2.6Other 13.3 35.5 20.5


13

Results: School Feeding Programme Implementation Reliability of Delivery and Distribution Among intervention schools, the reliability of the WFP SFP programme appears to have been quite good. Only 17% of headmasters reported that there was ever a shortfall in supply, and said that this occurred only about once per year. Table 6 shows the frequency of supply shortfall and the results of that shortfall. Note that some response options were cut from the table because no one reported them. Among the reasons for supply shortfall, no one reported theft or damaged supplies, and among the consequences, no one reported that they cut the quantity or restricted distribution to boys, to girls, or to low income children in the classroom. Table 6. Frequency and Timeliness of SFP Food Delivery to School, Headmaster Report

Headmaster Report N=Schools

Deliveries per month (#) 1.0 128Ever shortfall (%) 17.0 22Frequency of shortfall (# times per yr) 1.2 22Reason for Shortfall:

Supply did not arrive on time 72.7 16Supply was less than requested 18.2 4Other 9.1 2

Outcome of Shortfall: Did not give any food 73.9 17Used existing stock 4.3 1Gave only to certain grades 8.7 2Other 13.0 3

Source: Tufts University/DATA/WFP survey, 2004

Table 7 shows the children’s report of the reliability of delivery. Over 93% reported receiving the biscuits the previous day, and there is no evident difference between male and female students in their probability of receiving the food. Among the small number of children who reported that there was a day in the past week when they did not receive biscuits (only 7.1% of children responding), shortage of supply and a teachers’ strike that affected some non-government schools were the only reasons frequently cited.

14

Table 7. Frequency and Timeliness of Food Delivery to Beneficiary, Child Report Child Report N=Children

Received biscuits yesterday: % Male students 93.4 423% Female students 92.9 417% All students 93.1 840

Biscuits received (#) 8.3 840(% of recipients)

Received in a package? 100 839Time received from teacher

Upon arrival to school 1.1 10Just after name call 21.0 189During Tiffin break 40.1 361Just before leaving school 17.5 158Other 13.4 121

Any day not received in prior week? 7.1 64Why not received?

Shortage of supply 11.3 7Teacher’s strike 66.1 41Don’t know 22.6 14

Source: Tufts University/DATA/WFP survey, 2004

Acceptability and Consumption of the Biscuits The acceptability of the biscuits among the children receiving them appears to be high. Among children who received the biscuits the previous day, 98% said the biscuit was ‘tasty’ or ‘very tasty,’ and when asked what they liked about the biscuit, over half said the biscuit ‘tastes nice.’ Over 65% of children said there was nothing they disliked about the biscuit. In focus groups, we heard a number of complaints about the fact that the biscuits might be too hard for children to eat easily, but only 16% of children referred to this as a problem. About ten percent of children said they didn’t always like the taste. Table 8 shows the children’s report of their perception of the biscuits, and also the teachers’ perception. Note that 42% of teachers said the children found the biscuit too hard, while only 16% of children reported this. Among teachers, 42% said the children thought biscuits tasted bad, while under ten percent of children reported feeling this way. Overall, acceptability of the biscuits, as reported by the children who consume them, appears to be satisfactory. Possibly the hardness of the biscuits is not an issue because almost all the children who consumed the biscuits reported that they drank water at the same time.

15

Table 8: Child Perceptions of School Feeding Biscuits, reported by children and teachers Child Report (% yes) N=901 students of 128

SFP schools Child received biscuit yesterday 93.1 839Consumption and Satiety

Biscuits received were enough to feel full 91 763Feel full after finished eating biscuits 88 736Children wanted more 15 130Drank water with biscuit 98 821

Taste Really tasty+tasty 98 826Not Tasty 1.5 13

Likes: Biscuit is nutritious 34 285Nice packaging 4 30Biscuit is soft 6 54Biscuit has nice taste 55 462Other 1 8

Dislikes: Biscuit is always hard 5 38Biscuits is sometimes hard 12 103Biscuit is always round 6 50Biscuit is sometimes spoiled 3 26Taste is sometimes bad 8 65Taste is always bad 2 18Dislike packaging 0 2Dislike nothing about biscuit 65 547Overall, biscuit is always the same 3 28Other 4 34

Teacher Report % N=492 teachers from 128 schools

% Teachers reporting child biscuit complaints

53 262

Of these, % teachers saying children complained that:

Biscuits are too hard 42.0 110Broken biscuits 14.9 39Bad taste 42.7 112Damp biscuits 14.9 39Insect infested biscuits 7.3 19Quantity is not enough 1.1 3Biscuit is moldy 0.0 0Shape is always round 4.6 12Other 20.2 53


16

When teacher focus groups were asked whether they had suggestions for programme improvement, many teachers suggested making the biscuits more palatable and tasteful. They wanted better quality control so biscuits are not broken, burnt, hard, or blackened. Some teachers suggested that the shape and color of the biscuits should vary (eg. English letters or flower shapes). There were many comments from teachers and SMC members about the fact that the biscuits never varied. Another suggestion was to increase the biscuit quantity with the age of the student. Among the teachers and SMC members, several noted that the youngest children in the school (those below grade I) are not eligible to receive biscuits, and suggested that biscuits should be available to these children. Most children (97.4%) eat at least some of the biscuits at school, and about 70% eat them all (See table 9). Of the children who did not eat all their biscuits at school, two thirds gave the rest to younger siblings, and most of the rest shared them with other family members. About 90% said that the biscuits were sufficient to make them feel full, though 15% said that they would like to receive more. There were no differences between girls and boys in the likelihood of consuming all the biscuits nor in what they did with the biscuits not consumed by themselves. Table 9: Children’s Consumption of the Biscuits

Child Report % N

Ate any of the biscuits: % Male students 97.4 412% Female students 97.4 406% All students 97.4 818

Ate all of the biscuits: % Male students 70.6 298% Female students 69.1 288% All students 69.8 586

Of those not eating all at school Gave them all to younger sibling 66.9 168Gave them all to older sibling 6.0 15Shared them with family 29.5 74Shared them with non-family 1.6 4Gave them all to non-family member 0.8 2Saved them to eat another day 0.4 1Other 0.4 1

Source: Tufts University/DATA/WFP surveys, 2004 Attitudes Toward the School Feeding Programme

We conducted focus group discussions with groups of parents, teachers, and members of the SMCs. In general, these groups reported a high level of appreciation for the programme. Despite some criticisms of the biscuits, parents mentioned a wide range of benefits they perceived from the SFP, including increased motivation of children to attend school, increased energy and attention while at school, and willingness to stay for the whole school day. Parents also mentioned that the biscuits eliminated the need to give their children money for snacks,

17

though some parents said they still give money for fruit. They noted that there were no problems of the biscuits being diverted, stolen, or spoiled. SMC members noted similar benefits: reduced in-school hunger, better attendance, attention, and memory, and reduced drop out. Government officials (mostly Thana Education Officers) concurred that attendance and learning capacity have improved. All three groups (parents, SMC members, and government officials) noted that children’s health had also improved, with fewer intestinal and skin diseases and fevers, less acute hunger, and weight gain among the children. As a reflection of the high regard for the programme, SMC members’ suggestions included expanding the programme to the whole country, from just a few districts. All groups had suggestions for improving the programme. Parents suggested providing milk or water with the biscuit to make it easier to consume, and suggested that the programme provide greater variety---different types of biscuits, or even other kinds of food. Parents and SMC members suggested increasing the quantity for older children, and SMC members suggested extending the programme to children below grade I. There were numerous comments about the importance of monitoring the quality of the biscuits and the need to make efforts to improve it, including selecting only factories whose quality is reliable. From an administrative point of view, there were several suggestions. Government officials and SMC members mentioned the difficulty of using central (“mother”) schools as distribution points for other schools, particularly when transportation is disrupted in the rainy season. All groups suggested that each school should have a store-room for the biscuits, so they do not need to be stored in the headmaster’s house or other location. (However no one suggested that the biscuits were being diverted or misused.) SMC members said that their level of activity and their participation in school has increased. Some parents mentioned that they would like to have opportunities to assist in the program, but they hadn’t been invited to do so. Generally, the comments show a very positive attitude toward the programme and a desire to see it continued and expanded.

18

IV. Results: Program Impact (Descriptive Bivariate Analysis) Impact Indicators: Enrollment

Enrollment would ideally be calculated as a percentage of all children eligible to attend school. However, it is not possible, in these areas, to determine the precise catchment area of each school, and therefore it is not possible to know what the appropriate denominator would be for computing an enrollment rate. We report here total enrollment, at baseline and follow up, for the intervention and control schools. Implicitly, we assume that no significant changes in age-eligible population occurred between 2002 and 2004, and that if there were any such changes, they would be the same in the intervention and control areas, so that differences between intervention and control in the change in enrollment pre/post are attributable to the programme. Table 10 shows enrollment by gender for the intervention and control schools at baseline and follow up.1 In the intervention schools, there was no significant change in enrollment in the two periods. Without a control group, this result might be interpreted as a lack of any programme effect. However, in the control schools, enrollment declined significantly for both girls and boys (though more for boys). Because the treatment schools showed no change, and the control schools showed a decline, the difference between intervention and control schools in the change in enrollment is in fact quite significant, showing that the biscuits appear to have had a favorable impact on enrollment compared to schools without the SFP. Table 10: Enrollment by Gender, 2002 and 2004 School Years



Intervention %∆ – Control

%∆2

Post Pre %∆1 Post Pre %∆1

Male student enrollment 188.5

184.3

3.6

124.7

145.7

-13.1***

16.8***

Female student enrollment 185.7

184.7

0.1

115.4

133.7

-9.8***

9.9***

Total student enrollment 368.1

362.0

1.5

240.1

279.4

-11.9***

13.4***

Source: Tufts University/DATA/WFP surveys, 2002 and 2004 1Paired t-test, *p < .05, **p< .01, ***p< .001 2Independent t-test, *p < .05, **p< .01, ***p< .001

Impact Indicators: Attendance

The SFP was intended to increase attendance by providing an incentive to children (and their parents) for attending school each day. Attendance figures were collected for four months of

1 Note that because these are pre/post comparisons, they include only the schools that were in the sample at both baseline and follow up.

19

the year: April, May, August, and September, in order to capture any seasonal changes in the effect of the programme on attendance. As was noted in the report on the baseline survey (Coates and Hassan, 2002), the expected seasonal fluctuations in attendance were not observed at baseline, and this was the case for the follow up survey as well. Table 11 shows the baseline and follow up attendance figures for intervention and control schools, and shows the significance of the difference in the pre/post change between the two groups. Attendance rates increased between baseline and follow-up in all months, and for both genders in the treatment schools, and these differences were highly significant (p< .001). Increases in attendance were consistently very slightly higher for girls than boys, but these differences do not appear to be significant. In the control schools, small increases in attendance were observed, but in only a couple of cases were the differences statistically significant. Comparing the level of pre/post change between SFP schools and those without the programme, treatment schools consistently showed greater increases in attendance; these differences were significant in three of the months studied, as shown in the table.

Table 11: Seasonal Attendance Rates by Gender, 2001 and 2003 School Years Intervention schools

(n=122) Control Schools

(n=46) Intervention ∆– Control ∆2

Post Pre ∆1 Post Pre ∆1

April Male Female All students

78.2 80.5 79.4

70.771.170.7

7.6*** 9.4*** 8.6***

75.9 77.2 76.5

73.8 71.8 72.8

2.1 5.3*** 3.7*

5.5** 4.0 4.9*

May Male Female All students

77.5 79.4 78.5

71.272.271.5

6.3*** 7.3*** 6.9***

74.1 75.2 74.7

74.0 72.1 73.1

0.1 3.1 1.6

6.2** 4.2* 5.3**

August Male Female All Students

76.8 78.6 77.7

70.571.470.8

6.3*** 7.3*** 6.9***

72.5 74.0 73.2

71.4 70.3 70.8

1.1 3.6* 2.3

5.2* 3.6 4.6*

September Male Female All students

74.8 76.4 75.5

69.170.269.5

5.7*** 6.3*** 6.1***

73.0 73.9 73.4

71.2 70.4 70.7

1.8 3.4 2.6

3.9 2.8 3.4


Outcome Indicators: Dropout Rate

Drop out rates fell significantly between the baseline and follow up surveys in both the Intervention and the Control schools, as shown in Table 12 below. There was no difference between the Intervention and the Control schools in the change in drop out rate: the declines were not significantly different between the two groups.

20

Table 12: Annual Drop-out Rates by Year by Grade and by Gender Intervention Schools

(n=124) Control Schools (n=47) Intervention ∆– Control ∆2

Grades Sex Post Pre ∆1 Post Pre ∆1

Grade I-II Total 4.6 16.0 -11.6*** 5.2 19.1 -13.8*** 2.2

Boy 4.6 16.5 -11.9*** 4.9 17.2 -12.3*** 0.4

Girl 4.6 15.6 -11.0*** 5.7 21.2 -15.5*** 4.5

Grade II-III Total 5.1 11.2 -6.3*** 7.2 11.1 -3.9 -2.4

Boy 5.3 11.5 -6.2*** 6.5 10.1 -3.5 -2.7

Girl 4.6 10.8 -6.3*** 7.7 12.7 -5.1 -1.2

Grade III-IV Total 5.6 17.1 -11.5*** 7.4 13.9 -6.4** -5.1

Boy 6.0 17.4 -11.4*** 8.5 14.4 -5.8* -5.6

Girl 5.1 17.1 -12.1*** 7.6 14.1 -6.6** -5.5

Grade IV-V Total 7.8 13.8 -6.0*** 10.3 13.4 -3.0 -3.0

Boy 8.4 13.1 -4.8** 10.1 14.4 -4.3 -0.5

Girl 7.3 15.6 -8.1*** 10.7 14.3 -3.6 -4.5 Total (Grade I-V) Total 5.5 14.6 -9.1*** 7.1 14.7 -7.6***

-1.5

Boy 5.7 14.7 -9.1*** 6.9 14.1 -7.2** -1.9

Girl 5.3 14.7 -9.5*** 7.2 15.4 -8.1*** -1.4 Source: Tufts University/DATA/WFP surveys, 2002 and 2004 1Paired t-test, *p < .05, **p< .01, ***p< .001 2Independent t-test, *p < .05, **p< .01, ***p< .001. Note: No figure in “Intervention ∆ – Control ∆2” column is significant. Only differences for Grade III – IV are significant at 10 percent confidence level. Outcome Indicators: Repetition Rates

Repetition occurs when a student is held back and has to repeat a grade. Table 13 shows the repetition rates by gender for all grades and Table 13a shows the repetition rate by gender disaggregated by grades. Repetition rates actually increased notably between the baseline and follow up surveys for both the intervention and control schools, but in the intervention schools, repetition increased by a smaller amount than in the control schools, showing a positive impact of the SFP. There is no evidence of gender-based differences in repetition rates, nor does it appear that there is a substantial difference by gender in the effect of the SFP. It is true that the change in the rate of repetition between baseline and final was significant for boys and not girls, but the magnitudes are quite similar.

21

Table 13: Repetition Rates by Gender, Aggregated for All Grades Intervention schools

(n=124) Control Schools (n=47) Intervention ∆

– Control ∆2


Male student repetition rate 12.9 8.1 4.9*** 13.0 4.1 8.9*** -4.0*

Female student repetition rate 12.9 7.4 5.5*** 13.0 4.2 8.8*** -3.3

Total student repetition rate 12.9 7.7 5.2*** 13.0 4.1 8.9*** -3.7*


Table 13a: Repetition by Grade and Gender, 2000-2001 and 2003-2004 School Years: Intervention Schools

(n=124) Control Schools (n=47) Intervention ∆– Control ∆2

Repetition in Sex Post Pre ∆1 Post Pre ∆1

Grade I Total 15.2 6.6 8.6*** 17.9 3.0 14.9*** -6.3** Boy 15.0 6.5 8.6*** 17.5 3.0 14.5*** -5.9

Girl 15.4 6.9 8.5*** 18.6 3.3 15.3*** -6.8* Grade II Total 11.1 6.4 4.8*** 10.1 4.0 6.1** -1.3 Boy 11.2 7.1 4.3** 10.0 4.0 5.9** -1.6 Girl 11.1 5.6 5.6*** 10.3 4.2 6.1* -0.5 Grade III Total 14.4 10.3 4.0** 13.2 4.9 8.3*** -4.3 Boy 14.5 9.9 4.8*** 12.9 4.9 8.0*** -3.2 Girl 14.1 9.6 4.6** 13.1 5.8 7.4*** -2.8 Grade IV Total 11.6 10.2 1.5 10.9 5.8 5.1* -3.6 Boy 11.1 10.5 0.8 11.0 5.5 5.5* -4.7 Girl 11.9 9.0 3.0 10.5 6.9 3.7 -0.7 Grade V Total 6.2 10.5 -4.4** 4.5 5.6 -1.1 -3.3 Boy 7.6 6.8 0.8 4.7 3.8 0.9 -0.1 Girl 5.1 6.6 -1.5 4.2 3.1 1.1 -2.6 Grade I-V Total 12.9 7.7 5.2** 13.0 4.1 8.9*** -3.7* Boy 12.9 8.1 4.9*** 13.0 4.1 8.9*** -4.0* Girl 12.9 7.4 5.5*** 13.0 4.2 8.8*** -3.3


22

Outcome Indicators: Achievement Test Scores

All students in Grade V in our sample were given a standardized achievement test covering proficiency in Bangla, English, and mathematics (see Ahmed 2002). One of the goals of the SFP is to improve test scores through both increased attendance and improved capacity to learn. Table 14 shows the test scores (averaged for each school for female students, male students, and all students) separately for programme and non-programme schools. (Note that there are some all male and some all female schools, so the numbers are not the same in each column.) While the scores are consistently very slightly higher in the schools that offer the biscuit, none of the differences is significant, and the scores are in fact very close.

Table 14. Average Achievement Test Scores by School at Follow up for Intervention and Control Schools Intervention Schools Control Schools

% correct N % correct N Prob (t)

Male Students 62.7 127 61.6 56 ns Female Students 56.1 122 52.2 57 ns All Students 59.7 128 57.4 61 ns Source: Tufts/DATA/WFP Survey 2004

Outcome Indicators: Classroom Behavior

In the baseline survey, teachers were asked to comment about the proportion of children in their classrooms who displayed behaviors that might be associated with suffering from hunger, specifically: whether students appeared sleepy in class, whether they were disruptive, and whether they had difficulty concentrating. Teachers were asked a general question about the proportion of children who usually displayed the particular behavior. This kind of subjective reporting is not always reliable when captured quantitatively. It is well known that such reporting may reflect a general attitude of a particular teacher as much as it might reflect the actual behavior of children in the classroom. Yet such subjective assessments are widely used in studies of the effect of school feeding, presumably because of the logistic challenge and the expense of trying to develop both measurable indicators and a feasible strategy for blinding the teacher to the short-term hunger status of the children. In any case, data were collected at baseline and follow up. Table 15 shows the responses of teachers to the question, what proportion of students display each of the behaviors. In the control schools, teachers reported an increase in the proportion of children who seemed to be sleepy during class, and this increase was significant for both boys and girls. In the schools offering the biscuit (the intervention schools), teachers reported a small but significant decrease in the proportion of children who appeared to have difficulty concentrating in class. The difference in the change between the two groups of schools is significant and favorable for intervention schools. There is also a significant improvement in reported disruptive behavior in the intervention schools compared to the control schools, at least overall and for boys. Note

23

that although there is no way to link these changes to the actual consumption of the biscuits, since Table 9 has demonstrated that virtually all children in the intervention schools receive and consume the biscuits, we may be confident that in the intervention schools, the children described by the teachers’ report actually did consume the food. Table 15: Teacher Reported Rate of Hunger-related Classroom Behaviors, by Gender Intervention schools


– Control ∆2


Sleepy % Male students % Female students

% All students

10.5 9.4 9.9

9.1 7.9 8.5

1.4 1.4 1.4

15.6 15.1 15.3

7.9 7.1 7.4

7.6*** 8.0*** 7.9***

-6.2** -6.6*** -6.5***

Disruptive % Male students % Female students

% All students

19.2 12.3 15.7

19.9 11.9 15.8

-0.7 0.4 -0.1

22.4 15.5 18.8

20.1 11.9 16.1

2.3 3.5 2.7

-3.0 -2.9 -2.8

Difficulty Concentrating % Male students % Female students % All students

19.4 17.0 18.1

22.5 19.2 20.6

-3.1* -2.2 -2.6*

22.4 22.9 22.6

24.5 19.6 22.1

-2.1 3.3 0.5

-1.0 -5.5* -3.1


It is probably unwise to put too much credence in these quantitative results alone, because of the problem of subjective reporting noted above. In order to strengthen our understanding of the relationship between hunger, eating breakfast, and classroom behavior and to triangulate the results reported, above, focus groups of teachers were asked several questions related to this issue. For instance, teachers were asked to describe the behavior of children who came to school without breakfast compared with those who came after eating. Kids who ate breakfast were considered to be “in good mind, “fresh and cheerful”, “delighted”, “attentive to studies”, “physically well” and “not sleepy”. By contrast, children who generally did not eat breakfast were described as “inattentive”, “often ill”, ”feeble”, “dispirited”, and “tired”. Teachers were also asked to describe whether they had noticed any changes in their children’s behaviors since the start of the SFP. Almost all the teachers interviewed said that the children’s attentiveness and interest in their studies had increased dramatically since they began to receive biscuits. Before the programme they were said to be “inattentive”, “sleepy”, and “depressed”. They were often sick. According to the teachers the kids now seem “cheerful and fresh”, their “strength, merriment, and cheerfulness are multiplied”. Teachers noted that kids are not as sleepy, their health condition seems good, they are more interested in reading and writing, and there are fewer quarrels in the classroom. One teacher mentioned, “now they

24

are so excited that it is difficult for us to pacify them,” and another said ”their interest in school has increased so much it is difficult to imagine.” While these teachers were obviously not blinded to whether their school was in the intervention or not (i.e. knowing their students received biscuits may have influenced their responses), all of the focus group discussions seemed to relate consumption in biscuits to decreases in hunger during the school day and to improved classroom behavior that was more conducive to learning. Parents too reported that their children seemed to be more motivated and enthusiastic about going to school once the biscuit was introduced, and that the children were more energetic and less sleepy. And they reported that their children were in better health, with fewer illnesses including rashes and fevers. SMC members also commented that children were more energetic, less likely to be sleepy, and more attentive in class. School Management Committees

One goal of the WFP in implementing the SFP was to activate School Management Committees (SMCs) and to promote women’s participation in these groups. In each school, the SMC consists of the headmaster and respected individuals from the community. The Committees are responsible for operational decisions regarding the school. Table 16 shows female membership and meeting attendance rates of male and female members for both groups of schools at baseline and follow up. There was a small but significant increase in female membership in the SFP schools, but this increase was not significantly greater than the (smaller) change among the control schools. Attendance rates differed only slightly by gender, with women attending a bit less frequently than men. But there was no significant difference in attendance rates, neither from baseline to follow up nor between treatment and control groups. Attendance is in fact fairly high at about 65% for women and about 75% for men. At the time of the baseline survey, committees were generally meeting eight or more times per year; at follow-up the average number of meetings was 9.8.

Table 16: School Management Committees: Gender Representation and Participation Intervention schools

(n=124) Control Schools

(n=45) Intervention ∆

– Control ∆Post Pre ∆1 Post Pre ∆1

Female members (%) 15.0 13.5 1.5 13.0 12.8 0.2 1.22

Female member meeting attendance rate

64.0 68.9 -4.3 64.0 62.1 1.8 -6.1

Male member meeting attendance rate

73.0 75.2 -2.5 75.4 77.2 -1.8 0.7

Overall meeting attendance rate

68.8 72.1 -3.4 69.7 69.7 0.0 -3.4

Source: Tufts University/DATA/WFP surveys, 2002 and 2004 1Paired t-test, *p < .05, **p< .01, ***p< .001 In focus group discussions, SMC members said that the introduction of the SFP had increased their sense of participation in the school. They said that members are more interested because of a sense of purpose; that attendance has improved; and that they do not

25

mind the extra work because they find their involvement in the programme rewarding. They also reported an increase in female participation because it is the female members who monitor biscuit distribution and management. Water and Sanitation Facilities

One objective of the SFP was to serve as an incentive for schools to upgrade their sanitation facilities. Schools that were offered the SFP were encouraged to provide separate latrines for girls, on the assumption that lack of such facilities might be a barrier to girls’ attending school. As shown in Table 17, there was in fact a significant increase in the percentage of schools providing separate toilets for girls, while there was no change at all among the control schools. The availability of tubewells, and of tubewells certified to be arsenic-free, increased substantially in both the treatment and the control schools over the period of the study, but there are no apparent difference based on the availability of the SFP.

Table 17: SFP impact on school-based water and sanitation facilities Intervention schools

(n=125) Control Schools (n=47) Intervention ∆1

– Control ∆2

Post Pre ∆1 Post Pre ∆2 ∆1 - ∆2

Latrine (% w/) 92.0 88.8 3.2 83.0 85.1 -2.1 5.3

Separate girl toilets (%w/) 37.6 24.8 12.8** 34.0 34.0 0.0 12.8**

Arsenic-free tubewell (on site or nearby)

40.0 8.8 31.2*** 51.1 23.4 27.7** 3.5

Tubewell on-site, untested (%) 20.0 0.0 20.0*** 19.1 0.0 19.1** 0.9

Piped water supply 18.4 18.4 0.0 0.0 0.0 0.0 0.0


Other Programme Effects: Substitution of Meals

We were concerned that the SFP, by providing a snack in school, might discourage parents from providing their children with a meal before coming to school. In both rounds of data collection, children were asked if they ate ”pet pure khaoa” (local term for a meal that is filling or substantial) before school on the day of the interview. While Table 18 shows no change between the baseline and follow up in the control schools, children in the treatment schools were significantly more likely to have eaten before school after the implementation of the SFP. It is difficult to account for this difference, other than to think that some nutrition education or other sensitization to the importance of eating well occurred in the context of the program. It is reassuring to note that there is no evidence that the SFP biscuits are substituting for home consumption of family food.

26

Table 18: Programme Side-Effects: Meal Substitution Intervention schools

(n=125) Control Schools (n=47) Intervention

∆–Control ∆2


Substitution Ate full stomach meal before school

43.2 33.2 9.9*** 37.3 34.3 3.0 6.9**


In the focus group discussions, parents reported that they try to give their children the same amount of food as they did before the programme started. But some parents reported that the children ate less at home after eating the biscuits at school. An important substitution is that some parents reported they no longer needed to give their children 1-2 taka to purchase food at tiffin break, representing a cash saving to the family. A few parents said that their children stopped eating breakfast at home once the biscuit was introduced, but the quantitative data show that in fact, the likelihood of eating breakfast was higher after introduction of the biscuit than before, an increase not observed in the control schools. Other Programme Effects: Classroom Crowding and Student/Teacher Ratio

Another possible side effect of the SFP is that, by encouraging greater enrollment and attendance, the programme might result in more crowding in the schools and more burden on teachers, with possible negative consequences for educational quality. Looking just at the schools where the SFP was implemented, there was no change in the level of classroom crowding, nor in the student/teacher ratio (See Table 19). In the control schools, though, the student/teacher ratio actually fell significantly between baseline and follow up, possibly because (as we have seen) total enrollment fell in the control schools. Thus the SFP schools did show a difference in the ratio of students to teachers, but not because the situation worsened in the SFP schools, only by comparison with the declining ratio in the control schools. There was one comment among SMC members in the focus group discussions that increases in attendance due to the SFP had increased classroom crowding in the intervention schools. This may be an isolated case, though, since the quantitative data show no change in crowding in the intervention schools (though they showed relatively more crowding in the follow up period than the control schools).

27

Table 19: Programme Side Effects: Classroom Crowding and Student/Teacher Ratio Intervention schools


– Control ∆2


Student/teacher ratio 64.4 67.4 -3.0 56.5 71.1 -14.5*** 11.5***

Student/teacher ratio by shift 35.7 34.8 1.0 33.3 39.8 -6.5*** 7.5***

Student/classroom ratio by shift 44.3 41.2 3.0 45.2 45.4 -0.2 3.2


Other Programme Effects: SFP Effect on Teaching

A concern about the SFP is that it might have an effect on teaching time in school. Table 20 shows the teachers’ perceptions, from SFP Schools, on the effect that the programme had on their class time. What is most noticeable in these reports is that many teachers acknowledge that the programme affects their teaching, and over half report that total teaching time is reduced, but only about 8% of teachers believe that the disruption is sufficient to be considered a problem. Either the amount of time lost is fairly small, or the teachers perceive the benefits of the SFP to offset any negative effects of classroom time lost to the programme.

The latter hypothesis is supported by the teacher focus group discussions, in which the teachers were overwhelmingly in favor of the programme and its benefits. Teachers claimed that the SFP and the cash stipend programmes have had a huge impact on attendance, and that children no longer need to be forced to attend since they come on their own. They like that children are more attentive since receiving biscuits, that more children enroll, that the guardians are more aware of school importance, that children’s health seems to improve, hunger is reduced, and kids “can become more intelligent”. Other teachers mentioned that the vitamins from the biscuits have helped children to be interested in their education, and that kids no longer leave school at tiffin period without returning. Some Thana Education Officers mentioned that biscuit distribution and consumption cut into class time, both because of teacher administrative requirements and because of the time children take to eat the biscuit, get water to drink, and leave to go to the WC. But teachers report (and SMC members in focus groups confirm) that the teachers do not mind the disruption because they recognize educational benefits of the programme.

28

Table 20. Programme Side Effects: Teacher Burden and Teaching Time Teacher Report (% yes) N=492 teachers of 128

SFP schools

SFP affects teaching work 40.7 492Biscuit distribution disrupts lessons

25.0 50

Increased record keeping 5.5 11Reduce teaching time 57.5 115

Students leave to drink water 44.5 89Students leave to urinate 13.5 27Other 3.5 7

% teachers considering above to be problematic

7.7 38


29

V. Results: Program Impact (Multivariate Analysis) In the descriptive presentation above, we observed apparent effects of the SFP on total enrollment, attendance, and a small effect in reducing the drop out rate. It was noted that the schools that received the SFP and those in the control group were quite comparable in terms of urban/rural location, proportion of government/non-government schools, proportion of schools with higher versus lower rankings, and indicators of teacher and school quality. Still, in order to be sure that any observed effect is actually due to the programme, and not to other differences between the treatment and control schools, we use regression analysis to test for the effect of the SFP on five key outcome indicators -- enrollment, attendance, drop-out, repetition, and test score – controlling for other factors that might influence these outcomes. In the implementation of the SFP evaluation study, schools were assigned to intervention and control groups chosen to be comparable in all ways. However, when the SFP was implemented, all the urban schools originally assigned to the control group were actually provided with the SFP. As a result, if we take advantage of the panel nature of the data, with the same schools measured at baseline and follow up, there are no urban schools in the control group with which the treatment group can be compared. In the follow up survey, we added a sample of urban schools that had not received the SFP, to substitute for the urban “control” schools that actually received treatment, but of course we have no baseline data for these schools. This matters, because as was noted above, there are significant differences between urban and rural schools in terms of size, classroom crowding, student/teacher ratio, teacher absenteeism, availability of separate toilets for girls, and a number of other factors that might affect the outcome indicators. For this reason, we present in this report two sets of estimates, one taking advantage of the paired nature of the data, that looks at pre/post differences by school as a function of changes in other factors during the study period, and the second using data from the baseline and follow up samples, pooled. The pooled analysis makes use of all the schools in the sample, but treats the baseline and follow up samples as though they were independently chosen and not matched. (Recall that the urban control schools in this unmatched sample are better off and serve a more affluent population that the schools that were in the sample at baseline.) V.1 Fixed Effects Estimates The following analyses make use of the paired data set, including all schools for which we have both baseline and follow up data. The model used is a “difference in differences” or fixed effects model, in which we estimate the impact on pre/post change in the outcome indicator of changes in other factors that might influence the outcome: This fixed effects approach has the advantage that it controls for any unobserved, time-invariant characteristics of the schools that might affect the outcomes. The model looks at changes between baseline and follow up; unobserved characteristic that do not change over time, while they might affect the outcome variable, do not affect its change.

30

The general form of the model is as follows:

∆ outcome = F(programme, ∆ cash stipend, ∆girls’ toilet, ∆crowding, urban, government) where: ∆ Outcome measures the change in enrollment, attendance, dropout or repetition rate

between baseline and final. (A positive score indicates an increase; a negative score indicates a decrease.)

Programme is a dummy variable indicating the school was receiving the SFP at follow-up

∆ cash stipend indicates whether the school changed in offering a cash stipend as an enrollment or attendance incentive (a positive score means the programme was added between baseline and final; a negative score means the programme existed at baseline but was withdrawn. A zero score means the school did not change between baseline and final.)

∆ girls’ toilet indicates whether the school changed in making separate toilets available to girls.

∆ crowding measures the change in classroom crowding between baseline and follow up. (An increase in crowding is a positive score; a decrease is negative)

Urban indicates the school is in an urban area Government indicates the school is a government school

Two of these variables, availability of girls’ toilets and classroom crowding, were expected to be affected by the provision of the SFP, and indeed, we noted above that both these variables were influenced by the programme: availability of girls’ toilets increased, and classroom crowding also increased after the implementation of the SFP, compared to the control schools. For this reason, we estimated the impact of the programme first without these intervening variables, and again including them. Note that the crowding variable poses measurement problems: crowding is computed as enrollment divided by the number of classrooms, divided by the number of shifts. As total enrollment rises, if classrooms and shifts do not change, crowding is bound to increase. An econometric problem with this variable is that enrollment is included in the calculation of crowding and of attendance rate (calculated as attendance as a percentage of enrollment on the interview date). Any errors in the measurement of enrollment, therefore, are also present in the computation of crowding and attendance. As a result, the correlation of the error terms on the right and left hand sides of the equation may artificially inflate the significance of the crowding variable in the enrollment and attendance estimates. We planned to include another potential confounding factor: the availability of other on-site feeding (other than the WFP SFP). Inspection of the data showed that no school had any other on site feeding programme operational at the time of data collection, either at baseline or follow up, though a few schools had an on site feeding programme for a few months during the study period.

Impact Indicators: Enrollment

Table 21 shows the results of the model estimation for total enrollment. As we saw in the earlier descriptive tables, schools with the SFP showed an increase in enrollment that is

31

statistically significant, but when urban location is included in the model, it accounts for much of the programme’s impact on total enrollment. When the analysis is restricted to rural schools only, availability of the SFP is associated with a statistically significant increase in enrollment of 24 students. Adding the presence of girls’ toilets and crowding to the model adds little to its explanatory power, but also does not reduce the estimated effect of the SFP. It remains significant in rural schools, and of similar magnitude, though not statistically significant, in the whole sample. It seems that the SFP does result in an enrollment increase in rural areas compared to control schools. (Recall that enrollment declined in control schools, so in fact the SFP appears to have maintained enrollment over the period of the study.) Urban intervention schools showed a large increase in enrollment over time, but even after controlling for urban location, the programme showed an effect on the change in enrollment of about 25 students (marginally significant at p=.10). Table 21 Impact of SFP Programme on Total Enrollment: Fixed Effects Model Dependent Variable=Enrollment

All Schools

(1)

All schools

(2)

All schools

(3)

Rural schools

(4)

Rural schools

(5) Programme area=1

21.30 25.11 45.68** 24.02* 20.74

Urban area=1

69.91*** 89.89*** __ __ __

Government School=1

-13.94 -12.38 -5.67 -17.52 -17.37

∆ Offering cash stipend program

18.56 19.21 .58 10.72 9.97

∆ Girls toilets on-site 7.88 __ __ __ 1.86

∆ Students per classroom by shift

1.61*** __ __ __ 1.614***

Intercept (∆Time, pre-post)

-30.91* -34.97* -36.00* -29.73* -26.84*

Model Adj. R2 0.24 0.15 0.03 0.03 0.19

F 9.87 8.53 2.86 2.3 7.62

Sig. F

0.000 0.000 0.39 0.08 0.000

p <.05 ** p<.01 *** p<.001, + p <.10

Models estimated separately for boys’ and girls’ enrollment showed no apparent differences from the models estimated for total enrollment.

32

Presence of girls’ toilets shows no effect on enrollment, even of girls, in any of the models estimated. Crowding shows very small effects on the change in enrollment (about 1-2 students). The consistent statistical significance of this variable is undoubtedly an artifact of the correlated error structure described above. The availability of a cash stipend also showed no significant effect on the change in enrollment over time. This is somewhat surprising, since a substantial proportion of schools in both treatment and control groups added a cash stipend programme between the baseline and the follow up period.

Impact Indicators: Attendance

Attendance rates increased in intervention schools, compared with control schools. The difference in attendance rate increase between treatment and control schools was about five percentage points. This result was quite consistent and statistically significant across models controlling for urban/rural location, and whether or not girls’ toilet and crowding change were included in the model. About the same magnitude of difference in the pre/post change in attendance is shown among rural schools. Table 22 Impact of SFP Programme on Average Annual Attendance: Fixed Effects Model Dependent Variable=Attendance Rate

All Schools

(1)

All schools

(2)

Rural schools

(3)

Rural schools

(4) Programme area=1

5.54** 5.78** 5.36** 5.80**

Urban area=1

-6.22** -6.02** __ __

Government School=1

0.70 0.78 1.91 2.08

∆ Offering cash stipend program

-1.02 -0.10 -1.87 -1.73

∆ Girls toilets on-site -2.65 __ -1.53 __∆ Students per classroom by shift

0.07 __ 0.11 __

Intercept (∆Time, pre-post)

2.73 2.25 2.16 1.51

Model Adj. R2 0.07 0.05 0.07 0.05

F 2.92 3.06 2.91 3.31

p (F)

0.01 0.02 0.16 0.22

p <.05 ** p<.01 *** p<.001

33

No significant effects are observed for the change in availability of girls’ toilets, nor for change in crowding, and the programme impact remains significant in all models. Once again, a change in the provision of a cash stipend did not appear to affect the change in attendance. There were no evident differences in results when run separately for girls’ attendance change and boys’ attendance change. There is no evidence that the programme affects girls and boys differently.

Impact Indicators: Drop-Out We estimated a similar model for the change in drop-out rate, for the total sample, for rural schools separately, and for girls and boys separately. No significant effect of the SFP on drop out rate was observed in any of the models. No variables reached statistical significance, except that in a few cases crowding showed a small negative effect on drop out rate, suggesting that as drop out rates decline, school enrollment increases, and therefore crowding increases. To see if perhaps there was a programme effect in only certain grades, we tested the same models looking specifically at drop-out between grades one and two, and between grades four and five, but the results were similarly uninformative. No significant effect was observed for the SFP nor for any of the other variables tested.

Impact Indicators: Repetition As with drop-out, the multivariate analysis of repetition provided little insight. There was a consistent negative association between the SFP and repetition rate (suggesting the rate of repetition fell more in intervention than in control schools), but the models were quite weak and in most cases statistically insignificant. Once again, the results did not vary between boys and girls, nor did the results change when the analysis was restricted to rural schools.

V. 2 Pooled Analysis To test the robustness of the fixed effects models estimated above, which were run on a sample that included no urban control schools, we ran the same model on all the schools. In this approach, schools are included in the sample with an indicator for whether their data were collected at baseline (time=0) or follow up (time=1). All schools are included, whether they are paired or unpaired, and the samples are treated statistically as independent. This approach loses some of the precision of the paired analysis, but allows us to include control schools from urban areas, to test whether the programme might have different effects in urban and rural areas. We tested the SFP impact on enrollment and attendance. The form of the model estimated in the pooled analysis is as follows:

Outcome = f(time, programme, time*programme, urban, urban*programme, government, cash stipend, girls’ toilet, crowding)

34

Where Time indicates whether data were collected at baseline or followup and provides an

estimate of the average change over time irrespective of the programme Programme indicates whether the school was in the intervention group

Time*Programme is an interaction term that indicates the effect of the programme over time, after accounting for intervention/control school differences and changes over time. The coefficient on this term gives the “difference in differences” estimate: the difference between intervention and control schools in the change in outcome.

Urban indicates whether the school is in an urban area Urban*Programme is an interaction term indicating whether the effect of the

intervention is different in urban from rural areas Government indicates a government school Cash stipend indicates a cash stipend is given at the school Girls’ toilet indicate the school has separate toilets for girls Crowding measures number of children per classroom

Impact Indicators: Enrollment (Pooled Analysis) Table 23 shows the results of the estimation for enrollment. The results are consistent with the fixed effects models presented above. The SFP shows a positive effect on enrollment of 62 children, though the effect is not significant. The largest effects are being in an urban area (where schools are much larger) and being a government school. Neither cash stipends nor girls toilets appear to affect enrollment, and the minimal effect of crowding is significant most likely as a statistical artifact, as discussed above. Once again, no gender differences were observed. Table 23: Impact of SFP Programme on Enrollment: Pooled Model Dependent Variable=enrollment

All Schools P(t)

Time (follow-up=1) -72.92 *Programme area=1 -5.67 Time*Programme 62.12 Urban area=1 262.26 ***Urban*Programme 100.08 Government School=1 105.80 ***Cash stipend -25.25 Girls toilets on-site 25.19 Crowding 4.35 ***Intercept 42.30 Model Adj. R2 0.54F 48.39 p (F) <. 0001p <.05 ** p<.01 *** p<.001, + p <.10

35

Impact Indicators: Attendance (Pooled Analysis) The pooled estimates for attendance suggest similar conclusions to those of the fixed effects estimates. Results show a small increase in attendance over time and a small effect of the SFP on attendance after controlling for change over time and for initial intervention/control differences. The result here is not statistically significant, but of comparable magnitude to the estimate based on the paired analysis. The one notable difference in results is that in this model, the availability of a separate toilet for girls appears to have a very significant effect, of almost four percentage points, on attendance. This may have to do with the introduction into the sample of schools serving a relatively wealthier population (Motijheel) Table 24: Impact of SFP Programme on Attendance: Pooled Model Dependent Variable= Attendance

All Schools (1)

P(t) All schools (2)

P(t)

Time (follow-up=1) 3.92 4.55 *Programme area=1 0.67 2.36 Time*Programme 3.96 3.48 Urban area=1 -- -5.51 *Urban*Programme -- -6.81 ***Government School=1

-- -2.47 *

Cash stipend -- -1.65 Girls toilets on-site -- 3.86 ***Crowding -- -0.02 ***Intercept 69.16 *** 72.78 ***Model Adj. R2 0.09 0.25F 13.05 14.56 p (F) <.0001 <.0001p <.05 ** p<.01 *** p<.001, + p <.10

Impact Indicators: Achievement Test Score Achievement tests were administered to the children in Grade V at follow up. No such test was given at baseline. Therefore, we estimate the intervention’s effect simply by comparing intervention and control schools at the time of the follow up survey. The model used to estimate the effect of the SFP on achievement test score was run on the paired sample, as follows.

Score = f ( programme, urban, government, crowding) Table 25 shows the results of the estimation of this model. The largest effect is that of being in an urban area. The programme does not show a significant effect in this model. Since this sample contains no urban control schools, we also ran the model on rural schools only. Here again, the impact of the SFP is not significant, and the model overall is also not significant.

36

Table 25 Impact of SFP Programme on Standardized Test Scores Variable All

Schools (1)

P(t) Rural schools

(2)

P(t)

Programme=1

4.48 4.41

Urban =1

15.22 *** __

Government school=1

2.89 3.24

Students per classroom by shift

0.00 -0.02

Intercept

49.60 *** 50.10 ***

Model Adj. R2

0.15 0.00

F 8.29 1.02

p (F)

0.00 0.39

p <.05 ** p<.01 *** p<.001, + p <.10 We performed a similar analysis on the pooled sample of schools. We have already seen that the test scores are consistently much higher in urban areas, and hoped to use the pooled sample to see if the program had a significant impact in urban areas. We estimated the simplest possible model, as follows:

Score = f (programme, urban, urban*programme) To this model, we added three measures of schools quality: teacher absence (average % of days of teacher absence in the past academic year); crowding (students per classroom per shift), and number of days school was closed due to any cause. Finally, since we have seen that the SFP has a positive effect on attendance, we added average attendance (all grades, all months) to the model to see if schools with higher attendance rates showed higher test score performance. The results of these three model estimations are shown in Table 26. Consistent with the earlier analysis, urban schools show higher scores, and the programme shows a consistent, small, positive, non-significant effect on test score. Most notable is the significant negative coefficient on the interaction term, urban*program. This seems to suggest that in urban areas,

37

Table 26 Impact of SFP Programme on Standardized Test Scores Dependent Variable=Test score

All Schools

(1)

P(t) All Schools

(2)

P(t) All Schools

(3)

P(t)

Programme=1 4.16 3.44 2.24

Urban =1 27.55 *** 26.46 *** 28.69 ***

Urban*Programme -12.21 * -11.50 * -9.23

Crowding (Students per classroom by shift)

-- -0.02 0.00

Teacher Absenteeism (Days missed past year)

-- -0.03 -0.03

Days closed (past year) -- -0.27 --

Attendance Rate -- -- 0.29

Intercept 51.56 *** 74.43 ** 30.57 *

Model Adj. R2.24 .23 .23

F 20.48 10.34 10.39

p (F) .000 .000 .000

p <.05 ** p<.01 *** p<.001, + p <.10 the SFP actually has a negative effect on test scores. But this result is entirely an artifact of the non-comparability of the urban control schools in Motijheel from the rest of the sample. As mentioned earlier, the Motijheel schools serve a wealthier population and are better off than the schools in the rest of the sample, which were chosen specifically to represent those serving low-income populations. The better performance of these urban control schools compared to intervention schools is no doubt due to the fact that these control schools are more privileged. Indicators of schools quality --- crowding, teacher absence, and days closed --- showed no effect on test score performance but,(as shown in the last column of Table 26), absence rate did show a very small but still statistically significant effect on test score. Based on previous studies that have shown school breakfast or other school feeding to affect mathematics test scores in particular (Ahmed 2000, Meyers et al., 1989), we tested these same models using math score as the dependent variable, but the results were essentially identical to

38

those for the overall score. There was no indication of any difference in programme impact by gender.

VI. Discussion and Conclusions

The data show that we can be confident the programme is actually delivering the services it promised, since just about every student interviewed reported receiving biscuits on the previous day. And despite concerns about the texture and palatability of the biscuits, they do appear to have a reasonable level of acceptability, since almost all the students reported consuming at least some of them while at school. Most students reported positive opinions of the biscuits, with well over half saying there was nothing they disliked about it, and 98% saying they found it tasty. And the quantity appears to be appropriate, since close to 90% of students who ate the biscuits reported that they felt full after eating them. There should be no concern for gender discrimination: there were no apparent differences in receipt or consumption of the biscuits by gender. We did find that among the students (about 30%) who do not consume all of the biscuits at school, most share them with younger siblings, and the rest with other family members. There appear to be few problems with delivery of the biscuits. Very few programs (17%) reported having a shortfall in supply, and that occurred rarely. There were no reports of biscuits being diverted or sold, and almost no report of spoilage. We saw no evidence that the availability of the biscuit resulted in students reducing the amount of food consumed at home. In fact, students in the intervention schools increased their likelihood of consuming a substantial meal before coming to school, while there was no change in the control schools. Some parents of children in SFP schools reported that they no longer needed to give money for snacks in school. There is no evidence that the SFP has different effects on girls and boys. All the multivariate analyses run separately for boys and girls showed very similar effects, no different from those on the total sample. In fact, it is encouraging to note that enrollment, attendance, repetition and drop-out, and test scores did not show any notable gender differences in this study. Implementation of the SFP showed a positive effect on the availability of girls’ toilets compared to control schools, but also appears to have had the effect of increasing classroom crowding. We did not find, in most analyses, a positive effect of girls’ toilets on the outcomes of interest, nor was there a negative effect of crowding. The availability of water from an arsenic-free tube well increased significantly in both the treatment and the control schools---by about 30 percentage points in both groups. Forty percent of treatment schools have arsenic free tube wells, and another 18% have piped water, while 50% of control schools have arsenic free tube wells, and none has piped water (at the time of the follow up survey). The availability of clean water is quite important for the SFP, since almost all children report drinking water along with the biscuit. Encouraging the availability of clean water would be useful both for current participating schools and for identifying schools to receive the SFP in future.

39

There appears to have been a positive effect of the SFP on the morale and functioning of the SMCs. Members reported a greater sense of purpose when given responsibility for the SFP, and they reported better attendance and more participation at meetings. They said they found running the SFP to be rewarding, and that they did not mind the extra work involved. Similarly, teachers reported that the SFP increased their work burden, but the large majority (93%) said that they did not object to the extra work and class disruption because of programme benefits.

Descriptive results showed a positive effect of the SFP on two key outcomes: enrollment and attendance. Enrollment did not increase between baseline and follow up in the schools offering the SFP biscuits. But enrollment fell significantly in the non-programme schools, suggesting that some factors that were causing a decline in enrollment were counteracted by the availability of the SFP. Attendance rates showed a significant increase between baseline and follow up in the schools offering biscuits, while increases in attendance were smaller and mostly non-significant in the non-programme schools. In multivariate analyses controlling for other factors likely to affect these outcomes, the SFP did appear to have a significant impact on enrollment. Even though much of this effect was due to the location of intervention schools in urban areas (since urban schools are much larger), the SFP’s effect remained even after urban location was controlled, and was significant for rural schools. The SFP also showed a positive effect on attendance rates, with attendance increasing more in intervention than control schools over the period of the study---a net improvement of intervention over control schools of between five and six percentage points. This effect, measured two years after programme implementation, is somewhat larger than the attendance improvement of 1.1 percentage points reported in an earlier study of the same SFP (Ahmed 2004), possibly because the programme had had a longer time to work at the time of our study. Descriptive results showed a significant decrease in drop-out rate in the intervention schools, but this decrease was not greater than that observed in control schools. A small decrease in repetition rate was observed, that was significantly greater than the change in repetition in the control schools, but both these effects lost their significance when other factors were controlled in multivariate analysis. This contrasts with the Ahmed (2004) study, in which significant decreases in the probability of children repeating a grade or dropping out were reported. The SFP showed a consistent, small, non-significant positive effect on achievement test scores. Test scores were significantly higher in urban than rural schools, and there was a small but significant impact of attendance on test scores. Other possible influences, including crowding, teacher absence, and days of school closing, were not significant. The lack of significant programme impact on achievement test scores contrasts with the positive results reported in the Ahmed study, in which test scores were dichotomized, and the SFP was associated with a significantly higher probability of scoring above 50%.. Our analysis treated test score as a continuous variable. Given the high level of participation among children in the intervention schools, and the reported positive effects of the programme on children’s motivation, energy, and attentiveness, and on their general level of health and nutrition, it is perhaps surprising that we did not observe a greater impact on academic achievement, particularly when controlling for several measures of schooling quality. It may be that with a larger sample size, the consistent observed effects might have reached statistical significance, but we must

40

acknowledge that there may simply be too many other factors affecting test score, factors that may swamp any effect of the SFP. Given the limitations of the study, we were not able to measure programme effects on children’s growth or micronutrient status. Teachers, parents, and SMC members reported that children’s weight improved, and that they appeared to be less subject to a variety of illnesses. In future studies, it would be useful to include measures of health and nutrition outcomes, as these can be very important benefits of school feeding programmes. This information could be used to inform decisions about whether the quantity and nutrient content of the biscuit are appropriate. It might also be useful to explore the usefulness of complementary interventions (deworming, for example) that might be promoted in the same way that girls’ toilets were promoted. Parents, teachers, and SMC members expressed a variety of suggestions regarding the SFP. They expressed concern about the quality of the biscuits provided, suggested provision of milk along with the biscuits, and thought that a greater variety of food should be provided by the programme. Certainly, quality control is important, even though reports of spoilage were rare. No doubt the provision of milk along with the biscuits would represent a valuable nutritional contribution, but in fact the children all drink water with the biscuits, so the hardness appears to be less of a problem than it seemed to be from our focus group discussions. The issue of feeding children in classes below grade I was raised by a number of respondents. If these younger children are attending pre-school in a school that offers the SFP, it is actually hard to justify not providing them with a nutritional supplement, especially given the recognized value of intervening as early as possible to improve nutritional status of at-risk children, both for health and growth and for cognitive benefit. The acceptability of the biscuits for younger children would of course need to be tested, but it seems to be a small step, once the administrative structure is in place, to extend the SFP to these younger classes. Some respondents suggested expanding the programme to more areas of the country. In light of the positive effects on enrollment and attendance, reported impact on attention and motivation, and likely benefits for nutritional status and health, providing the SFP to more low-income schools is probably justified.

41

References

Ahmed, Akhter U., Inpacts of Feeding Children in School: Evidence from Bangladesh. Washington, D.C.: International Food Policy Research Institute, mimeo. June 1, 2004 (draft for comments)

Coates, Jennifer and Zahidul Hassan, School Feeding in Vulnerable Rural Areas and Urban Slums in Bangladesh: A Baseline Report to the World Food Programme, Bangladesh. Medford, Mass.: Friedman School of Nutrition Science and Policy, Tufts University, and Dhaka, Bangladesh: DATA Analysis and Technical Assistance, Inc., mimeo. October 22, 2002 (for internal use only)

Dallal, Gerard. (1986) “PC-Size Consultant for Calculating Sample Size”, Downloaded from http://www.tufts.edu/~gdallal/STATPKG.HTM.

Meyers, AF; AE Sampson; M. Weitzman, BL Rogers; H.Kayne. “School Breakfast Program and School Performance,” American Journal of Diseases of Children 143:1234-1239, 1989.

wfp bangladesh school feeding programme mid term...

Documents