Social funds: evidence on targeting, impacts and sustainability

Download Social funds: evidence on targeting, impacts and sustainability

Post on 11-Jun-2016

212 views

Category:

Documents

0 download

TRANSCRIPT

  • Journal of International Development

    J. Int. Dev. 14, 627642 (2002)Published online in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/jid.900

    SOCIAL FUNDS: EVIDENCE ONTARGETING, IMPACTS AND

    SUSTAINABILITY

    JULIE VAN DOMELEN*

    Social Protection Unit, Human Development Network, World Bank, Wishington DC, USA

    Abstract: Impact evaluations show that social fund resources are pro-poor, and that targeting

    has improved over time. Despite the leakage which occurs to better-off areas and households,

    social fund performance compares favorably with other public programmes. Investments

    largely reflect community needs and priorities and have increased access to, quality and

    utilization of basic social infrastructure. These benefits have generally translated into

    improvements in the health and education status of households, though specific impacts

    vary by country, region, and sector. The vast majority of facilities are operating several years

    after completion, but long-term sustainability of water systems is particularly problematic

    given insufficient cost recovery. Copyright # 2002 John Wiley & Sons, Ltd.

    1 INTRODUCTION

    Social funds channel investment resources to local needs. Since their beginning in the late

    1980s, social funds have expanded rapidly to over 60 countries, and have gained wide

    support from donors and Governments. Social funds differ from many traditional

    development programmes. They fund proposals for small-scale investments generated

    from the local level rather than pre-identifying projects or locations to receive investments.

    They have therefore received the label demand-driven. An incentive structure is

    established that allows open access by communities to a list of eligible types of projects.

    A broad range of actors are typically involved in presenting proposals, including direct

    requests from community groups, which is often a novelty in these countries.

    After more than a decade of implementation experience in many countries, several key

    questions about social fund performance remain. Various myths have evolved, largely

    untested by empirical evidence. Social funds have been successful in delivering results

    on the ground, but good information on who is reached by these programmes, the

    Copyright# 2002 John Wiley & Sons, Ltd.

    *Correspondence to: Julie Van Domelen, Social Protection Unit, Human Development Network, World Bank,1818 H Street, NW, Washington DC 20433, USA. E-mail: jvandomelen@worldbank.org

  • sustainability of these local investments and the ultimate impact on improving household

    welfare has been scant.1

    In order to address these questions, a cross-country research effort was undertaken by

    the World Bank, other donors, Governments, and national and international researchers.

    Six case study countries were chosenArmenia, Bolivia, Honduras, Nicaragua, Peru and

    Zambiabased on having a sufficient number of completed community investments and

    an impact evaluation planned or on-going that allowed for representative sample sizes and

    robust evaluation methodologies.2 The research focused on the developmental impact of

    social funds, namely: (i) do social fund interventions reach the poor, both in terms of

    reaching poor districts, as well as reaching poor households?; (ii) do these investments

    reflect community needs and priorities?; (iii) have these investments increased access to,

    quality and utilization of basic services? ; (iv) what impact have these investments had on

    improving living standards?; and (v) how sustainable are these basic services?

    The research compared welfare outcomes for communities that undertook social fund

    investments with control or comparison groups constructed using propensity score

    matching and matched comparisons.3 Bolivia is the only case with both baseline and

    follow-up data available, as well as using an experimental design based on a randomized

    control group, which has rarely been applied in evaluating development programmes.4

    Over 21 000 household surveys were applied in communities benefiting from social

    fund investments. National household surveys, with a sample size of over 46 000 house-

    holds, were used to create control groups. In all cases, the household data are representa-

    tive of social fund beneficiaries in that country. Over 1200 facilities surveys collected

    information on the staffing and operation of schools, health centers, water and sanitation

    systems, both social fund as well as comparator matched facilities. Qualitative assess-

    ments using both household surveys and focus group interviews were implemented either

    in conjunction with the impact evaluation or as free-standing beneficiary assessments to

    provide further insights.

    The results from six social funds are not necessarily indicative of the universe of social

    funds, particularly social funds that do not focus on social infrastructure (e.g. programmes

    in Argentina & Chile) or have other goals. Because the main focus was on measuring

    impacts and sustainability, the sample frames assess projects that concluded between

    199499, and so may not represent the more recent efforts towards community capacity

    1One such myth concerns the institutional role of social funds. This is not discussed in this paper, but seede Haan, and Carvalho et al. in this issue.2This article draws on the main results of that research. The cross-country analysis is presented in a forthcomingWorld Bank publication, Communities Take the Lead: Evaluating Social Fund Performance Across Countries byRawlings, Sherburne-Benz and Van Domelen. The full study also includes analysis of the cost efciency ofsocial funds versus comparator mechanisms. Due to space constraints, this topic has not been includedhere. The following evaluation reports carried out in the case study countries served as the principalbackground studies used for the cross-country analysis: Armenia: Chase (2001); Bolivia: Newmanet al. (2000); Honduras: Walker et al. (1999); Nicaragua: Rawlings et al. (2000); Peru: Instituto Apoyo(2000b); Instituto Apoyo. Alcazar and Wachtenheim (2000a): Paxson and Schady (1999); Zambia: Chaseand Sherburne-Benz (2000). Unless otherwise noted, all statistical ndings cited on specic countries aredrawn from these reports.3The counterfactual applied is what would the state of households been in the absence of the social fundintervention. In most cases, control group households already had access to basic services like primary schools orhealth services that may or may not have benefited from recent investments by other agencies like line ministries,local government or NGOs. So, the impact evaluations measure the net social fund effect.4The Bolivia impact evaluation only evaluated results in rural areas.

    628 J. Van Domelen

    Copyright# 2002 John Wiley & Sons, Ltd. J. Int. Dev. 14, 627642 (2002)

  • building and training, linkages with local government, or some of the sustainability

    measures taken in recent years.

    2 DO SOCIAL FUNDS REACH POOR AREAS AND POOR HOUSEHOLDS?

    A myth has developed that demand-driven programs are incapable of reaching the poor

    because the poorest communities and households are not organized, motivated or capable

    of effectively expressing their demands (Stewart, 1995; Tendler, 2000). To assess targeting

    results, the research looked at two levels: the degree to which resources were focused on

    poor areas and the level of poverty of people that benefited.

    As shown in Table 1, the geographic distribution of social fund resources has been

    generally progressive in all cases. Poorer districts and municipalities have received more in

    per capita terms than better-off areas. The cumulative distribution of resources since

    inception, which includes the early years of emergency objectives often focused in urban

    areas, shows that the poorest 30 per cent of districts have received between 35 per cent

    (Zambia) and 66 per cent (Peru) of social fund resources.5 Looking at data only for the more

    recent years, targeting has improved over time, in some cases substantially. In Bolivia, the

    share of resources going to the poorest 30 per cent increased from 13 per cent in 1991 to 58

    per cent in 1998. In Nicaragua, the figures are 28 per cent to the lowest 30 per cent in 1991,

    increasing to 51 per cent in 1998. And in Peru, the share of total funding allocated to the

    poorest 30 per cent of districts increased from 43 per cent in 1992 to 79 per cent in 1998.

    Improvements in geographical poverty targeting over time can be explained by a

    number of factors. On the supply side, a more aggressive policy of targeting poor areas,

    proactive outreach, including information campaigns and technical assistance to poorer

    areas, and the establishment of regional offices has helped focus efforts on poor areas. On

    the demand side, studies have shown that communities often mobilize based on word of

    mouth and visits to other communities (Owen & Van Domelen, 1998). More risk averse

    and remote communities may begin to seek programme resources slightly later, after

    seeing the benefits in more dynamic communities. And in the case of Bolivia, significant

    fiscal decentralization in 199495 appears to have enabled poor municipalities to increase

    their demand for social fund financing.

    Data from household surveys shows that social fund beneficiaries are concentrated

    among the poor. The percentage of beneficiaries beneath the national poverty line ranged

    from 71 per cent in Zambia to 55 per cent in Nicaragua (Figure 1). Comparing these

    outcomes with the national incidence of poverty and extreme poverty, the share of social

    fund participants is equal to or surpasses the national population, depending on the

    country, both for the poverty line and extreme poverty line. In Peru, Nicaragua and

    Honduras, the poor and the extreme poor account for between 6 and 45 per cent more than

    their representation in the national population. In Zambia, the incidence of social fund

    beneficiaries mirrors the high level of poverty (over 70 per cent) at the national level.

    Despite this neutral distribution of beneficiaries compared with the national poverty line,

    5The most progressive overall geographical targeting appears in the Latin American examples. The Peru socialfund, FONCODES, has the most progressive distribution of resources geographically, largely due to thepredominantly rural focus of FONCODES. In the Zambia case, given the broad sweep of poverty in the country,the social fund sought a relatively equal distribution across districts. In Armenia, additional priorities, likefocusing investments in earthquake reconstruction zones, have tended to dilute the poverty criteria of investmentallocation.

    Targeting, Impacts and Sustainability 629

    Copyright# 2002 John Wiley & Sons, Ltd. J. Int. Dev. 14, 627642 (2002)

  • Tab

    le1

    .G

    eog

    rap

    hic

    ald

    istr

    ibu

    tio

    no

    fso

    cial

    fun

    dre

    sou

    rces

    sh

    are

    of

    exp

    end

    itu

    res

    by

    geo

    gra

    ph

    ical

    pover

    tyd

    ecil

    e1

    Arm

    enia

    Bo

    liv

    iaH

    on

    du

    ras

    Nic

    arag

    ua

    Per

    uZ

    amb

    ia

    Mu

    nic

    ipal

    19

    96

    9

    9A

    SIF

    19

    91

    9

    81

    99

    81

    99

    1

    19

    98

    19

    99

    19

    91

    9

    81

    99

    81

    99

    2

    98

    19

    98

    19

    91

    9

    8S

    RP

    or

    dis

    tric

    tII

    11

    dec

    ile

    1

    po

    ore

    st

    11

    12

    11

    91

    51

    51

    52

    02

    53

    21

    08

    21

    31

    21

    43

    21

    11

    31

    41

    62

    12

    51

    11

    3

    31

    21

    31

    51

    71

    11

    41

    31

    51

    92

    21

    51

    7

    41

    21

    21

    31

    39

    10

    11

    10

    12

    13

    98

    59

    91

    11

    11

    09

    11

    11

    85

    91

    0

    66

    61

    21

    01

    08

    11

    45

    21

    41

    2

    76

    68

    88

    98

    10

    31

    88

    87

    88

    05

    97

    53

    11

    11

    4

    91

    01

    16

    01

    57

    65

    20

    67

    10

    rich

    est

    14

    12

    31

    67

    54

    10

    93

    1In

    each

    cou

    ntr

    y,m

    un

    icip

    alit

    ies

    or

    dis

    tric

    tsw

    ere

    ran

    ked

    fro

    mp

    oo

    rest

    tori

    ches

    tb

    ased

    on

    the

    pover

    tyin

    dic

    ato

    rsu

    sed

    inn

    atio

    nal

    pover

    tym

    aps.

    Po

    pu

    lati

    on

    dec

    iles

    wer

    eth

    end

    raw

    n,

    wit

    hth

    ep

    oo

    rest

    dec

    ile

    rep

    rese

    nti

    ng

    10

    per

    cen

    to

    fth

    en

    atio

    nal

    po

    pula

    tio

    nre

    sid

    ing

    inth

    ep

    oo

    rest

    dis

    tric

    ts.

    Sources

    :A

    rmen

    ia

    Soci

    alIn

    ves

    tmen

    tF

    und

    dat

    abas

    e,B

    oli

    via

    S

    oci

    alIn

    ves

    tmen

    tF

    und

    dat

    abas

    e,H

    ondura

    sS

    oci

    alIn

    ves

    tmen

    tF

    und

    dat

    abas

    e,N

    icar

    agua

    E

    mer

    gen

    cyS

    oci

    alIn

    ves

    tmen

    tF

    un

    dd

    atab

    ase,

    Per

    u

    Pax

    son

    and

    Sch

    ady

    (19

    99),

    Zam

    bia

    :S

    oci

    alR

    ecover

    yP

    roje

    ctd

    atab

    ase.

    630 J. Van Domelen

    Copyright# 2002 John Wiley & Sons, Ltd. J. Int. Dev. 14, 627642 (2002)

  • the Zambia social fund was effective in targeting resources to the poorest of the poor, with

    15 per cent of beneficiaries drawn from the lowest ten per cent of the income distribution.

    How do these results compare with other programmes operating in the same countries? In

    terms of geographical distribution of resources, social funds appear more pro-poor than other

    government programmes, although data on other programmes is fairly limited. In Peru, the

    social fund has the most pro-poor geographical expenditure distribution of three targeted

    social programmesthe other two studied include a similar national school and health

    center construction programme and the national school feeding programme (Paxson &

    Schady, 1999) and compared favorably to the generally regressive per capita student

    expenditures by the Ministry of Education (Hentschel, 1998). In Bolivia, a social fund

    expenditure was three times more likely to reach a poor municipality than general fiscal

    transfers to local governments (World Bank, 2001). Results were similar at the household

    level. In Armenia, although the social fund household targeting was relatively neutral, the

    social fund performed in the mid-range of eight national social assistance programmes

    analysed (World Bank, 1999), including programmes that screen by household means

    testing. In Peru, the social fund had the most pro-poor distribution of beneficiaries of the three

    types of support generally available for local school infrastructure. In Nicaragua, social fund

    targeting is more likely to benefit the poor than expenditures made through the ministries of

    health and education. Therefore, despite their limitations, social fund targeting appears to

    have improved resource flows going to poor areas and poor households.

    Figure 1. Percentage of social fund beneficiaries below the national poverty lineNote: The household incidence of beneficiaries could not be measured in Bolivia for lack of anational household survey, and the national poverty line had not yet been established from the recenthousehold survey in Armenia.

    Targeting, Impacts and Sustainability 631

    Copyright# 2002 John Wiley & Sons, Ltd. J. Int. Dev. 14, 627642 (2002)

  • 3 DO THESE INVESTMENTS REFLECT COMMUNITY NEEDSAND PRIORITIES?

    Based on household surveys and focus group interviews, community members report that

    the projects selected correspond to important needs of the community.6

    * In Armenia, 79 per cent of community members interviewed reported that the

    implemented micro-project had solved the most important problem in the community,

    17 per cent thought that while the problem addressed was important more important

    ones remained, and only three per cent answered that the problem addressed was not

    important at all (Development Programs, 1999).

    * In Honduras, in communities that implemented water, education and health

    investments, these were the top-ranking investment priority of community members.

    However, latrine and sewerage investments did not reflect the felt needs of the

    communities where they were implemented (Table 2) (ESA Consultores, 1999).

    * In Nicaragua, 72 per cent of respondents answered that the FISE projects were those

    that most benefited the community out of all investments implemented recently

    (IDEAS, 1999).

    Table 2. Honduras: Community investment priorities

    Type of project actually chosen by the community

    Expressed CommunityPriorities

    Health Education Water Sewerage Latrines Average*

    Options on the FHIS

    Menu

    Health 35 11 3 18 6 15

    Education 1 47 1 6 5 12

    Water 4 7 64 10 23 22

    Sewerage 9 3 3 4 5 5

    Latrines 6 1 1 0 7 3

    Options not on the FHIS

    menu

    Electricity/Public 13 0 3 1 15 7

    lighting

    Roads and bridges 8 11 5 28 6 12

    Secondary education 8 0 0 0 0 2

    Pre-school 3 0 0 1 14 4

    Other 8 10 6 19 7 10

    Dont know/no opinion 5 9 15 13 13 11

    Total 100 100 100 100 100 100

    *Numbers may not add up due to rounding.Source: ESA Consultores, Ex-Post Evaluation of the Honduras Social Investment Fund (FHIS II), May 1999.

    6In the case of Armenia and Peru, data was obtained through beneficiary assessments, with sample sizesrepresentative of completed projects and respondents randomly sampled. In Honduras, the questions wereincluded in the impact evaluation survey, also randomly sampled and representative at the national level. InNicaragua and Zambia the findings should be treated as indicative due to small sample sizes. In Zambia, althoughonly 12 project communities were visited, an in-depth methodology was used that involved deploying teams tospend one week in each community, using rapid rural appraisal techniques, including community workshops,focus group discussions and individual interviews.

    632 J. Van Domelen

    Copyright# 2002 John Wiley & Sons, Ltd. J. Int. Dev. 14, 627642 (2002)

  • * In Peru, when asked if the project selected was the one that the community most

    needed, 90 per cent responded that it was the highest priority investment. 92 per cent of

    women queried responded similarly. Overall, 58 per cent of community members

    interviewed judged the project as very necessary, 40 per cent as necessary and one

    per cent as not necessary (Instituto APOYO, 1999).

    * In Zambia, based on an intensive ranking process, for new projectswhere the

    community had either only recently applied or begun activities83 per cent

    corresponded to the top community priority. For ongoing or completed projects, 67

    per cent ranked the investment as the top community priority (Milmo, 1994).

    The qualitative assessments point to several factors that affect whether a communitys

    top priority is met. For example, the Zambia research concluded that in the few cases

    where schools or health centers were constructed even though they were not the top

    priority of the community, this may be due to the fact that these facilities are run by

    people that know where to go when they want assistance (Milmo, 1994). In Honduras,

    focus group interviews in two communities where lower priority projects were selected

    revealed that, in the village of La Empalizada, men would have preferred a water project,

    but the initiative was blocked by the refusal of a neighboring village to share its water

    source, and women would have preferred a health center, but the Ministry of Health would

    have to be lobbied to provide personnel. The teacher of the deteriorated school was able to

    mobilize broad community support for a school rehabilitation project. In the community of

    Morales, a sewerage system was chosen largely in order to take advantage of additional

    resources from a foreign donor, which were limited to this type of project (ESA

    Consultores, 1994).

    4 HAVE SOCIAL FUNDS INCREASED ACCESS TO, QUALITYAND UTILIZATION OF BASIC SERVICES?

    Social funds have largely achieved their primary objective of increasing the availability

    and quality of basic social infrastructure, and that this has increased utilization of these

    services. The following section presents findings on schools, health centres, and water and

    sanitation projects.7

    4.1 Education

    Social fund investments improved the quality of school infrastructure. Social fund schools

    had more and better classrooms than comparators (Table 3). In all cases, students had

    better access to latrines and toilets. School fund schools had better access to safe water in

    all cases but Honduras. The availability of desks was typically better (in many cases the

    social fund investment included provision of student desks). In Armenia and Bolivia,

    social fund schools had a consistently better supply of learning materials (books, maps

    charts) even though these inputs were not financed by the social fund.

    7The impact evaluations focused on these sectors as they account for between 62 and 100 per cent of theinvestments made by the social funds in the six case study countries. However, not all sectors are covered in eachcountry.

    Targeting, Impacts and Sustainability 633

    Copyright# 2002 John Wiley & Sons, Ltd. J. Int. Dev. 14, 627642 (2002)

  • Social funds typically require that prior to investing in schools, the line ministries

    guarantee the provision of an adequate number of teachers. The outcomes of these

    agreement have never been systematically analysed and much speculation has arisen about

    the potential risk of empty classrooms. The impact evaluations found that, on the

    contrary, a rise in staffing accompanied the improvements in infrastructure in all cases. In

    Nicaragua the number of teachers in social fund schools rose by 20 per cent after the social

    fund intervention. In Zambia, Peru, and Armenia additions to teaching staff were observed

    as well. In Honduras, studentteacher ratios remained constant in spite an increase of 40

    per cent in the number of students enrolled, implying an increase in teachers.8

    This improved infrastructure and staffing led to an increase in the size of the student

    body in these schools. In Zambia, applications to social fund schools outnumbered those to

    control group schools (514 versus 259). In Honduras and Nicaragua, schools grew by 40

    and 20 per cent respectively; and rural schools in Peru saw an average increase of 34

    students. Increases observed in schools in Bolivia and Armenia were not significantly

    different from control group schools.

    4.2 Health Centres

    Overall, social fund-supported health centers had better physical conditions after the

    intervention than comparators. Table 4 shows that social fund facilities were more likely to

    be in average or better condition, not surprising since the investments were largely targeted

    at improving infrastructure. In Zambia and Bolivia, there was a significant expansion in

    the number of medical and delivery rooms available. In all country cases, sanitation

    Table 3. Improvements in physical aspects of primary schools following a social fund investment

    Social fund schools Non-social fund schools

    Percentage of classrooms in good physical condition1

    Bolivia 58 27

    Honduras 44 38

    Peru 68 44

    Number or net increase in Classrooms

    Honduras Net increase of 2.5

    Nicaragua 6 4.5

    Peru Net increase of2

    1Engineers assessments rated classrooms as good, normal or bad.2In Zambia, although physical condition was not measured, classes were more likely to be held outside innon-social fund schools (24 per cent) than in social fund schools (12 per cent).

    Table 4. Percentage of health facilities in average or better condition

    Social fund facilities Non-social fund facilities

    Honduras 97 62

    Nicaragua 70 25

    Zambia 94 71

    8Studies found no evidence of a decrease in the levels of staffing in comparator schools. Conclusions regardingthe impact of social fund investments on the overall distribution of teachers with the education system in casestudy countries cannot be drawn from the data available.

    634 J. Van Domelen

    Copyright# 2002 John Wiley & Sons, Ltd. J. Int. Dev. 14, 627642 (2002)

  • services were better in social fund-supported health centers than in comparator facilities,

    although improvements in water supply to health facilities varied by country.

    Project proposals are submitted to the line ministries for their approval, including an

    upfront agreement to provide for any recurrent cost obligations. Surveys of the availability

    of medicines and supplies in social fund-supported and comparator health centres found

    that in most cases the social fund centre had better availability. For example, in Bolivia,

    social fund health centres experienced an increase in the index of available medical

    supplies of 62 per cent between 199397, versus an increase of 23 per cent in control

    group facilities. However, both social fund and non-social fund facilities were far from

    having a full complement of needed medicines and supplies. For example, in Zambia and

    Honduras, about 20 and 60 per cent of all facilities lacked certain essential medicines,

    respectively. So while a social fund intervention may improve availability, there is

    evidence of continued weaknesses in the national distribution systems of essential health

    inputs to all heath centres.

    The story is similar for the provision of staff. Social fund supported health facilities

    were as well or better staffed than comparators, although for both there is evidence that

    staffing is below ministry norms in some countries. In the three countries with data

    Honduras, Nicaragua, and Zambiasocial fund facilities were more likely to have both

    professional staff (doctors, nurses and nurses aids) and well as non-professional staff

    including community volunteers.

    These improved facilities attracted significantly higher use by community members

    (Table 5). In Bolivia, although the likelihood of using a public health facility was the same

    between social fund and control group households, the percentage of women attending

    prenatal controls as well as the percentage of births attended by trained medical personnel

    increased significantly due to the social fund investment. In Honduras, households were

    more likely to seek attention for a medical problem than control groups. In Nicaragua,

    Table 5 Impact on household utilization of health facilities

    Social fund Propensity match P-valuemean control mean

    Bolivia (Post-intervention 1997/8)Use of Public Health Service (unconditional on illness) 0.067 0.061 0.56

    Use of Public Health Service (conditional on illness) 0.37 0.37 0.95

    Per cent of women with any prenatal control 61 38 0.001

    Per cent of births attended by trained personnel 18 14 0.04

    Treatment of diarrhoea 0.26 0.17 0.09

    Treatment of cough 0.25 0.31 0.41

    Honduras (per cent) (Post intervention 1997/8) naSeek medical attention at health post when 51 41 na

    have health problem

    Attended social fund health center 35 4 na

    Attended other ministry health center 6 29 na

    Total Attended ministry Primary Network 41 33 na

    Attended ministry of Health Hospital 2 3 na

    Attended Private Provider 6 4 na

    Nicaragua (Per cent) (Post-intervention 1998)Visitation Rate 10 6 0.000

    Visitation rate children

  • overall visitation rates as well as visitation rates of children under 6 increased substantially,

    although prenatal check-ups were less likely in the social fund health centres. In Zambia,

    the picture is less clear. In general, control groups were more likely to seek treatment if

    sick, but were more likely to go to hospitals rather than primary health care centres, whereas

    in social fund communities the sick were more likely to go to health centres.

    4.3 Water and Sanitation

    Social fund investments in water resulted in an increase in direct access to piped water in

    all countries with data.9 In Armenia, 93 per cent of households reported access to running

    water compared with 87 and 72 per cent in the various control groups constructed. In

    Honduras, access increased to 92 per cent compared with 87 per cent in control groups. In

    Nicaragua, the percentage of households in social fund communities that have access to

    piped water increased from 57 per cent in 1993 to 85 per cent in 1998, while control

    groups saw an increase from 52 to 57 per cent. In Bolivia, where the survey only included

    rural areas, access rose from between 4449 per cent to 5467 per cent depending on the

    region. This increased access was matched by more reliable supply than comparator

    systems, although in many cases supply remained seasonal or limited to specific hours

    during the day.

    The majority of systems appeared to be relatively well constructed and operating

    adequately, though selected quality problems were reported. In Peru, 86 per cent of

    community respondents felt the systems were functioning well or with minor problems,

    7 per cent cited major problems and 7 per cent said the system was not functioning

    (Instituto APOYO, 2000b). In Honduras, which concentrated on rehabilitating urban

    networks, comparator systems tended to be better built and 23 per cent of households

    queried declared the quality of works as poor in the social fund projects (but only 6 per

    cent were judged as poor in a review by engineers).

    Sanitation investments included both latrines and household sewerage systems.

    Sewerage systems showed relatively low connections rates, with overall rates reported

    at 44 per cent in Honduras and 61 per cent in Nicaragua. High costs of connecting to the

    system and of acquiring and installing a toilet were cited as the main reasons for low

    connection rates. Systems generally operated well and quality of construction was

    considered good where surveyed.

    Latrine projects were widely used by community members, with utilization rates ranging

    from 89 to 100 per cent in the countries with data. In Honduras, the only case that did a

    technical review of construction quality latrines, original construction quality was deemed

    good in 95 per cent of social fund latrines versus 45 per cent in the control group.

    5 WHAT IMPACT HAVE THESE INVESTMENTS HAD ON IMPROVINGHOUSEHOLD LIVING STANDARDS?

    Beyond the benefits of increased access and utilization, the impact evaluations measured

    changes in the likelihood of a household sending their child to primary school, infant

    9The water system investments made by social funds cover a wide range, including rehabilitation of exiting urbannetworks, as well as the new household connection systems and public standpipes in both rural and urban areas.

    636 J. Van Domelen

    Copyright# 2002 John Wiley & Sons, Ltd. J. Int. Dev. 14, 627642 (2002)

  • mortality rates and other indicators of improvements in the condition of household

    members.

    5.1 Education Investments

    Improving school infrastructure may increase the likelihood that parents decide to send

    their children to school, resulting in an increase in the primary education enrollment rate.

    In Armenia, Nicaragua and Zambia, significant increases, by between 5 and 8 percentage

    points, were observed over control groups. However, disagreggating between rural and

    urban areas showed significant improvements only in the urban areas. In Honduras and

    Bolivia, no significant changes in enrollment rates were observed.10 In Peru, two studies

    came to different conclusions. At the national level, increased primary school enrollment

    was positively correlated to the amount of social fund resources a district received. In the

    rural impact evaluation, social fund investments increased the probability of primary

    school enrollment, but coefficients were not statistically significant. These cross-country

    findings suggest a tendency for generating enrollment impacts mainly when investments in

    schools are urban based. This is consistent with capacity constraints in peri-urban

    neighborhoods; whereas demand side factors may present a greater constraint in rural

    areas.

    The impact evaluations included variables on education attainment, as measured by age

    for grade and accumulated years of education. As the more years of schooling a person has

    is directly correlated with future earnings, this indicator can point to future poverty

    reduction. In the countries with dataNicaragua, Peru, Honduras and Zambia

    educational attainment indicators generally showed improvements as a result of social

    fund school investments. In Zambia and Peru, these impacts were limited to rural areas.

    This finding may suggest that in rural areas the impact is more likely to be on having a

    child who is already enrolled remain in school, particularly since many of these

    investments allowed an additional grade to be added to the school. In Bolivia, the only

    case to administer academic achievement tests in the impact evaluation, no significant

    improvement was found in test scores.

    5.2 Health Projects

    Bolivia was the only country able to collect robust information on mortality rates

    stemming from investments in health infrastructure. Results show significant declines in

    infant and child mortality rates attributable to the social fund intervention (Figure 2).

    Infant and child mortality rates dropped by about 50 per cent in the period between the

    baseline and follow-up surveys in communities that had social fund health interventions,

    in comparison to increases of over ten per cent in the control group households.11

    Reductions in mortality rates were correlated with families that used the health facilities.

    10In Honduras, sample sizes may not have been large enough to detect statistical significances due to high overallenrollment rates. The household enrollment rates did increase and the size of the student body in social fundschools increased significantly.11The findings control for the age and education of the mother, family per capita consumption, availability ofpiped water, adequacy of water throughout the day and year, distance to water source, adequacy of sanitationfacilities and the presence of non-social fund water or sanitation improvements.

    Targeting, Impacts and Sustainability 637

    Copyright# 2002 John Wiley & Sons, Ltd. J. Int. Dev. 14, 627642 (2002)

  • A further indicator of future health outcomes is the vaccination rate. In Zambia,

    significant increases were found in rural BCG, DPT and polio vaccinations in social fund

    communities, though measles were somewhat lower than control groups. In Nicaragua, no

    significant differences were found between social fund and control groups, but overall

    vaccination rates are uniformly high in both groups.

    5.3 Water and Sanitation Projects

    Households benefited from water investments in two ways. First, in all country cases with

    data, there was a significant decrease in the time and/or distance to the water source. This

    time saving effects the household directly in terms of reducing the effective cost of water

    collection, and easing access and perhaps promoting greater consumption.

    Significant reductions in child mortality were observed in the two countries where this

    could be measuredBolivia and Peru. In the study of rural areas of Peru, deaths of

    children under five were 33 per 1000 for households benefiting from a social fund water

    investment versus 60 per 1000 in control group communities. Of note in explaining these

    results, community training included orientation in water use and hygiene, and significant

    reductions in the incidence of diarrhoea were observed. In Bolivia, deaths to children

    under five were reduced by 41 per cent as a result of improved water systems. No

    significant impact was found on the overall incidence of diarrhoea, but the data point to

    possible improvements in the duration of diarrhoea. Results in Nicaragua suggest an

    Figure 2. Impacts of social fund health investments on infant and child mortality rates in Bolivia

    638 J. Van Domelen

    Copyright# 2002 John Wiley & Sons, Ltd. J. Int. Dev. 14, 627642 (2002)

  • improvement in childrens health status, but results on the incidence of diarrhoea are not

    statistically significant. In Armenia, household surveys report that social fund water

    investments resulted in less frequent incidence of illness and less likelihood of reporting

    inactivity due to illness compared with control groups. In Honduras, no measurable

    impacts on incidence of diarrhoea were discerned, largely explained by the concentration

    on rehabilitation of existing urban systems.

    Household impacts of sanitation investments varied between sewerage and latrine

    projects. No health impacts were detected at the household level as a result of sewerage

    investments, perhaps reflecting the relatively incomplete connection rates.12 Latrines

    reduced the incidence of diarrhoea in Honduras and Peru.

    6 SUSTAINABILITY

    Sustainability of service delivery depends on a host of factors, including initial technical

    quality, community participation and commitment, on-going provision of key inputs and

    personnel, capacity of local organizations to run facilities in cases of community

    management, and so forth.13 The vast majority of social fund investments surveyed are

    functioning and delivering benefits several years after completion of the initial investment.

    This section reviews evidence from some of the factors affecting sustainability not

    discussed in the previous sections, namely community participation, maintenance and

    local financing.

    Evidence from household surveys suggests that there was significant community

    participation during the identification of investments. The form and level of community

    participation during execution varied by country, and in some cases between rural and

    urban areas. The proportion of household reporting that they had participated in execution

    ranged from about one-third in Armenia to two-thirds in Peru. This initial involvement

    appears to have carried through to some level of continued community support during

    operations, which manifests itself through several channels. In Honduras 92 per cent of

    schools had formed specific maintenance committees and in Zambia 50 per cent of

    communities had health centre maintenance committees. Local financial support was more

    likely in social fund-assisted facilities, including greater likelihood of providing financing

    to parentteacher associations in Zambia, more active fundraising efforts in Nicaragua,

    and additional financial support from 15 per cent of households in Armenia.

    However, one area that appears particularly problematic is the collection of fees to

    finance local water systems. In Honduras, even though revenues collected for social fund

    water systems were found to be almost twice those of control group facilities, these were

    typically insufficient to fully cover operational costs. In Nicaragua, half the sampled water

    systems (also urban based) reported that revenues were not adequate to meet costs. In rural

    areas of Peru, half of community members reported paying for water, though this figure is

    affected by the inclusion of public standpipe systems. The existence of a local water

    committee was critical for fee collection in these rural areas. In communities with

    established committees, two-thirds of household reported paying for services, versus

    only one-third in communities without committees (Instituto Apoyo, 2000b).

    12Studies have found that sanitation coverage of 75 per cent or more in urban communities is needed beforea health impact is apparent (Hogrewe et al., 1993).13See Carvalho et al. in this issue for an alternative approach to analyzing sustainability.

    Targeting, Impacts and Sustainability 639

    Copyright# 2002 John Wiley & Sons, Ltd. J. Int. Dev. 14, 627642 (2002)

  • 7 IMPLICATIONS OF THE FINDINGS

    The evidence on targeting, impacts and sustainability illuminate the debate about the

    development effectiveness of social funds. The failures of social funds expressed by critics

    do not seem to stand up to closer inspection. But nor do any claims of perfection. More

    important than coming up with a thumbs up or thumbs down judgment on social funds,

    the empirical findings highlight many of the challenges and tensions, as well as clear

    benefits, of channelling public development assistance to the local level.

    In terms of reaching the poor, there is no evidence of systematic bias against access by

    the poorest districts or households, contradicting the myth that the weakest areas and

    households are incapable of benefiting from demand-driven programs. And while

    leakage of benefits to better-off areas and households is significant, compared to other

    programmes with available evidence social funds typically present a more attractive

    alternative. The ability of social funds to positively discriminate towards the poor to the

    exclusion of access by the non-poor may be limited given the community-wide nature of

    the social infrastructure constructed. No screening is done at schools, health centres or

    water systems to ensure that only the poor access these improved services. Income

    heterogeneity within districts and communities will inevitably lead to leakage of benefits

    to the non-poor. The usefulness of geographical poverty maps is limited. Inclusion of

    urban areas automatically worsens targetingwhereas many programmes have goals of

    national coverageand most maps are not disaggregated enough to provide assurances

    that only poor communities will be reached. It may be possible to introduce better

    mechanisms for identifying the poorest communities within districts, but this relies on the

    development of better and more disaggregated poverty data.

    There may also be tensions between national objectives and community preferences.

    For example, a community may want to improve the physical space of their primary

    school, including better ventilation and lighting, improved sanitary facilities and so on,

    without necessarily expanding the primary enrollment ratewhich may be the over-

    arching national education priority. Similarly, a community may prioritize investments in

    a local water system in order to increase convenience and availability of water, without a

    specific goal of reducing infant mortality. Community-based programmes will need to

    balance how much weight should be given to communities stated priorities as measures

    for success versus external social planners indicators of success.

    The impact findings point to significant achievements in improving the access to,

    quality and utilization of basic social infrastructure. In addition to these improvements,

    households benefited from improvements in their health and education status. However,

    these household level impacts varied by sector and country and point to the need for a

    more comprehensive and circumspect understanding of the conditions under which public

    investments in different types of social infrastructure lead to poverty reduction.

    The sustainability of these services appears strongest for education and sanitation

    investments. Health investments delivered significant improvements in services and

    impacts, despite less than full provision of essential inputs. In the water sector, particular

    attention needs to be paid to improving the financial basis for sustained service delivery.

    Blanket approaches to sustainability will not workstrategies need to be designed that

    are sector and country specific.

    It is also clear that these community-level investments should be viewed as a

    complement to, rather than a substitute for, effective sector policies. Much of the impact

    and sustainability of these investments rely on the performance and policies of other

    640 J. Van Domelen

    Copyright# 2002 John Wiley & Sons, Ltd. J. Int. Dev. 14, 627642 (2002)

  • institutions, namely line ministries and local governments. Social fund cannot substitute

    for good policies for staffing and provisioning public primary schools and health centers.

    Instead, synergies should be sought that can help to solidify these gains achieved in

    improving the well-being of the poor.

    REFERENCES

    Barrientos JC, Jorgensen S. 1998. A Practical Approach for Designing Community Based

    OperationsWith Special Reference to Implementation Arrangement. HDNSP. World Bank:

    Washington, DC.

    Chase R. 2001. Social Funds 2000 Impact Evaluation: Armenia Social Investment Fund. Case Study.

    World Bank: Washington, DC.

    Chase R, Sherburne-Benz L. 2000. Impact Evaluation of the Zambia Social Fund. World Bank:

    Washington, DC.

    Development Programs Ltd. 1999. Sociological StudyArmenia Social Investment Fund. Armenia

    Social Investment Fund. Yerevan.

    Development Programs Ltd. 1997. Report of Armenian Social Investment Fund Project Impact

    Assessment Study. Armenia Social Investment Fund. Yerevan.

    ESA Consultores. 1999. Ex-Post Evaluation of the Honduras Social Investment Fund. Tegucigalpa.

    Hogrewe W, Joyce S, Perez E. 1993. The unique challenges of peri-urban sanitation. WASH

    Technical Report No. 86. United States Agency for International Development: Washington, DC.

    Instituto Apoyo. Alcazar L, Wachtenheim E. 2000a. Determinants of project success: case study of

    FONCODES. Monograph prepared for the World Bank: Washington, DC.

    Instituto Apoyo. 2000b. 6th Ex-Post Evaluation of FONCODES: Study of Water Supply Projects.

    Lima.

    Instituto Apoyo. 1999. Quinta Evaluacion Ex-Post de los Proyectos de FONCODES. Lima.

    Instituto de Desarrollo Empresarial Asociativo (IDEAS). 1998. Evaluacion cualitativa de

    beneficiarios del FISE 1993/96. Processed. Managua.

    Jorgensen S, Van Domelen J. 1999. Helping the poor manage risk better: the role of social funds. In

    Shielding the Poor: Social Protection in the Developing World, Lustig N (ed.). Brookings

    Institution and Inter-American Development Bank: Washington, DC.

    Newman J, Pradhan M, Rawlings L, Ridder G, Coa R, Evia JL. 2000. An impact evaluation of

    education, health and water supply investments of the Bolivian social investment fund. December.

    World Bank: Washington, DC.

    Newman J, Jorgensen S, Pradhan M. 1991. How did workers benefit from Bolivias emergency social

    fund? World Bank Economic Review 5(2).

    Owen D, Van Domelen J. 1998. Getting an earful: a review of beneficiary assessments of social

    funds. Social Protection Discussion Paper No. 9816. Social Protection Department. World Bank:

    Washington, DC.

    Paxson C, Schady N. 1999. Do school facilities matter?: the case of the Peruvian social fund

    (FOCNODES). World Bank Poverty Reduction and Economic Management Network, Policy

    Research Working Paper #2229. World Bank: Washington, DC.

    Rawlings L, Pradhan M, Ozler, B, et al. 2000. Nicaragua: ex-post impact evaluation of the

    emergency social investment fund (FISE). Report #20400-NI. World Bank: Washington, DC.

    Schady N. 1999. The political economy of expenditure by the Peruvian social fund (FONCODES),

    199195. Policy Research Working Paper No. 2166. World Bank: Washington, DC.

    Stewart F. 1995. Adjustment and Poverty. Routledge: London.

    Targeting, Impacts and Sustainability 641

    Copyright# 2002 John Wiley & Sons, Ltd. J. Int. Dev. 14, 627642 (2002)

  • Tendler J. 2000. Why are social funds so popular? In Local Dynamics in an Era of Globalization,

    Yusuf W, Evenett (eds). World Bank: Washington, DC.

    Walker I, del Cid R, Ordonez F, Rodrguez F. 1999. Ex-post evaluation of the Honduran social

    investment fund (FHIS 2). Produced by ESA Consultants. July.

    Honduras, for the World Bank, Latin American and Caribbean Region (LCSHD). World Bank:

    Washington, DC.

    World Bank. 2000. Nicaragua: ex-post impact evaluation of the emergency social investment fund.

    Report No. 20400-NI. World Bank: Washington, DC.

    World Bank. 2001. Implementation completion report: Bolivia second social investment fund

    project. World Bank: Washington, DC.

    642 J. Van Domelen

    Copyright# 2002 John Wiley & Sons, Ltd. J. Int. Dev. 14, 627642 (2002)

Recommended

View more >