world bank document€¦ · document of the world bank report no: icr2682 . implementation...

Document of The World Bank

Report No: ICR2682

IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF55745, TF55763, TF57678, TF90543, TF57778, TF56693)

ON A

GRANT FROM THE GLOBAL ENVIRONMENT FACILITY TRUST FUND IN THE AMOUNT OF

US$2.62 MILLION TO THE FEDERAL DEMOCRATIC REPUBLIC OF ETHIOPIA

US$2.55 MILLION TO THE REPUBLIC OF MALI

US$4.0 MILLION TO THE KINGDOM OF MOROCCO

US$1.7 MILLION TO THE REPUBLIC OF SOUTH AFRICA

US$6.87 MILLION TO THE UNITED REPUBLIC OF TANZANIA

US$4.0 MILLION TO THE REPUBLIC OF TUNISIA

FOR THE

AFRICA STOCKPILES PROGRAM – PROJECT 1

Environmental and Natural Resources Management Unit Sustainable Development Department Africa Regional Integration Africa Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective October 10, 2013)

Currency Unit = Ethiopian Birr (ETB)

ETB 18.94 = US$1

Currency Unit = West African CFA Franc (XOF)

XOF 484.91 = US$1

Currency Unit = Moroccan Dirham (MAD)

MAD 8.29 = US$1

Currency Unit = South African Rand (ZAR)

ZAR 9.96 = US$1

Currency Unit = Tanzanian Shilling (TZS)

TZS 1608.00 = US$1

Currency Unit = Tunisian Dinar (TND)

TND 0.61 = US$1

Vice President: Makhtar Diop

Regional Integration Director: Colin Bruce

Sector Manager: Magda Lovei

Project Team Leader: Dinesh Aryal

ICR Team Leader: Dinesh Aryal

EXECUTIVE SUMMARY

The Africa Stockpiles Program – Project 1 (ASP-1) was designed to address the issue of obsolete pesticides by developing and implementing disposal and prevention activities through the establishment of a new partnership between African countries and their collaborating partners, including the World Bank, the United Nations Food and Agriculture Organization (FAO), the pesticide industry represented by CropLife International (CLI), the Pesticides Action Network (PAN) in the UK and Africa (PAN-UK and PAN-Africa, respectively) and the World Wide Fund for Nature (WWF).

ASP-P1 was launched in September 2005 and was designed as an Adaptable Program Lending (APL) operation to be implemented over a 4-year period in seven countries: Ethiopia, Mali, Morocco, Nigeria, South Africa, Tanzania, and Tunisia. Program preparation took more than four years, which was mainly the result of a lengthy partnership-building process, caused by a complex envisaged implementation and partnership arrangement.

Despite the lengthy preparation, the program’s design suffered from several shortcomings. Implementation was subsequently characterized by a challenging collaboration among partners, substantial implementation delays and challenges, and varying performance across the participating countries. At mid-term review, it was considered difficult to achieve the PDO within the originally set timeframe and budget in all ASP-P1 participating countries, and it was agreed to restructure the program. A Level 1 restructuring conducted for four ASP-P1 countries in 2011 eventually positively affected implementation and facilitated PDO achievement. The overall outcome of ASP-P1 is rated moderately unsatisfactory. However, despite the program’s complexity, challenges, and poor initial performance, mitigation efforts eventually resulted in complete disposal of inventoried obsolete pesticides in ASP-P1 Ethiopia, ASP-P1 Tanzania, and ASP-P1 Tunisia, and partial disposal in ASP-P1 South Africa, having significantly reduced risks to the environment and public health. In addition, measures to reduce future re-accumulations by establishing sustainable systems of pest and pesticide management were substantially strengthened across ASP-P1 participating countries.

The elimination of obsolete pesticide stocks remains a development priority. Rural communities cannot hope to advance if the soil and water upon which their livelihoods and health depend are contaminated with pesticides, and urban populations cannot prosper if they are suffering from severe illnesses caused by pesticide poisoning. National development strategies of ASP-P1 participating countries confirm the importance of eliminating hazardous waste, as do World Bank priorities. The World Bank’s dual goals of ending extreme poverty and promoting shared prosperity are to be achieved in an environmentally sustainable manner by taking account of the impacts of climate change, environmental degradation, and ecosystem changes. New analytical work on hazardous wastes, POPs, and the clean agenda are expected in the future.

ABBREVIATIONS AND ACRONYMS

APL Adaptable Program Lending

ASP-P1 Africa Stockpiles Program – Project 1

CCAME Crosscutting Management Entity

CESA Country Environmental and Social Assessment

CIDA Canadian International Development Agency

CLI CropLife International

EC European Commission

FAO United Nations Food and Agriculture Organization

FFEM French Global Environment Facility

GEF Global Environment Facility

GEO Global Environment Objective

ICM Implementation Completion Memorandum

IPM Integrated Pest Management

IVM Integrated Vector Management

MDTF Multi-Donor Trust Fund

M&E Monitoring & Evaluation

NEAP National Environment Action Plan

NEPAD New Partnership for Africa’s Development

NGO Non-Governmental Organization

PAD Project Appraisal Document

PAN Pesticides Action Network

PCU Project Coordination Unit

PDO Project Development Objective

POP Persistent Organic Pollutants

TSU Technical Support Unit

UNEP United Nations Environment Program

WHO World Health Organization

WWF World Wildlife Fund

TABLE OF CONTENTS

1. PROGRAM CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES, AND DESIGN ....... 2 1.1 CONTEXT AT APPRAISAL ................................................................................................................................................ 2 1.2. ORIGINAL GEO, PDO, AND INDICATORS ..................................................................................................................... 4 1.3 REVISED GEO, PDO, AND INDICATORS ........................................................................................................................ 5 1.4 MAIN BENEFICIARIES....................................................................................................................................................... 5 1.5 ORIGINAL COMPONENTS ................................................................................................................................................ 6 1.6 REVISED COMPONENTS ................................................................................................................................................... 7

2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES ......................................................................... 7 2.1 PROJECT PREPARATION, DESIGN, AND QUALITY AT ENTRY ................................................................................ 7 2.2 IMPLEMENTATION .......................................................................................................................................................... 10 2.3 MONITORING AND EVALUATION (M&E) DESIGN, IMPLEMENTATION, AND UTILIZATION .......................... 12 2.4 SAFEGUARDS AND FIDUCIARY COMPLIANCE ......................................................................................................... 13 2.5 POST-COMPLETION OPERATION/NEXT PHASE ........................................................................................................ 14

3. ASSESSMENT OF OUTCOMES ........................................................................................................................................... 14 3.1 RELEVANCE OF OBJECTIVES, DESIGN, AND IMPLEMENTATION ........................................................................ 14 3.2 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVE ..................................................................................... 15 3.3 ACHIEVEMENT OF GLOBAL DEVELOPMENT OBJECTIVE ..................................................................................... 17 3.4 EFFICIENCY ...................................................................................................................................................................... 17 3.5 JUSTIFICATION OF OVERALL OUTCOME RATING ................................................................................................... 18 3.6 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS .............................................................................. 19 3.7 SUMMARY OF FINDINGS OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS ...................... 19

4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME ............................................................................................ 19 5. ASSESSMENT OF WORLD BANK AND BORROWER PERFORMANCE .................................................................... 20

5.1 WORLD BANK PERFORMANCE .................................................................................................................................... 20 5.2 BORROWER PERFORMANCE ......................................................................................................................................... 21

6. LESSONS LEARNED .............................................................................................................................................................. 22 7. COMMENTS ON ISSUES RAISED BY BORROWER, IMPLEMENTING AGENCY, AND PARTNERS .................. 23 8. ANNEXES ................................................................................................................................................................................. 24

8.1 PROJECT COSTS AND FINANCING ............................................................................................................................... 24 8.2 OUTPUTS BY COMPONENT ........................................................................................................................................... 25 8.3 ECONOMIC AND FINANCIAL ANALYSIS .................................................................................................................... 30 8.4 WORLD BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS ................................... 30 8.5 BENEFICIARY SURVEY RESULTS ................................................................................................................................ 30 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS .............................................................................................. 30 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR ................................................................... 30 8.8 COMMENTS OF CO-FINANCIERS AND OTHER PARTNERS/STAKEHOLDERS ..................................................... 30 8.9 VERBATIM COMMENTS ON DRAFT ICR ..................................................................................................................... 33

TABLE OF TECHNICAL ANNEXES

TECHNICAL ANNEX I: ASP-P1 ETHIOPIA .......................................................................................................................... 36 TECHNICAL ANNEX II: ASP-P1 MALI .................................................................................................................................. 77 TECHNICAL ANNEX III: ASP-P1 MOROCCO ................................................................................................................... 115 TECHNICAL ANNEX IV: ASP-P1 SOUTH AFRICA ........................................................................................................... 140 TECHNICAL ANNEX V: ASP-P1 TANZANIA ...................................................................................................................... 178 TECHNICAL ANNEX VI: ASP-P1 TUNISIA ......................................................................................................................... 220

iii

A. Basic Information1

Country:

Federal Democratic Republic of Ethiopia, Republic of Mali, Kingdom of Morocco, Republic of South Africa, United Republic of Tanzania, Republic of Tunisia

Project Name: Africa Stockpiles Program - Project 1

Project ID:

P075776 (South Africa/Tunisia), P103189 (Mali/Morocco/Tanzania), P105711 (Ethiopia)

L/C/TF Number(s):

TF55763/TF55745 (South Africa/Tunisia), TF57678/ TF57778/TF56693 (Mali/Morocco/Tanzania), TF90543 (Ethiopia)

ICR Date: 06/28/2013 ICR Type: Core ICR

Lending Instrument:

APL Borrower:

Federal Democratic Republic of Ethiopia, Republic of Mali, Kingdom of Morocco, Republic of South Africa, United Republic of Tanzania, Republic of Tunisia

Original Total Commitment:

US$21.7 million Disbursed Amount: US$16.32 million

Revised Amount: - Environmental Category: A Global Focal Area: CHEM Implementing Agencies: Ethiopia: Crop Protection Department; Mali: Direction Nationale de l’Assainissement et du Contrôle des Pollutions et des Nuisances (DNACPN); Morocco: Direction de la Protection des Végétaux des Contrôles Techniques et de la Répression des Fraudes (DPVCTRF); South Africa: Department of Environmental Affairs and Tourism (DEAT); Tanzania: National Environment Management Council (NEMC); and Tunisia: Agence Nationale de Gestion des Déchets (ANGed) Co-financiers and Other External Partners: National Governments of ASP-P1 participating countries, Multi-Donor Trust Fund (EU, Denmark, Sweden, Switzerland), Bilateral Financing (Belgium, Finland, Japan, Canada, and the Netherlands), French Global Environment Facility (FFEM), UN Food and Agriculture Organization (FAO), CropLife International (CLI), and Development Grant Facility (DGF).

1 The information presented in the data sheet was inserted manually. Information provided in the system was incomplete resulting from coding issues experienced during implementation; these were not resolved prior project closing.

iv

B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review:

07/17/2002 Effectiveness:

ASP: 09/30/2005 ASP: 11/21/2005 Ethiopia: 06/30/2007 Ethiopia: 07/13/2007 Mali: 02/12/2007 Mali: 12/26/2006 Morocco: 02/12/2007 Morocco: 02/05/2007 South Africa: 09/30/2005 South Africa: 04/04/2006 Tanzania: 02/12/2007 Tanzania: 12/19/2006 Tunisia: 09/30/2005 Tunisia: 11/21/2005

Appraisal:

ASP: 04/19/2004

Restructuring:

ASP: NA Ethiopia: 04/19/2005 Ethiopia: 06/30/2011 Mali: 04/19/2004 Mali: NA Morocco: 04/19/2004 Morocco: NA South Africa: 04/19/2004 South Africa: 06/30/2011

Tanzania: 04/19/2004 Tanzania: 06/30/2011 Tunisia: 04/19/2004 Tunisia: 06/30/2011

Approval:

ASP: 09/08/2005

Mid-term Review:

ASP: NA ASP: NA Ethiopia: 06/19/2007

Ethiopia: 06/30/2009

Ethiopia: 10/28/2009

Mali: 12/05/2006 Mali: 02/11/2009 Mali: 02/06/2009 Morocco: 12/05/2006 Morocco: 11/30/2008 Morocco: 12/11/2009 South Africa: 09/08/2005

South Africa: 03/2008 South Africa: 02/06/2009

Tanzania: 12/05/2006 Tanzania: 08/31/2008 Tanzania: 10/16/2009 Tunisia: 09/08/2005 Tunisia: 03/2008 Tunisia: 10/31/2008

Closing:

ASP: 06/30/2011 ASP: 05/31/2013 Ethiopia: 06/30/2011

Ethiopia: 12/31/2012; 05/31/2013.

Mali: 08/31/2010 Mali: 12/31/2011; 06/30/2012; 12/31/2012.

Morocco: 06/30/2010 Morocco: 06/30/2010 South Africa: 09/30/2009 South Africa: 04/30/2011;

06/30/2012. Tanzania: 06/30/2010 Tanzania: 12/31/2011;

12/31/2012; 05/31/2013. Tunisia: 09/30/2009 Tunisia: 07/31/2011;

06/30/2012.

v

C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Global Environment Outcome: Not Applicable2 World Bank Performance: Moderately Unsatisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of World Bank and Borrower Performance World Bank Ratings Borrower Ratings

Quality at Entry: Moderately Unsatisfactory Government: Satisfactory

Quality of Supervision: Moderately Unsatisfactory

Implementing Agency/Agencies: Moderately Satisfactory

Overall World Bank Performance:

Moderately Unsatisfactory

Overall Borrower Performance:

Moderately Satisfactory3

C.3 Quality at Entry and Implementation Performance Indicators Implementation Performance

Indicators QAG Assessments (if any)

Rating

Potential Problem Project at any time (Yes/No):

Yes Quality at Entry (QEA):

Moderately Satisfactory4

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

Moderately Unsatisfactory5

GEO rating before Closing/Inactive status:


D. Sector and Theme Codes Original Actual Sector Code (as % of total World Bank financing) Sanitation 33.3 33 Health 33.3 33 General Agriculture 33.3 34

2 The risk to development outcome varies across ASP-P1 participating countries and has been assessed at the country-level. Please see Section 4 of the Technical Annexes for details. 3 Note that this rating represents an average rating of the participating ASP-P1 countries’ performances. See Section 5.2 for details. 4 A Quality at Entry Assessment for FY06-07 was undertaken in for P075776. The assessment evaluated ASP-P1 at the program-level by taking ASP-P1 projects in South Africa and Tunisia into account. 5 A Learning Review of Regional Projects for FY07-08 was undertaken for P103189. The review evaluated ASP-P1 at the program-level by taking ASP-P1 projects in Mali, Morocco, and Tanzania into account.

vi

Theme Code (as % of total World Bank financing) Environmental policies and institutions 50 50 Pollution management and environmental health 50 50 E. World Bank Staff

Positions At ICR At Approval

Vice President: Makhtar Diop

South Africa /Tunisia Christiaan J. Poortman Gobind T. Nankani

Mali/Morocco/Tanzania Gobind T. Nankani Ethiopia Obiageli Katryn Ezekwesili

Director Regional Integration:

Colin Bruce

Country Director: - South Africa/Tunisia Theodore O. Ahlers Mali/Morocco/Tanzania Mark D. Tomlinson Ethiopia Mark D. Tomlinson

Sector Manager: Magda Lovei South Africa/Tunisia Vijay Jagannathan Mali/Morocco/Tanzania Mary Barton-Dock Ethiopia Mary Barton-Dock

Project Team Leader:

Dinesh Aryal South Africa/Tunisia Steven P. Maber Mali/Morocco/Tanzania Peter Kristensen/Denis Jordy Ethiopia Peter Kristensen/Denis Jordy

ICR Team Leader:

Dinesh Aryal

ICR Main Author: Veruschka Schmidt F. Results Framework Analysis6 Project Development Objective (PDO) - as presented in the PAD (Report No: 32232-MNA): Selected African countries eliminate inventoried publicly held obsolete pesticide stockpiles and associated waste, and implement measures to reduce and prevent future related risks.7 Revised PDO - The ASP-P1 PDO at the program-level was not revised. However, four ASP-P1 country projects underwent design adjustments under a combined Level I restructuring in June 2011 (see Technical Annexes on ASP-P1 Ethiopia, South Africa, Tanzania, and Tunisia). Global Environment Objective (GEO)- as presented in the PAD (Report No: 32232-MNA): Reduction of the effects of Persistent Organic Pollutants (POPs) on the global environment.

6 Annex 3 in the PAD outlined a results framework with slightly differing indicators in comparison to the arrangements for results monitoring. This ICR draws its evaluation on the results framework. 7 Corresponding Legal Agreements with ASP-P1 participating countries are discussed in each Technical Annex.

vii

Revised GEO - The ASP-P1 GEO was not revised. (a) PDO Outcome Indicator(s) - as presented in the PAD (Report No: 32232-MNA)

Indicator Baseline Value

Original Target Values

(from approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1: Inventoried publicly held obsolete pesticide stockpiles in 6 ASP-P1 countries eliminated

Value (quantitative or qualitative) 0% 100% removed - 35% Date Achieved 04/19/2004 06/30/2011 - 06/30/2013

Comments (incl. % achievement)

Target partly achieved (35%): Overall, 3,164 (35%) of the inventoried 8,949 tonnes of publicly held obsolete pesticides and associated waste were disposed of in four ASP-P1 countries (Ethiopia, South Africa, Tanzania, and Tunisia). ASP-P1 Mali and ASP-P1 Morocco closed without disposal activities having begun.

Indicator 2: Quantity of new publicly held obsolete pesticide stockpiles Value (quantitative or qualitative) 0 0 stockpiles - N/A Date Achieved 04/19/2004 06/30/2011 - 06/30/2013


Target not measurable: Inventories conducted during implementation revealed additional obsolete pesticides; however, it is difficult to assess whether additional stockpiles were accumulated during implementation or whether they resulted from inaccurate estimates used at appraisal.

Indicator 3: No further accumulation of publicly held obsolete pesticide stocks

Value (quantitative or qualitative) 0 0 additional stockpiles

- N/A

Date Achieved 04/19/2004 06/30/2011 - 06/30/2013

Comments (incl. % achievement) Indicator corresponds to PDO Outcome Indicator #2 and is therefore not taken into account.

(b) GEO Indicator(s) – The GEO was measured by the PDO and associated indicators. (c) Results Indicator(s) - as presented in the PAD (Report No: 32232-MNA)


Original Target Values (from

approval documents)

Formally Revised

Target Values



viii

Indicator 1: An inventory database in place in 78 ASP-P1 countries, accepted by the ASP partnership and being used and updated by government staff

Value (quantitative or qualitative) 0% 100% - 83% Date Achieved 04/19/2004 06/30/2011 - 06/30/2013


Target partly achieved (83%): An inventory database is in place, accepted by the partnership, and being used by all participating ASP-P1 countries, except for ASP-P1 South Africa where it was considered cost-ineffective by the government.

Indicator 2: In case of accident, emergency plans successfully implemented. Value (quantitative or qualitative) 0% 100% - N/A Date Achieved 04/19/2004 06/30/2011 - 06/30/2013

Comments (incl. % achievement) No accidents were reported; therefore, this indicator is not taken into account.

Indicator 3: Disposal contract completed in 6 ASP countries in line with national and international regulations

Value (quantitative or qualitative) 0 100 - 50% Date Achieved 04/19/2004 06/30/2011 - 06/30/2013


Target partly achieved (50%): Disposal contracts, in line with national and international regulations, were completed in four ASP-P1 countries (Ethiopia, Mali, Tanzania, and Tunisia); but only implemented in three (Ethiopia, Tanzania, and Tunisia).

Indicator 4: Quantity of accumulation of new obsolete pesticide stockpiles minimized

Value (quantitative or qualitative) 0 0 additional stockpiles - N/A

Date Achieved 04/19/2004 06/30/2011 - 06/30/2013

Comments (incl. % achievement) Indicator corresponds to PDO Outcome Indicator #2 and is therefore not taken into account.

Indicator 5: Government adopts of International Code of Conduct on pesticide distribution and use



Indicator was achieved prior to appraisal and is therefore not taken into account. The International Code of Conduct on pesticide distribution and use was adopted by all ASP-P1 participating countries during a FAO council session in 2002.

Indicator 6: Technical Support Unit (TSU) fully funded and staffed by end of year 1

8 Project activities for Nigeria, the seventh ASP-P1 country, were funded entirely from a Nigeria-specific, World Bank-managed Trust Fund provided by the Canadian International Development Agency. Project activities in Nigeria were therefore evaluated separately (see Implementation Completion Memorandum (ICM) – Nigeria, Africa Stockpiles Program, March 14, 2012).

ix

Value (quantitative or qualitative) 0% 100% - 100% Date Achieved 04/19/2004 End of year 1 - 04/01/2005

Comments (incl. % achievement) Target fully achieved (100%): The TSU was fully funded by two grants through FAO (Government of the Netherlands and GEF) and staffed by end of year 1.

Indicator 7: Client satisfaction as determined by survey Value (quantitative or qualitative) 0% 85% - 0% Date Achieved 04/19/2004 06/30/2011 - 06/30/2013

Comments (incl. % achievement) Target not achieved (0%): A survey to determine client satisfaction was not conducted; therefore client satisfaction was not determined.

Indicator 8: Preparation activities for ASP-P2 countries complete – national programs identified, selection conditions met, country counterparts in place and funding secured



Target partly achieved (50%): National programs were identified and selection conditions met. Country counterparts were not in place yet; and funding not secured as no support was obtained from the GEF Secretariat.

Indicator 9: Strategic studies delivered, e.g., alternative disposal technologies and locust control issues assessed and adopted if ASP standards met


Comments (incl. % achievement) Target fully achieved (100%): A study on disposal technology options was completed by WWF in June 2008; a study on locust control issues was carried out by FAO as part of the Red Locust Control Program.

Indicator 10: NGO participation in project activities in all ASP-P1 countries participating in at least two of the following: awareness raising, monitoring, capacity building and communication


Comments (incl. % achievement) Target fully achieved (100%): NGO participation in project activities was achieved in all participating ASP-P1 countries.

Indicator 11: Communication strategy implemented Value (quantitative or qualitative) 0% 100% - 67% Date Achieved 04/19/2004 06/30/2011 - 06/30/2013

Comments (incl. % achievement) Target partly achieved (67%): Communication strategies were developed in collaboration with WWF, and implemented in all ASP-P1 participating countries, except for ASP-P1 South Africa and Morocco.

Indicator 12: Monitoring reports delivered with sufficient data on time to GEF, donors and the World Bank

x


Comments (incl. % achievement) Target fully achieved (100%): Technical and financial reports were annually produced by the World Bank and shared with GEF and donors.

Indicator 13: ASP- Project Coordination Unit (PCU) moving to African organization within ASP-P1


Comments (incl. % achievement) Target not achieved (0%): An ASP- PCU was not moved to an African organization within ASP-P1.

G. Ratings of Project Performance in ISRs

No. Date ISR Archived

DO IP Actual Disbursements (USD millions)

1 10/31/2005 Satisfactory Satisfactory 0.00 2 06/30/2006 Satisfactory Satisfactory 0.50 3 12/27/2006 Satisfactory Satisfactory 0.57 4 06/28/2007 Moderately Satisfactory Moderately Satisfactory 0.58 5 12/18/2007 Moderately Satisfactory Moderately Satisfactory 0.59 6 05/30/2008 Moderately Satisfactory Moderately Satisfactory 1.38 7 12/18/2008 Moderately Satisfactory Moderately Satisfactory 1.65 8 03/25/2009 Moderately Satisfactory Moderately Satisfactory 2.03 9 12/29/2009 Moderately Unsatisfactory Moderately Unsatisfactory 2.56 10 06/29/2010 Moderately Unsatisfactory Moderately Unsatisfactory 2.66 11 03/27/2011 Moderately Satisfactory Moderately Satisfactory 3.66 12 12/25/2011 Satisfactory Satisfactory 3.66 13 06/29/2012 Moderately Unsatisfactory Moderately Unsatisfactory 5.37 06/30/2013 Final disbursement 16.32 H. Restructuring – Following World Bank Management’s decision, ASP-P1 at the program-level was not restructured. However, four ASP-P1 country projects underwent design adjustments under a combined Level I Board restructuring in June 2011 (see Technical Annexes on Ethiopia, South Africa, Tanzania, and Tunisia). I. Disbursement Profile – A disbursement profile is not available.9

9 The World Bank’s budgeting system did not allow to capturing all ASP-P1 country-projects under one project number; therefore an aggregate disbursement profile is not available. Disbursement data presented in Section G was retrieved from the Implementation Status and Results Reports (ISRs).

2

1. PROGRAM CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES, AND DESIGN

1.1 CONTEXT AT APPRAISAL

1. Country and sector background: During the second half of the 20th century, the African continent accumulated large quantities of obsolete pesticides.10 The accumulation of such obsolete products occurred as a result of inadequate stock management, non-distribution to farmers, bans on several pesticides, and/or un-coordinated or inappropriate supply from donor agencies. Often in severely deteriorated condition, poorly stored and located close to urban and rural settlements and public infrastructure, including water supplies, these obsolete pesticide stockpiles represented a serious risk to human and livestock health, ground and surface water, land use, and the broader environment. In addition, some of these obsolete pesticide stocks included Persistent Organic Pollutants (POPs)11, posing serious long-term danger to human health and the environment because of their mobility, toxicity, bioaccumulation potential, and persistence.

2. At appraisal of the Africa Stockpiles Program – Project 1 (ASP-P1), the amount of publicly held obsolete pesticides stockpiled across Africa was estimated at 50,000 tonnes.12 Most African countries were lacking adequate technical, institutional, and financial capacity to properly manage the destruction and safeguarding of obsolete pesticide stocks, and clean contaminated sites. They also had limited capacity and means to implement sound prevention practices and develop the necessary policy and regulatory framework to avoid continuous accumulation. Although several development agencies and donors13 had supported initial activities targeting disposal and prevention operations, resources remained limited because of countries’ pressing development needs in areas such as poverty alleviation, food security or other elements of sustainable development. In 2000, countries and the donor community began to realize that important efficiencies could be gained by promoting a strategic programmatic approach to obsolete pesticide elimination for Africa. Such an approach, together with the provision of funds specifically dedicated to pesticide/hazardous waste management and safe disposal of obsolete pesticides, was therefore sought.

3. ASP was designed to address the issue of obsolete pesticides by developing and implementing cost-efficient disposal and prevention activities through the establishment of a new partnership between African countries and their collaborating partners, including the World Bank, the United Nations Food and Agriculture Organization (FAO), the pesticide industry represented by CropLife International (CLI), the Pesticides Action Network (PAN) in the UK and Africa (PAN-UK and PAN-Africa, respectively) and the World Wide Fund for Nature (WWF). In order to remove all estimated

10 The ASP classified pesticides as obsolete when they were banned, had deteriorated, or had been damaged, had passed their expiration date, could not be used for another reason, or were no longer wanted by their owner. 11 POPs as addressed under the Stockholm Convention, and in this program are: Aldrin, Dieldrin, Endrin, Chlordane, Heptachlor, DDT (dichlorodiphenyltrichloroethane), Mirex, Hexachlorobenzene and Toxaphene. 12 In Africa, a total of 51,794 tonnes of obsolete pesticides had been identified by the Food and Agriculture Organization (FAO). This rough estimate did not include contaminated soils (except for contaminated soils in Mali and Botswana), and was intended to be refined through the country-specific detailed inventories planned under ASP. 13 For example FAO, Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ; previously called Gesellschaft für Technische Zusammenarbeit (GTZ)), Danish International Development Agency (DANIDA), United States Agency for International Development (USAID), Canadian International Development Agency (CIDA), Belgium, Finland, Japan, and the Netherlands.

3

obsolete pesticide stockpiles in Africa (50,000 tonnes), ASP was expected to require funding of US$200-250 million.14

4. The program was planned to be implemented through a series of projects within a 10-15 year timeframe, with ASP-P1 representing the first. ASP-P1 was launched in September 2005 and was designed as an Adaptable Program Lending (APL) operation to be implemented over a 4-year period in seven countries: Ethiopia, Mali, Morocco, Nigeria, South Africa, Tanzania, and Tunisia. South Africa and Tunisia represented the first APL phase, Mali, Morocco and Tanzania the second, and Ethiopia the third. ASP-P1 Nigeria, was not considered part of the APL as it was entirely funded by the Canadian International Development Agency (CIDA). Project activities in Nigeria were therefore not reviewed as part of this Implementation Completion Report (ICR); however, their outcomes are summarized in Box 1.115 All seven countries had ratified the Stockholm Convention on POPs and the Basel Convention on Trans-boundary Shipment of Hazardous Waste.16 During the first year of ASP-P1 operations, an additional eight countries were expected to be identified and the corresponding projects prepared for inclusion in a follow-on project called ASP-P2.17 This ICR pertains to the first group of projects – ASP-P1.

5. The primary financing instrument for ASP-P1 was a Global Environment Facility (GEF) grant of US$25 million, split between the two principal GEF Agencies; the World Bank (US$21.74 million) and FAO (US$3.26 million) towards a total program cost of US$60 million for the seven countries.

14 This figure was based on estimates of obsolete pesticides in Africa, taking into account that actual costs could rise significantly if additional stocks were discovered during country level inventories. Since no adequate hazardous waste destruction facilities existed in Africa, it was agreed that pesticide waste would be exported to approved facilities in developed countries at a cost of approximately US$3,500 per tonne, resulting in an estimated total of US$150-175 million. The total budget for prevention measures was estimated at US$50-75 million; thus the total ASP costs were expected to be in the range of US$200-250 million. 15 See Implementation Completion Memorandum (ICM) – Nigeria, Africa Stockpiles Program, March 14, 2012. 16 The Stockholm Convention on Persistent Organic Pollutants (POPs) is an international environmental treaty, signed in 2001 and effective since May 2004, aiming to eliminate or restrict the production and use of POPs. In implementing the Convention, governments take measures to eliminate or reduce the release of POPs into the environment. The Basel Convention on the Control of Trans-boundary Movements of Hazardous Wastes and Their Disposal is an international treaty that was designed to reduce the movements of hazardous waste between nations, and specifically to prevent transfer of hazardous waste from developed to less developed countries. 17 Pre-identified candidate countries for ASP-P2 were Benin, Botswana, Cameroon, Cote d’Ivoire, Ghana, Lesotho, Namibia, Niger, Rwanda, Senegal, Sudan, and Swaziland.

Trust Fund Amount: US$3.0 million provided by the Canadian International Development Agency (CIDA) PDO: To strengthen the recipient’s capacity for pesticide management and prevention of future accumulation of publicly held obsolete pesticides and associated waste. Overall outcome – Rating Moderately Unsatisfactory: ASP-P1 Nigeria was a pilot targeted at sensitizing stakeholders for and generating knowledge on obsolete pesticides and their impact on the environment and health. Despite operational difficulties encountered during implementation as a result of weak project management, the following outcomes were achieved: (i) a national inventory of obsolete pesticides was completed; (ii) a Country Environmental and Social Assessment (CESA) was prepared, found satisfactory by the World Bank, and publicly disclosed; (iii) improved draft legislation was endorsed by the Steering Committee; (iv) a prevention workshop was held; (v) ties with the private sector and NGOs were established; (vi) an emergency containment of two identified problem sites was carried out; and (vii) communication and awareness raising activities were carried out.

Box 1.1: ASP-P1 Nigeria – Project Outcome

4

Additional funds were mobilized through CIDA, a Multi-Donor Trust Fund (MDTF),18 and the French Global Environment Facility (FFEM). Fundraising was expected to be continued by the partnership throughout the life of the ASP, including from the partners themselves.

6. Rationale for World Bank assistance: The World Bank intended to support ASP-P1 based on its comparative advantage in having: (i) significant global experience in managing MDTFs and resource mobilization; and (ii) unique expertise in managing complex, and regional programs, establishing broad-based partnerships and leveraging additional financing. As a GEF Implementing Agency, the World Bank had a responsibility to implement activities toward achievement of GEF operational programs.19

7. Higher-level objectives: At the national level, ASP-P1 intended to contribute to the participating countries’ development strategies in the areas of public health, poverty alleviation, environmental protection and agriculture, and to help them implement their respective National Implementation Plans (NIPs) for elimination of POPs. At the global level, ASP-P1 intended to support international efforts to eliminate POPs by contributing to the objectives of international agreements such as the Stockholm Convention on POPs, and the Basel, Bamako, Rotterdam and Biological Diversity Conventions. The program further intended to improve management of toxic chemicals, and promote alternatives to pesticide usage, including Integrated Pest Management (IPM) and Integrated Vector Management (IVM). Capacity building for pesticide management and disposal aimed at enhancing the overall capability of partner countries to manage other toxic chemicals and hazardous waste.

1.2. ORIGINAL GEO, PDO, AND INDICATORS

8. Global Environment Objective (GEO): Reduction of the effects of POPs on the Global Environment. 9. Program Development Objective (PDO): Selected African countries to eliminate inventoried publicly

held obsolete pesticide stockpiles and associated waste; and implement measures to reduce and prevent future related risks.20

10. Indicators: The success of ASP-P1 in meeting its PDO was based on the following indicators:

18 The MDTF was funded by the European Commission (EC), the Swedish International Development Cooperation Agency (SIDA), the Swiss Agency for Development and Cooperation (SDC) and the Government of Denmark. 19 By implementing ASP-P1, together with FAO (which became a GEF Agency under the direct access to GEF financing in 2001), the World Bank intended to support the first large scale GEF-funded activity toward achieving objectives of the GEF Operational Program 14 (GEF OP14) for Reducing and Eliminating Releases of POPs, and support the role of GEF as a financing mechanism for the Stockholm Convention on POPs. 20 Corresponding Legal Agreements with ASP-P1 participating countries are discussed in each Technical Annex.

5

Table 1.1: ASP-P1 Original PDO Outcome and Results Indicators

Original PDO Outcome Indicators as presented in the PAD21 a. Inventoried publicly held obsolete pesticide stockpiles in 6 ASP-P1 countries eliminated.22 b. Quantity of new publicly held obsolete pesticide stockpiles. c. No further accumulation of publicly held obsolete pesticide stocks.

Original Results Indicators as presented in the PAD a. An inventory database in place in 7 ASP-P1 countries, accepted by the ASP partnership and being used and updated by

government staff. b. In case of accident, emergency plans successfully implemented. c. Disposal contract completed in 6 ASP countries in line with national and international regulations.23 d. Quantity of accumulation of new obsolete pesticide stockpiles minimized. e. Government adopts of International Code of Conduct on pesticide distribution and use. f. Technical Support Unit (TSU) fully funded and staffed by end of year 1. g. Client satisfaction as determined by survey. h. Preparation activities for ASP-P2 countries complete – national programs identified, selection conditions met, country

counterparts in place and funding secured. i. Strategic studies delivered, e.g., alternative disposal technologies and locust control issues assessed and adopted if ASP

standards met. j. NGO participation in project activities in all ASP-P1 countries participating in at least two of the following: awareness

raising, monitoring, capacity building and communication. k. Communication strategy implemented. l. Monitoring reports delivered with sufficient data on time to GEF, donors and the World Bank. m. ASP- Project Coordination Unit (PCU) moving to African organization within ASP-P1.

1.3 REVISED GEO, PDO, AND INDICATORS

11. The ASP-P1 GEO, PDO, and indicators were not revised.

1.4 MAIN BENEFICIARIES

12. Local beneficiaries: Primary beneficiaries were intended to be poor and farming communities. The project aimed to improve their quality of life by reducing environmental health risks. (i) Poor communities: Since obsolete pesticide stockpiles were often located near poorer communities where storage conditions were sub-standard and awareness to the proper use of pesticides was low, the intended primary beneficiaries were the urban and rural poor. (ii) Farming communities: A wide range of pesticides had been used for pest and vector management in agricultural areas, with pesticide distributors and small-scale farmers often having low levels of awareness for exposure risks, little knowledge of safe pesticide use, inadequate storage facilities, and limited alternative pest control methods.

13. National beneficiaries: The governments, in particular the ministries of health, environment, and agriculture in the participating countries were intended to benefit from the program through reduced loss in capital, and increased pesticide management capacity.

21 The indicators presented in the results framework differ from those presented in the arrangements for results monitoring. This ICR draws its evaluation on the indicators presented in the results framework. 22 Elimination of obsolete pesticides was not targeted in ASP-P1 Nigeria. 23 Elimination of obsolete pesticides was not targeted in ASP-P1 Nigeria.

6

14. Global beneficiaries: The global communities of people, species, and ecosystems were considered as global beneficiaries. Taking into account that obsolete pesticides are often characterized by high persistence in the environment,24 the negative environmental impact in the form of damaged ecosystems had been considered serious. Extensive cleanup and contamination-prevention activities were, therefore, expected to generate substantial global environmental health benefits.

1.5 ORIGINAL COMPONENTS

15. In support of its PDO/GEO, ASP-P1 had four components: (1) country operations; (2) technical support; (3) crosscutting activities; and (4) project coordination.

• Component 1 - Country Operations (GEF: US$21.74 million; Total: US$51.37 million): This was the program’s principal component, encompassing a full range of disposal, cleanup and prevention activities to achieve the PDOs in the participating countries. Sub-components were: (i) cleanup and disposal; (ii) prevention of pesticide accumulation; (iii) capacity building; and (iv) project management and monitoring. Country-specific details on these activities are outlined in the respective annexes.

• Component 2 - Technical Support (GEF: US$3.26 million; Total: US$4.31 million): This component was intended to provide highly technical and specialized expertise required for implementation, supervision and monitoring of country-level activities outlined under the first component, including: technical advice requested by the countries for the design and delivery of training; production of technical guidelines for cleanup and prevention operations; assistance in managing technical aspects of procurement and supervision of specialized contractors; advice on linking countries with other specialized agencies and organizations; enhancement of health and safety; assessment of laboratory capacities; and oversight of Monitoring and Evaluation (M&E). The technical support was also intended to include advice on disposal options, targeted to specific country and site conditions. FAO was expected to be responsible for hosting a Technical Support Unit (TSU) to coordinate delivery of technical services envisaged under this component and to tap into pertinent expertise of other ASP stakeholders.25

• Component 3 - Crosscutting Activities (Total: US$2.44 million): This component was expected to deliver knowledge management, awareness-raising, strategic studies and outreach services to ASP-P1 through a coordinated multi-country, multi-partner approach. The key sub-components included outreach to promote ratification of conventions relevant to the PDO, building Non-Governmental Organization (NGO) capacity, knowledge management, and communication. The strategic studies sub-component was intended to include an evaluation of alternative technologies for the disposal of obsolete pesticides. In addition, these studies were expected to support better understanding of environmental health risks associated with the specific pesticides, and to contribute to the prioritization and decision-making process within each country. WWF and PAN-UK were to host the Crosscutting Activities Management Entity (CCAME) responsible for delivery of support services under this component.

24 For example, DDT persistence in soil ranges from 22 to 30 years, Toxaphene up to 14 years, and Dieldrin up to 7 years. 25 E.g., United Nations Environment Program (UNEP), World Health Organization (WHO), Basel Convention Regional Centers, NGOs, and the private sector (represented by CLI).

7

• Component 4 – Project Coordination (Total US$2.44 million): This component aimed to coordinate the implementation and monitoring of ASP-P1 across the various partners. In addition to overall coordination, key activities included developing and ensuring implementation of a fund-raising strategy, acting as a Secretariat to the ASP Steering Committee (ASP-SC), maintaining awareness at the policy level about the project, reporting to donors, preparing work plans, progress reports and financial reports, coordinating independent project monitoring, and coordinating the design of follow-on projects. It was intended that initially, the World Bank would host a Project Coordination Unit (PCU), which during the implementation phase of ASP-P1 would be transferred to an appropriate African organization (e.g., the New Partnership for Africa’s Development (NEPAD)).26

1.6 REVISED COMPONENTS

16. The ASP-P1 components were not formally revised through a World Bank-approved restructuring. However, four ASP-P1 country projects (Ethiopia, South Africa, Tanzania and Tunisia) underwent design adjustments, including revision of components. The associated revisions made are outlined in the country-specific Technical Annexes.

2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES

2.1 PROJECT PREPARATION, DESIGN, AND QUALITY AT ENTRY

17. ASP-P1 was prepared between 2001 and 2005. It took more than four years from the time of GEF eligibility in July 2001, and over three years from concept note review in July 2002 to the program’s approval by the Board in September 2005. The long preparation time resulted from a lengthy partnership-building process at the program-level caused by a complex envisaged implementation and partnership arrangement. Associated discussions included defining each partner’s role, clarifying the World Bank’s legal, fiduciary and operational requirements for a high-risk regional program, and avoiding conflicts of interest among contracting partners. Particularly challenging was the collaboration between FAO and the World Bank as a result of competing interests in accessing GEF funding resources.27 While collaboration between CLI and the World Bank was adequate, it was also characterized by lengthy negotiations on finding modalities for channeling industry funds to the projects. A mutually endorsed funding agreement was not finalized prior to ASP-P1 Board approval in 2005; thus, a definition of CLI’s role and envisaged funding commitments was not included in the PAD or the Operational Manual.

18. Soundness of background analysis: ASP-P1 preparation was comprehensive and built on an extensive consultation process carried out during preparation comprising: (i) a body of analytical work on pesticide disposal and prevention, which informed project design; (ii) various workshops

26 NEPAD is an economic development program of the African Union, and aims to provide an overarching vision and policy framework for accelerating economic co-operation and integration among African countries. 27 Originally, FAO was envisaged to be a technical support agency to the program. However, FAO was approved as a GEF Agency in 2001, and therefore was able to access GEF funds directly without going through a GEF Implementing Agency as previously required. FAO’s role and responsibilities in ASP were therefore changed, and caused confusion among partners.

8

focusing on sharing experience and lessons learned,28 which highlighted several crucial areas that were incorporated into design; and (iii) a country workshop focusing on gathering feedback from ASP-P1 participating countries.29 ASP-P1 participating countries were selected, appropriately, on the basis of FAO guidelines such as country ownership and commitment, a well-identified lead agency, and stakeholder participation and national buy-in. Substantial background analysis was also undertaken at the country-level with close collaboration between governments and partners, as was evident through joint pre-appraisal missions to all participating ASP-P1 countries, and pledged financial contributions.

19. However, while key preparatory outputs such as the PAD and the ASP-P1 generic Operational Manual provided adequate and comprehensive information for implementing program-level activities, country-specific information was lacking (see Section 2.1 of the individual Technical Annexes). Despite extensive consultations and preliminary inventories conducted by FAO, the nature and size of the obsolete pesticide stocks scattered across ASP-P1 participating countries were only broadly estimated during preparation; the discovery of higher or lower amounts of stocks in participating countries during implementation was therefore a possibility, which the program’s plans did not reflect sufficiently. It was further assumed that no new stocks of obsolete pesticides would be accumulated during program implementation and thereafter, which was unrealistic in view of food security concerns and intensified agriculture. The STAP Roster Technical Review of ASP-P1 rightly pointed to these risks and suggested “to investigate whether the conditions that had led to the initial stockpiling are still applicable, and if changed, how this would affect preventative measures to be promoted by ASP-P1”.30

20. Assessment of project design: The program’s design suffered from several weaknesses:

• The program design gave insufficient attention to specifying and defining the partnership arrangements, and the roles and responsibilities of each partner involved. ASP-P1 was a complex multi-regional, multi-country program involving a variety of partners and heavily relying on efficient and effective collaboration. With the exception of a set of ‘Partnership Principles for Implementation of ASP’ articulated in Annex 1 of the ASP-P1 PAD, no formal documents defined the partners’ respective roles and their collaboration in detail. Further, NGO participation in ASP-P1 was configured and intended to be managed at the program-level, whereas the collaboration arrangements at the country-level were unclear. An independent evaluation of the program’s design and initial implementation, finalized in June 2009, suggested that unclear definition of the roles and working relationships among the institutional partners, particularly those of the World Bank and the TSU, negatively affected the program’s efficiency and effectiveness.31

• The uniform approach ASP-P1 used to the project implementation in participating countries did not take individual country contexts sufficiently into account. The PDO, associated indicators, and

28 In addition to a series of informal partnership meetings, a lessons learned exercise at the First Continental Conference for Africa on the management of hazardous waste (Rabat, January 2001), and a lessons learned workshop for disposal contractors (March 2002), were conducted. 29 ASP-P1 country workshop (April 2002) 30 See Annex 15 in South Africa and Tunisia: Africa Stockpile Program – Project 1: PAD (Report No: 32232-MNA), August 19, 2005 31 For more detail see Independent Evaluation of Design and Initial Implementation of Africa Stockpiles Program – Evaluation Report, May 2010, COWI – Document no. 69292-A-1

9

components were pre-configured at the program-level and, as such, applied to all ASP-P1 participating countries. Country-specific environments were not sufficiently taken into account, and during implementation, it became clear that the program approach was not suitable for all ASP-P1 participating countries. While the PAD noted a need for flexible implementation at the country-level, discussions between the World Bank and ASP-P1 participating countries on how to revise the design, together with internal World Bank discussions on how to process these restructurings, were lengthy.

• The design of the PDO, associated indicators, and components was overly ambitious. Given the lack of accurate data on obsolete pesticide stocks that was used during ASP-P1 preparation, the PDO, associated indicators, and components were unrealistically designed. A comprehensive inventories and Country Environmental and Social Assessments (CESAs) to determine the size and nature of stocks and associated disposal requirements were only envisaged to be conducted during implementation.

• The sequential design of Component 1 was not realistically reflected in the planned implementation and disbursement schedule. The completion of an inventory was necessary to prepare a CESA, which at the same time had to be completed before disposal activities could be initiated. Potential delays and their subsequent impact on overall project implementation were not adequately taken into account in the forecasted disbursement schedule and overall project timeframe.32

21. Government commitment: Adequate commitment at entry of the participating governments towards the issue of obsolete pesticides was an eligibility criterion for participation in ASP-P1. Governments had to ratify the relevant conventions and show a good level of readiness to implement planned activities. Please refer to each participating country’s Technical Annex for details.

22. Assessment of risks: Program-level risks were adequately identified and outlined in the PAD. They were presented in the form of a matrix and represented a summary of risk assessments undertaken during identification, pre-appraisal and appraisal missions to ASP-P1 participating countries. Even though the assessment of risks was comprehensive and included a broad range of mitigation measures in generic terms, an assessment of country-specific risks and mitigation measures was lacking. At the same time, it should be noted that it was difficult to assess country-specific risks in the absence of inventories of obsolete pesticides and CESAs, which were only planned to be undertaken during project implementation.

23. Quality at Entry Assessment (QEA): An assessment conducted by the Quality Assurance Group (QAG) on ASP-P1 in 2007, rated the QEA of the program moderately satisfactory.33 The panel considered that despite the lengthy preparation time the project was approved prematurely, but concurred with the World Bank team that delaying Board presentation could have halted the hard-negotiated momentum among clients and partners and led to the partnership’s collapse. The panel further stated that more thought should have been given to the policy environment, that much of the

32 For example, procuring a consulting firm to conduct the CESA and a service provider for the disposal of obsolete pesticides, and overall preparation of the CESA were lengthy procedures that should have been allocated sufficient time. 33 AFRICA: 3A-W Africa Stockpiles 1 GEF (FY06) (P075776): Eighth Quality at Entry Assessment (QEA8) Fiscal Year 06-07, June 27, 2007. The assessment reviewed ASP-P1 on the basis of two of the participating countries – ASP-P1 Tunisia and ASP-P1 South Africa.

10

first year’s program should have been carried out prior to approval, and that the prospect for completing the approved project within the prescribed four-year-time frame was poor.34

2.2 IMPLEMENTATION

24. ASP-P1 was implemented at the program-level, and in ASP-P1 participating countries. Implementation was characterized by a complex partnership, substantial delays, and varying performance across the participating countries. In 2010, activities at the program-level were terminated due to several partners having exhausted their resources, and ASP-P1 continued to be implemented solely at the country-level.

25. Key factors that affected implementation and outcomes at the program level, which simultaneously translated into implementation delays at the country-level included: • The envisaged governance structures as outlined in the PAD never became operational, and

affected program implementation. The PCU, intended to coordinate, monitor, and evaluate the implementation of ASP-P1 among the partners, and the ASP-Steering Committee (SC), intended to serve as advisory structure, were never established.35 Instead, an ASP Implementation Committee (ASPIC) was created as a forum for partners to oversee program implementation. The partners (i.e., FAO, CLI, PAN-UK, WWF, and the World Bank) met regularly through a combination of virtual and face-to-face meetings. However, ASPIC was never formalized, which resulted in an unclear overall governance structure, and unclear roles and responsibilities, cumbersome decision-making processes and coordination, and a lack of an independent M&E function. Eventually, following the GEF failure to approve ASP-P2 in 2009, future initiatives in the areas of pesticide management were prepared and managed individually by partners, outside the program.36

• Lack of clearly defined roles, responsibilities, and partnership principles significantly affected collaboration among partners, and translated into implementation delays at the country-level. o Collaboration between FAO and the World Bank was challenging throughout

implementation. The unclear roles of the two agencies, and their different organizational cultures, and internal procedures and requirements, translated into difficult discussions on standard document formats for project-related activities, delays and conflicting messages given to ASP-P1 participating countries,37 exacerbating the already weak operational and technical capacity of the ASP-P1 participating countries. In 2008, FAO and the World Bank prepared and agreed on a memo, which defined exact roles, responsibilities, and procedures. While this memo improved collaboration between the two entities, disagreements remained.38

34 Further ratings included: Environmental aspects satisfactory; strategic relevance and approach, technical, financial and economic aspects, poverty, gender and social development, fiduciary aspects, policy and institutional aspects, risk assessment moderately satisfactory; and World Bank inputs and processes moderately satisfactory. 35 NEPAD was intended to host the PCU, which did not materialize due to insufficient capacity. 36 ASP-P2 was not approved by GEF due to a lack of resources at the end of a GEF replenishment cycle at the time, and poor performance of ASP-P1. 37 Standard documents and generic Terms of Reference (TORs) that had been agreed on in the context of the ASP-P1 Operational Manual were reconsidered during implementation. 38 For example, FAO and the World Bank had different views on the roles, the national government staff could play in disposal operations: while FAO was in favor of including government staff to pursue cost-savings and increase national capacities, the World Bank opposed their participation as health and safety risks were high.

11

In 2010, difficulties subsided as FAO had exhausted its GEF funds and significantly minimized its support to the participating countries. To compensate for this, the World Bank increased its own support through technical and operational consultants funded by the MDTF.

o Delayed financial support obtained from CLI. In the beginning of implementation, it was planned that CLI funds would be channeled through the World Bank and/or FAO; however, shortly after, this was judged to be too complex and time-consuming. Instead, it was agreed that CLI would channel its contributions through direct financing agreements with recipient governments to support Technical Advisors for Disposal (TADs) as well as parallel contracts with hazardous waste management companies. This processes proved to be equally lengthy, ultimately further delaying the technical support and the financial contributions from CLI.

• World Bank institutional challenges contributed to program implementation delays. ASP-P1 was affected by a complex supervision arrangement within the World Bank. Since the program operated in countries from two different Regions (Africa and the Middle East and North Africa), different staff were involved. In addition, ASP-P1 was supervised by a program-level Task Team Leader (TTL) with full budget authority, a program-level co-TTL responsible for francophone ASP-P1 participating countries, and co-TTLs at the country-level with no budget authority. Particularly during the first three years, this arrangement translated into confusion among staff, partners, and Project Management Units (PMUs), cost overruns, and lengthy World Bank review and clearance processes of project-related documents. The program also experienced a high turnover in World Bank TTLs,39 requiring transition periods for hand-over activities. Lastly, the World Bank’s internal systems were challenging for administration of an umbrella program with multiple sub-projects in more than one region, ultimately causing high transaction costs and inaccurate reporting. Efforts were undertaken to solve these system-related challenges, but without success.40

• Insufficient fundraising efforts. Higher than estimated stocks of obsolete pesticides and contaminated soils together with the depreciation of the US dollar resulted in a funding gap for the ASP-P1 disposal component of at least US$10.6 million by 2010.41 The lack of funds was subsequently mitigated by formally revising the PDO and associated indicators in four of the ASP-P1 participating countries where gaps existed. However, significant efforts to secure additional funds were not pursued by the partners and ultimately resulted in non-disposal of some of the inventoried obsolete pesticides and associated wastes.

• Procurement of disposal services represented a challenge in all ASP-P1 participating countries: Procurement of the highly specialized disposal services under World Bank procurement guidelines was a challenging and time-consuming process, which was exacerbated by weak national procurement capacity in almost all ASP-P1 participating countries (see Section 2.4 in the Technical Annexes).

39 Approximately eight different TTLs led the program during preparation and supervision. 40 The second and third set of country-level projects (ASP-P1 Mali, Morocco, Tanzania (P103189); and ASP-P1 Ethiopia (P105711)) could not be processed under the project number which was used for ASP-P1 South Africa and Tunisia (P075776) since each Board presentation required a separate number. Resulting recording and reporting challenges caused high administrative transaction costs, and did not allow for a detailed expenditure overview at the program-level. 41 For more detail see Independent Evaluation of Design and Initial Implementation of Africa Stockpiles Program – Evaluation Report, May 2010, COWI – Document no. 69292-A-1

12

• The Level 1 restructuring in four ASP-P1 countries positively affected implementation and allowed eventual PDO achievement in three countries: At mid-term review, it was recognized that the PDO was unlikely to be achieved by the original closing date. It was agreed to restructure ASP-P1 to better reflect country-specific environments. With the discovery of higher than budgeted stocks of obsolete pesticides, including associated wastes, it was decided to adopt a blended approach of disposal overseas and lower cost safeguarding in-country based on risk profiling of the inventories of pesticides and associated wastes, and the availability of satisfactory storage facilities (i.e., obsolete pesticides and higher risk stocks of heavily contaminated soils being disposed of overseas while lower risk stocks of associated wastes being safeguarded in-country). In addition, the original PDO had assumed no new accumulations of obsolete pesticides during the life of the project, which had been proven difficult to achieve in the context of agricultural intensification and food security concerns; therefore it was agreed to focus on developing a strategy for management of future accumulations. The restructuring, which was approved in 2011 for ASP-P1 Ethiopia, ASP-P1South Africa, ASP-P1 Tanzania, and ASP-P1 Tunisia, eventually, allowed the countries (except ASP-P1 South Africa) to achieve targeted outcomes.42

26. Learning Review of Regional Projects: A review conducted by the World Bank in 2009, rated the program moderately unsatisfactory. The review suggested to: (i) strengthen the program-level results framework and define country-specific ones; (ii) clarify roles and responsibilities among partners; and (iii) ensure low TTL turnover.43 Furthermore, an independent evaluation of the program’s design and initial implementation, finalized in June 2009, stated that while the program’s partnership was valid, it also was a challenge. It stated that each of the specialized partners was characterized by differing organizational cultures, traditions, practices, procedures, and interests, and thus making a close collaboration challenging. The report emphasized the importance of finding common solutions on tasks and procedures. 44

2.3 MONITORING AND EVALUATION (M&E) DESIGN, IMPLEMENTATION, AND UTILIZATION

27. Overall M&E – Rating Moderately Unsatisfactory. 28. M&E design – Rating Unsatisfactory: At the program-level, a complex M&E system, including a

results framework, had been developed and was described in detail in the PAD and Operational Manual. However, the results framework and arrangements for results monitoring showed slight discrepancies, and PDO and associated indicators were vaguely formulated and did not cover all expected outcomes, having made it difficult to assess impacts. Moreover, M&E systems specific to the individual ASP-P1 participating countries were not prepared during preparation. Country-specific results frameworks, intended to be finalized by the TSU during project launch, were not prepared;

42 For ASP-P1 Mali, the Level 1 restructuring was prepared but not approved because of the March 2012 coup d’état and subsequent suspension of activities. For ASP-P1 Morocco, a Level 1 restructuring was not prepared because the project closed as originally scheduled in 2010. For ASP-P1 South Africa, a Level 1 restructuring was approved but the late request for an extension of the closing date, eventually prevented PDO achievement. 43 AFRICA: 3A-Africa Stockpiles 1 MMT GEF (FY07) (P0103189): Learning Review of Regional Projects, December 11, 2009. The assessment reviewed ASP-P1 on the basis of three of the participating countries –Mali, Morocco, and Tanzania 44 For more detail see Independent Evaluation of Design and Initial Implementation of Africa Stockpiles Program – Evaluation Report, May 2010, COWI – Document no. 69292-A-1

13

only country-specific indicators linked to the different project components were developed, but these did not allow for adequate monitoring of project implementation progress as they focused on measuring outputs as opposed to progress towards project outcomes.

29. M&E implementation and utilization – Rating Moderately Unsatisfactory: ASP-P1 lacked systematic monitoring and reporting at the program-level throughout implementation, largely due to failure to establish the PCU, which among other tasks, was expected to coordinate the independent M&E of the program. The ASPIC attempted to monitor implementation of activities during its regular meetings, and the World Bank produced comprehensive annual consolidated technical and financial reports, which were disseminated to all partners and donors. However, these reports did not include systematic monitoring of implementation progress against the PDO and Intermediate Outcome Indicators; thus, overall progress (including the technical assistance and cross-cutting activities intended to be implemented by FAO and the NGOs) was difficult to assess.

30. At the country-level, until mid-term review, ASP-P1 participating countries operated without an M&E system in place. Eventually in 2009, the TSU provided the countries with an M&E tool, however, it was considered too complex by some of the countries, and introduced too late in the process to be used efficiently. Only in 2011, a results framework was introduced in four ASP-P1 participating countries as part of their restructuring, which strengthened the M&E of the projects.

2.4 SAFEGUARDS AND FIDUCIARY COMPLIANCE

31. Safeguard compliance – Rating Satisfactory: ASP-P1 triggered two safeguard policies: OP 4.01 on Environmental Assessment and OP 4.09 on Pest Management. In accordance with the World Bank’s environmental and social policies and due to potentially significant adverse environmental impacts, ASP-P1 was classified as Category A, requiring full assessment. Environmental and social impacts were adequately assessed in a detailed Framework Environmental Assessment (FEA), as were their mitigation measures in an Environmental and Social Management Framework (ESMF). Formal country consultations were held in Addis Ababa (April 2002) to discuss details of the program’s design, which led to strong endorsement of the approach.45 An Environmental and Social Assessment Synthesis Report was prepared based on these documents, adequately consulted in the ASP-P1 participating countries, and disclosed to the World Bank’s InfoShop in March 2004.46 The document provided guidance to the ASP-P1 participating countries on how to prepare country-specific safeguard instruments, including CESAs. For safeguard compliance at the country-level, see Technical Annexes.

32. ASP-P1 did not finance the procurement of pesticides or any other agricultural chemicals, nor did it lead to their increased use. The prevention component was initially designed to reduce reliance on pesticide use by promoting IPM, improving pesticide procurement and management systems and building capacity to prevent future accumulation of pesticides. To meet the World Bank's safeguards requirements on Pest Management, the program needed to promote IPM, improve pesticide management, and build capacity to address Integrated Pest Management (IPM) and pesticide management. ASP-P1 provided adequate training to all participating countries on promoting IPM, on

45 The meeting brought together representatives from Botswana, Cameroon, Cote d’Ivoire, Ethiopia, Lesotho, Mali, Morocco, Mozambique, Namibia, Niger, Nigeria, South Africa, Swaziland, Tunisia and Tanzania. 46 The same document was posted on the World Bank’s Public Information Center (PIC) on June 29, 2005, and distributed to the Board on July 4, 2005.

14

the adherence to the International Code of Conduct on pesticide distribution and use, and on using the Pesticide Stock Management System (PSMS) (all countries have access to the system and many have been using it beyond ASP). The project did not include country specific IPM activities, since they were outside the scope of the project. The activities proposed by ASP-P1, which were executed in all participating countries largely met the requirements of the pest management policy.

33. Financial Management compliance – Rating Satisfactory: As the signatory of all ASP-P1 GEF trust fund grant agreements, as well as the CIDA and MDTF administration agreements, the World Bank was adequately supervising the use of these funds through regular missions, review of progress and financial reports, audits, follow-up meetings, and regular contact as needed. The World Bank produced monthly financial reports, which could be accessed by authorized users via the World Bank donor portal, or upon request.

34. Procurement – Rating Not Applicable: The vast majority of ASP-P1 procurement activities was carried out by the ASP-P1 projects at the country-level and details can be found in Section 2.4 in the Technical Annexes. The ASP partners each used their own procurement procedures to procure necessary services, consultancies and pay for operating costs without any reported problems.

2.5 POST-COMPLETION OPERATION/NEXT PHASE

35. Following the disapproval of ASP-P2 for funding by GEF (due to a lack of resources at the end of a GEF replenishment cycle at the time and poor implementation performance of ASP-P1), CLI, FAO, PAN-UK and the World Bank began to individually develop and implement pesticide-related projects. These projects have been implemented outside the ASP-P1 framework, but have been using ASP guidelines, tools, and experience, more specifically:

• CLI has supported projects in Cameroon, Ghana, Kenya, Malawi, and Nigeria, which safeguarded a total of 811 tonnes of high-risk obsolete pesticides and containers.

• FAO has implemented projects in other countries including Mozambique, and currently conducts a disposal and prevention project in several West African countries using GEF and other donor resources.47

• The World Bank has been implementing a POPs project in Egypt, to help the government meet its obligations under the Stockholm Convention, and contribute to capacity. The World Bank is also currently discussing a possible follow-up project in Mali, which would focus on finalizing prevention and disposal activities.

• PAN-UK has been strengthening civil society in ASP-P1 countries through capacity building and training in health and ecotoxicology monitoring, and supporting prevention activities.

3. ASSESSMENT OF OUTCOMES

3.1 RELEVANCE OF OBJECTIVES, DESIGN, AND IMPLEMENTATION

36. Relevance – Rating High: The elimination of obsolete pesticide stocks remains a development priority. Rural communities cannot hope to advance if the soil and water, upon which their

47 The project is conducted in Comité permanent Inter-Etats de Lutte contre la Sécheresse dans le Sahel (CILSS) member countries such as Burkina Faso, Cape Verde, the Gambia, Guinea Bissau, Mali, Mauritania, Niger, and Senegal.

15

livelihoods and health depend, are contaminated with pesticides, and urban populations cannot prosper if they are suffering from severe illnesses caused by pesticide poisoning.

37. National development strategies of ASP-P1 participating countries confirm the importance of eliminating hazardous waste (see Section 3.1 in Technical Annexes), as do World Bank priorities. The World Bank’s environment strategy from 2001,48 which guided the institution’s work in this sector over the past decade highlighted pesticide residues as environmental and health risk factor, and today, the World Bank’s dual goal of ending extreme poverty and promoting shared prosperity is to be achieved in an environmentally sustainable manner, taking into account the impacts of climate change, environmental degradation, and ecosystem changes.49 The World Bank’s current environment strategy places particular attention on helping countries to address environment-related health issues by supporting the creation of regulatory, economic and financial incentives to reduce pollution and increase clean production.50 In Africa, the World Bank aims to ensure the availability of transparent environmental regulatory frameworks and competent institutions for monitoring environmental compliance. New analytical work at the regional level on hazardous waste, POPs, and the clean agenda is expected in the future.

3.2 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVE

38. This assessment is based on indicators captured in the results framework of the PAD.51 In 2011, project activities in four of the ASP-P1 participating countries were restructured, as it was recognized that their design required revision to permit PDO achievement; the restructuring subsequently permitted PDO achievement in three ASP-P1 countries. These revisions were, however, not captured at the program-level, as the PDO, associated indicators, and components at the program-level were not formally restructured.

39. PDO achievement – Rating Moderately Unsatisfactory: This assessment takes account of one PDO Outcome Indicator, which was partly (37%) achieved.52 It also takes account of ten Results Indicators, of which four were fully (100%), four partly (50-83%), and two not (0%) achieved.53 The assessment is framed by the program’s envisaged outcomes (i.e., risk reduction and pesticide disposal, improved pesticide management, and service delivery to ASP-P1 countries).

40. Risk reduction and pesticide disposal. The level of risk emanating from publicly held obsolete pesticides and associated wastes to nearby communities and critical natural resources was successfully determined and quantified through the completion of national inventories in five of the ASP-P1 participating countries evaluated in this ICR (Results Indicator #1 – 83%).54 The information obtained (location and state of protection of the storage sites, contaminant type and quantity, and

48 See Making Sustainable Commitments – An Environment Strategy for the World Bank, June 18, 2001, World Bank 49 See World Bank Group Strategy, August 16, 2013, World Bank 50 See How Environment Sustains Development – 2011 World Bank Group Environment Strategy: Toward a Green, Clean and Resilient World for All, January 4, 2012, World Bank 51 The indicators presented in the results framework differ from those presented in the arrangements for results monitoring. 52 The second PDO Outcome Indicator was not measurable; the third PDO Outcome Indicator was not taken into account as it corresponded with the second PDO Outcome Indicators. 53 Three additional Results Indicators were not taken into account as one was not applicable, one corresponded with a PDO Outcome Indicator, and one had already been achieved prior to program approval. 54 The originally planned countrywide inventory of publicly held obsolete pesticides and associated waste was not carried out in ASP-P1 South Africa, as it was considered cost-ineffective in an environment where most stocks were held by private farmers and not by the public sector (see Technical Annex IV for details).

16

general condition of the stockpiles) provided the governments with the baseline data and knowledge necessary to inform eventual disposal of stockpiles.

41. Despite the program’s complexity, challenges, and poor initial performance, mitigation efforts eventually resulted in complete disposal of inventoried publicly held obsolete pesticides in ASP-P1 Ethiopia, ASP-P1 Tanzania, and ASP-P1 Tunisia, having significantly reduced risks to the environment and public health. A total of 3,164 of the inventoried 8,949 tonnes of publicly held obsolete pesticides and associated waste were eliminated overseas in an environmentally and technically sound manner (PDO Outcome Indicator #1 – 35%)55 by contracted specialized companies and in line with national and international regulations (Results Indicator #3 – 50%).56 In South Africa, where obsolete pesticides were predominantly privately held, ASP-P1 succeeded, using CLI funding, in eliminating obsolete pesticides in one pilot province, totaling approximately 100 tonnes.

42. Improved pesticide management. The original target of not accumulating new obsolete pesticide stockpiles and associated waste was not measurable. At appraisal, it was estimated that approximately 4,750 tonnes of obsolete pesticides were scattered across the ASP-P1 countries; inventories conducted during implementation in the ASP-P1 participating countries revealed the existence of approximately 800 additional tonnes of obsolete pesticides. However, it is difficult to assess whether further stockpiles were accumulated during program implementation or whether the difference resulted from inaccurate estimates at the time of appraisal (PDO Outcome Indicator #2 – N/A).

43. The program was also successful in raising awareness on the health and environmental hazards of pesticides and encouraging safe pesticide handling through various outreach activities across ASP-P1 participating countries. An overarching communication strategy was developed by WWF, together with a communication strategy toolkit. On the basis of these tools and with support from WWF, national communication strategies were subsequently developed by all; however, they were only implemented by four ASP-P1 participating countries (Results Indicator #11 – 67%). Overall, WWF and PAN-UK actively participated in project activities related to awareness raising, monitoring, and communication (Results Indicator #10 – 100%).

44. Service delivery to ASP-P1 countries. The TSU, hosted by FAO, was fully funded and staffed shortly after the program became effective (Results Indicator #6 – 100%). However, it should be noted that the technical assistance the ASP-P1 participating countries were expected to receive, was not provided to the extent foreseen in the PAD, particularly in the areas of M&E, and preparation of contracts (e.g., preparation of technical specification and carrying out technical evaluations for project consultants and contractors). This lack of technical support was compensated with technical experts, temporarily funded by the World Bank, and the recruitment of TADs financed by CLI. A survey, specifically targeted at measuring ASP-P1 participating countries’ satisfaction with services received from partners, was not conducted (Results Indicator #7 – 0%).

45. The program did not succeed in launching ASP-P2. In April 2009, the World Bank and FAO jointly submitted a funding proposal to the GEF for US$20.7 million in support of ASP-P2. The second

55 Only eliminated obsolete pesticides and associated waste stockpiles are included. The stocks that were safeguarded (as alternative to elimination following the Level 1 restructuring in four ASP-P1 participating countries), and those that were held privately in the case of ASP-P1 South Africa, are not included in the assessment of PDO Outcome Indicator #1. 56 Disposal contracts were completed for ASP-P1 Ethiopia, ASP-P1 Mali, ASP-P1 Tanzania, and ASP-P1 Tunisia, but only implemented in ASP-P1 Ethiopia, ASP-P1 Tanzania, and ASP-P1 Tunisia. In South Africa, privately held obsolete pesticides were disposed of overseas in line with national and international regulations, however, with co-financing provided by CLI.

17

series of projects of the program was planned to be implemented in 22 additional African countries, and the proposal reflected recommendations obtained through the independent evaluation previously mentioned. However, ASP-P2 was not accepted by GEF due to a lack of resources at the end of a replenishment cycle at the time, and the poor performance of ASP-P1; instead, FAO and the World Bank were encouraged to submit funding requests for individual country or regional ASP projects once GEF funds became available (Results Indicator #8 – 50%). See Section 2.5 for details.

46. The PCU did not become operational due to a lack of capacity within NEPAD, the entity that had been selected to host the unit (Results Indicator #13 – 0%). Consequently, ASP-P1 was not successful in strengthening regional capacity for management of future program-related activities.

47. To inform sustainable decision-making in the ASP-P1 participating countries, a study on disposal technology options was prepared by WWF in June 2008. The study provided information on the treatment and disposal options of obsolete pesticides and associated wastes. Furthermore, a study on locust control issues was prepared by FAO as part of the Red Locust Control Program (Results Indicator #9 – 100%).

48. While implementation was not systematically monitored at the program-level, the World Bank produced technical and financial reports on a yearly basis. These reports provided annual overviews of implementation progress achieved by the program and ASP-P1 participating countries, and were shared with partners and donors (Results Indicator #12 – 100%).

3.3 ACHIEVEMENT OF GLOBAL DEVELOPMENT OBJECTIVE

49. ASP-P1 aimed to directly contribute to the GEO of reducing the effects of POPs on the global environment through: (i) improving the quality of life in poor communities by reducing environmental risks; (ii) improving environmental protection; and (iii) enhancing the capacity of the agricultural sector to better manage crop pests. GEO achievement is measured by PDO achievement, and, thus is also rated Moderately Unsatisfactory.

3.4 EFFICIENCY

50. Efficiency – Rating Low: This rating represents a broad estimate, as financial and economic data to support this rating are scarce. At appraisal, neither an economic nor a financial analysis was undertaken for ASP-P1 at the program or at the country-level due to non-measurable benefits of reducing the risk of environmental contamination. A lack of reliable baseline data in combination with controversial aspects related to the valuation of human life and natural resources did not permit a cost-benefit analysis. For GEF purposes, the preparation of ASP-P1 at the program-level included an incremental cost analysis for achieving global benefits to address the issue of obsolete pesticides in the participating countries.

51. In general, ASP-P1 resource use could not be accurately evaluated because: (i) No accurate quantitative data were available on the financial and human resources invested in the program’s preparation by any of the partners;57 (ii) with an envisaged PCU that never became operational, there was no M&E of the resources spent by FAO available to the World Bank, and thus no accurate data available; and (iii) the World Bank’s budgeting system made it difficult to adequately capture

57 See Independent Evaluation of Design and Initial Implementation of Africa Stockpiles Program – Evaluation Report, May 2010, COWI – Document no. 69292-A-1.

18

allocated funds and associated expenditures of a regional program with sub-projects, ultimately causing high transaction costs and inaccurate reporting. In addition, efficiency of capacity building and prevention activities is difficult to quantify due to time disconnect between project support and actual results.

52. When specifically considering efficiency of the disposal component, the unit cost for disposal (US$/tonne disposed) is used for measuring efficiency. Based on approximately US$15 million spent on the disposal of 3,240 tonnes of obsolete pesticides between 1990 and 2004, it was estimated that 300-900 tonnes could be disposed of per annum at the cost of US$3,400 per ton. At closing, 3,164 tonnes of obsolete pesticides and associated waste had been disposed of at the cost of approximately US$3,103 per tonne, which represents a slightly lower cost than originally anticipated.58 Nevertheless, efficiency is rated low because, while ASP-P1 disbursed 75% of GEF resources, only 37% of inventoried publicly held obsolete pesticides and associated waste were disposed of at program closing.

3.5 JUSTIFICATION OF OVERALL OUTCOME RATING

53. Overall outcome – Rating Moderately Unsatisfactory: On the basis of the program’s relevance (high), efficiency (low), and PDO achievement (moderately unsatisfactory), the overall outcome is rated moderately unsatisfactory. This rating takes into account that the program’s preparation and implementation were accompanied by substantial challenges, but also suffered from shortcomings that significantly affected the envisaged outcomes. However, it should be highlighted that despite the program’s complexity, challenges, and poor initial performance, mitigation efforts, most notably a Level 1 restructuring, eventually permitted PDO achievement in three of the ASP-P1 participating countries. Risks to the environment and public health were significantly reduced through complete disposal of inventoried obsolete pesticides in ASP-P1 Ethiopia, ASP-P1 Tanzania, and ASP-P1 Tunisia. Table 3.1 briefly provides an overview of PDO achievements at the individual country-level; for details, see Technical Annexes.

ASP-P1 Ethiopia

ASP-P1 Mali

ASP-P1 Morocco

ASP-P1 South Africa

ASP-P1 Tanzania

ASP-P1 Tunisia

Outcome against Original PDO



Unsatisfactory Moderately

Unsatisfactory Unsatisfactory

Moderately Satisfactory

Outcome against Revised PDO


Not restructured

due to political situation

Closed as scheduled and not extended


Highly Satisfactory

Satisfactory

58 The calculation takes disposed amounts of obsolete pesticides and associated GEF resources spent in three ASP-P1 countries into account: (i) ASP-P1 Ethiopia disposed of 450 tonnes at a cost of US$1.70 million; (ii) ASP-P1 Tanzania disposed of 967 tonnes at a cost of US$5.07 million; and ASP-P1 Tunisia disposed of 1,730 tonnes at a cost of US$3.05 million. Costs include those associated with repackaging, transport, and incineration of obsolete pesticides and other related activities such as for example the inventory, safeguarding efforts, associated studies, and advisory services.

Table 3.1: Overview of ASP-P1 PDO Achievement at the Country-Level

19

Overall (Weighted Outcome)59


- - Moderately

Unsatisfactory Satisfactory Satisfactory

3.6 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS

54. Poverty impacts, gender aspects, and social development: The disposal of obsolete pesticides and associated wastes had a direct positive impact on poor and farming communities, species and ecosystems, located in proximity to the identified sites. Baseline data on affected populations, species, and ecosystems was not conducted at appraisal; exact impacts are therefore difficult to measure.

55. Institutional change/strengthening: Since the PCU had not become operational, institutional change and strengthening occurred at the country level: (i) the regulatory and legal frameworks for pesticide management were improved in Ethiopia, Mali, Morocco, Tanzania, and Tunisia; (ii) the logistics of hazardous waste transportation have been internalized in Ethiopia, Tanzania, and Tunisia, and the knowledge gained can inform future disposal activities; (iii) the experience and skills gained in managing the project can be applied to other projects and subsequently contribute to success; (iv) the experience gained in Ethiopia, Mali, South Africa, Tanzania, and Tunisia, in preparing social and environmental impact assessments and management plans; (v) knowledge gained on sound pesticide management through the various training courses all participating countries participated in (see Section 3.6 in Technical Annexes).

56. Other unintended outcomes and impacts: The private sector through CLI provided valuable technical support to ASP-P1 Ethiopia, Mali, South Africa, and Tanzania through the funding of international TADs. CLI further funded the disposal of stocks in Ethiopia, South Africa, and Tanzania; and played a key role in mobilizing South Africa’s crop protection industry to forming a public-private partnership with the government to dispose of obsolete pesticide stocks.

3.7 SUMMARY OF FINDINGS OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS

57. Beneficiary surveys were not carried out. ICR missions, including stakeholder workshops were carried out in Ethiopia and Tanzania (see Section 3.7 of corresponding Technical Annexes). All ASP-P1 participating countries provided completion reports, except ASP-P1 Morocco; information provided therein was incorporated in this ICR.

4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME

58. The risk to development outcome varies across ASP-P1 participating countries and has been assessed at the country-level. In Ethiopia and Tanzania, the risk is rated moderate and confined to the re-accumulation of obsolete pesticides, and a potential weakening of the governments’ commitment to sustaining and building on the projects’ outcomes; however, if improved legislations will be enacted, the governments are well positioned to sustainably manage and reduce the re-accumulation of

59 Overall weighted outcomes have been calculated by weighting the separate outcome ratings in proportion to the share of actual grant disbursements made in the periods before and after approval of the revision. See Section 3.5 in Technical Annexes for details.

20

obsolete pesticides. In Mali, the development outcome risk is rated high due to Mali's post-conflict and transitional situation, its strong focus on agricultural intensification, and the relatively low priority obsolete stock elimination is receiving in the country's recovery plans. In Morocco, risk is also rated high due to the government’s lack of capacity and resources to safeguard and dispose of its stocks. In South Africa, the risk is rated moderate given the uncertainty about the government and the private industry to approve and implement the newly developed Integrated Waste Management Plan (IWMP) to collect and dispose of both private and public stocks. Finally, Tunisia’s development outcome risk is rated low based on the government’s demonstrated commitment and capacity, as well as the improved legislation for pesticide management. See Section 4 of the Technical Annexes for further detail.

5. ASSESSMENT OF WORLD BANK AND BORROWER PERFORMANCE

5.1 WORLD BANK PERFORMANCE

59. World Bank performance in ensuring quality at entry – Rating Moderately Unsatisfactory: The World Bank together with the other partners conducted numerous preparatory missions to all participating countries, which resulted in considerable background information, project plans and candid analysis of issues and challenges. FM and procurement assessments were also adequately conducted for all countries (see Technical Annexes for details), as well as for WWF, PAN-UK and PAN-Africa, which were to be recipients of DGF and MDTF funds during project implementation. However, although the PADs were comprehensive and described in detail the envisaged activities at the program-level, they were lacking country-specific details, most notably country-specific results frameworks.

60. Preparation at the program-level was, lengthy due to difficulties in finding common agreement among partners on the program’s design and implementation arrangements (see Section 2.1 for details). The World Bank’s authority during these negotiations was undermined by a high turnover in TTLs. The challenging partnership-building process ultimately resulted in the PADs giving insufficient attention to specifying and defining the collaboration including oversight, and roles and responsibilities of each partner involved, which significantly affected program- and country-level implementation (see Section 2.1 for details).

61. Quality of World Bank supervision – Rating Moderately Unsatisfactory: Issues caused by the lack of a PCU and unclear definition of the partnership arrangement at the program-level, dominated the first three years of program implementation, and equally affected project implementation at the country-level. Lengthy discussions between FAO and the World Bank on standard formats and guidelines, which ASP-P1 participating countries needed for project implementation, consequently delayed implementation progress. 60 In 2008, corrective action in form of a jointly written memo, which defined exact roles, responsibilities, and procedures, alleviated some of the difficulties; however, disagreements remained on a few issues (see Section 2.2 for details).

62. Since efforts to create an interim PCU secretariat had failed, the World Bank team proactively substituted for a number of the PCU tasks in order to ensure satisfactory program implementation. In

60 Standard documents, Terms of Reference (TORs), and procurement guidelines, required for project implementation and originally agreed on in the context of the ASP-P1 Operational Manual were reconsidered during implementation.

21

addition, the World Bank’s internal implementation arrangements were challenging, due to the need to bridge two World Bank administrative regions, a program level TTL and multiple co-TTLs, which finally often translated into lengthy review clearance processes of project-related documents.61 Lastly, the World Bank’s internal systems provided challenges for administering an umbrella program with multiple sub-projects in more than one region, ultimately causing high transaction costs and inaccurate reporting. Efforts were undertaken to solve these system-related challenges, but without success.

63. Following mid-term reviews in 2008 and 2009, the World Bank team increased its efforts to effectively address the program’s implementation delays and maximize the likelihood of program success in ASP-P1 participating countries. Corrective actions included increased and more targeted operational and technical support through consultants, extension of closing dates, and reallocation of existing GEF funds to maximize outcomes. Project design weaknesses were addressed through the restructuring of four of the ASP-P1 participating countries, by both revising targets and better aligning project activities with country-specific environments (for details see Section 5.1 in the Technical Annexes).

64. Overall World Bank performance – Rating Moderately Unsatisfactory: The program suffered from shortcomings in preparation and design as well as from a PCU that did not become operational, which affected implementation at the program- and subsequently country-level, evident through substantial implementation delays. However, it must be acknowledged that the program was complex and particularly challenging in its implementation; and despite its complexity, challenges, and poor initial performance, the World Bank took important actions to improve implementation progress at the country-level, and eventually succeeded in supporting three participating ASP-P1 countries to achieve envisaged outcomes.

5.2 BORROWER PERFORMANCE

65. Overall Borrower performance – Rating Moderately Satisfactory: Borrower performance at the country-level varied across participating ASP-P1 countries; the rating represents an average. Table 5.1 provides an overview of borrowers’ individual performance. For details on performance at the government and implementing agency-level, refer to Section 5.2 of the individual Technical Annexes.

ASP-P1 Ethiopia

ASP-P1 Mali

ASP-P1 Morocco

ASP-P1 South Africa

ASP-P1 Tanzania

ASP-P1 Tunisia

Government Performance


Satisfactory Moderately

Unsatisfactory Satisfactory Satisfactory Satisfactory

Implementing Agency Performance





Overall Borrower Performance





61 The program was affiliated to two Regions (Africa and the Middle East and North Africa) with different procedures and regulations, and the supervision arrangement consisted of a program-level TTL with full and a co-TTL with no budget authority, which often caused confusion among staff. The program also experienced a high turnover in TTLs, requiring transition periods for hand-over activities.

Table 5.1: Overview of Borrower Performance at the Country-Level

22

6. LESSONS LEARNED

66. Lessons learned that were drawn at the program-level and could serve as guidance for similar programs in the future include the following:

• The World Bank is not presently well equipped to handle multi-regional and multi-country programs. The World Bank's internal systems are not geared for administering an umbrella program with multiple sub-projects in more than one region. Possible World Bank support for similar approaches in the future should weigh transaction costs and efficiency constraints in designing regional programs. Unless strong justification is provided for designing the program with country-specific - or at least region-specific - projects should be considered. Furthermore, the World Bank's internal processing arrangements for regional programs should be made sufficiently flexible to accommodate their characteristics and needs rather than applying coding and budget systems that hinder successful implementation.

• For multi-stakeholder projects, formalized and defined roles and responsibilities for all partners and their collaboration need to be agreed and operationalized prior to project effectiveness. ASP-P1 was a complex program, which depended on close collaboration and cooperation among several partners and drawing upon their comparative expertise in pesticide management, project design, implementation, communications, and monitoring and evaluation. To ensure clarity on, and commitment to, each entity’s roles and responsibilities, together with guidance on how the partnership should function, should be defined and agreed on during project preparation. Concise time and budget bound Terms of Reference (TORs) for all partners, and a clearly defined oversight and coordination entity would have greatly improved project design and implementation.

• ASP-P1 would have benefited from a design that (i) defined envisaged outcomes and PDOs at the program-level, and (ii) individually-tailored PDOs at the country-level. The uniform approach ASP-P1 applied to all participating countries caused significant delays and transactions costs, and is not desirable. Future programs should place emphasis on tailoring individual country projects to the country-specific environments.

• The dimension of the task should be carefully determined, and projects should be sufficiently well-prepared before implementation. Project activities such as a national obsolete pesticides inventory and a CESA are lengthy processes but crucial for determining the dimension of the disposal task ahead. Ideally, these activities should not be conducted during implementation; rather, they should either be conducted during preparation, or during a first implementation phase, followed by a second that comprises adequately timed and budgeted disposal activities. This could include an approach under which the inventory and the Country Environmental and Social Assessment (CESA) are prepared simultaneously.

• While it is important to determine the dimension of the task before implementation, it is equally important to develop a flexible implementation approach. With the volume of obsolete pesticides to be disposed of being a shifting target, especially in larger and infrastructure-poor countries, it is important to develop a flexible implementation approach that can compensate for actual quantities that are larger or smaller than those estimated at the time of appraisal.

23

• The program's initial assumption that no further obsolete pesticides would be accumulated was unrealistic in view of food security concerns and ongoing agricultural intensification. Prevention activities are crucial for any future operations and need to be sufficiently reflected in the project’s PDO and associated indicators. Prevention activities were not pursued to the extent disposal activities were implemented, mainly as a result of not having been sufficiently reflected in the indicators.

• Realistic and clear indicators and preparation of a reasonable results framework are of paramount importance. The PDO and associated indicators did not cover all expected outcomes that were expected to be achieved. In several instances they were vaguely formulated, making it difficult to assess progress and outcomes.

• Project plans should recognize the complexity of project tasks. Project activities such as a national inventory and a CESA proved to be lengthy processes. Since their completion was a requirement for disposal activities to begin, and the majority of funds was expected to be disbursed during disposal activities, disbursement remained low for most of project implementation. To avoid low disbursement levels being considered as low performance, project implementation plans and schedules need to be realistic.

• The participation of adequate technical experts is crucial during program preparation and implementation. Preparation of ASP-P1 would have benefitted from more specialized technical staff involvement, and, as confirmed by ASP-P1 sub-projects in Ethiopia, Tanzania, and Tunisia, technical advisors during disposal preparation activities proved to be very valuable and significantly contributed to successful disposal.

• Knowledge exchange between participating countries of a regional project should be facilitated. Preparation of the CESA in the individual participating ASP-P1 countries was a lengthy and complex task; the program would have benefitted from a central Country Environmental and Social Assessment (CESA) advisor. Furthermore, the procurement process could have been streamlined to contract disposal and CESA consultants for all countries to gain efficiency.

• It is important to comprehensively assess the capacity of African regional organizations before designating them as implementing agencies for donor-funded programs. The PCU did not become operational due to a lack of capacity within NEPAD, the entity that had been selected to host the unit. Consequently, ASP-P1 was not successful in strengthening regional capacity for management of future program-related activities.

7. COMMENTS ON ISSUES RAISED BY BORROWER, IMPLEMENTING AGENCY, AND PARTNERS

67. See Section 8.7 and 8.8.

24

8. ANNEXES

8.1 PROJECT COSTS AND FINANCING

a. Project cost by component – All sources (in US$ million equivalent) – as of 06/30/2013 Component

Appraisal Estimate (in US$ million

equivalent)

Actual/Latest Estimate

(in US$ million equivalent)

Actual as % of appraisal estimate62

1. Disposal of obsoletes pesticides 51.37 N/A N/A

2. Prevention of accumulation 4.31 N/A N/A

3. Capacity building 2.45 N/A N/A

4. Project management 1.88 N/A N/A

Total 60.00 N/A N/A

b. Financing – as of 06/30/2013 Sources of Funds Type of

Co-financing


equivalent)

Actual/Latest Estimate (in US$ million

equivalent)

Actual as % of appraisal

estimate

Global Environment Facility (GEF)

World Bank Grant 21.7 16.3 75%

FAO Grant 3.3 3.3 100%

Development Grant Facility (DGF)

WB – FY05 Grant 1.5 1.5 100%

WB – FY06 Grant 1.2 1.2 100%

Trust Fund Canada Parallel Financing

2.3 2.3 100%

Multi-Donor Trust Fund (Denmark, EU, Sweden, and Switzerland)

Parallel Financing

7.1 4.2 59%

Bi-later Financing

Belgium

Parallel Financing

4.1 N/A N/A

Finland 1.0 N/A N/A

France (FFEM) 1.8 1.7 94%

Japan 1.1 N/A N/A

The Netherlands 1.0 1.0 100%

Trust Fund African Development Bank

Parallel Financing

10.0 0 0%

FAO Parallel Financing

0.1 N/A N/A

62 Actual/latest estimates of cost by component and sources of funds were not available.

25

ASP-P1 Participating Countries’ Governments63

In-Kind 3.8 2.7 71%

CropLife International (CLI) Parallel Financing

0 2.1 -

Total 60.0 N/A N/A

8.2 OUTPUTS BY COMPONENT

1. The following table provides a comprehensive account of qualitative and quantitative information of outputs realized against original components (achievements at the time of program closing in June 2013).

Original Components as presented in the PAD

Outputs achieved at the time of program closing in June 2013

Component 1 - Country operations: Achievements under this component are rated Moderately Unsatisfactory This was the principal component of the program, encompassing a full range of cleanup and prevention activities for ASP-P1 participating countries. a. Country cleanup and disposal b. Prevention activities c. Capacity building d. Country project management and

monitoring

Achievements under this component varied across the ASP-P1 participating countries For a detailed account of outcomes achieved under this component in each ASP-P1 participating country, please see respective Technical Annexes. a. Country cleanup and disposal • An inventory database is in place in and being used by all participating ASP-

P1 countries, except for ASP-P1 South Africa where it was considered cost-ineffective by the government (Results Indicator #1 – 83%).

• CESA were completed, and publicly consulted and disclosed in five ASP-P1 participating countries (Ethiopia, Mali, South Africa, Tanzania, and Tunisia). Activities recommended therein were implemented in four ASP-P1 participating countries (ASP-P1 Mali was closed before disposal activities began in 2012 due to the unstable political situation in the country) (Results Indicator #2 – 66%).

• Disposal contracts, in line with national and international regulations, were completed in three ASP-P1 participating countries (Ethiopia, Tanzania, and Tunisia) (Results Indicator #4 – 50%).

• Overall, 3,164 of the inventoried 8,543 tonnes of publicly held obsolete pesticides and associated waste were disposed of in four ASP-P1 countries (PDO Outcome Indicator #1 – 37%).

b. Prevention activities • The original target of not accumulating new obsolete pesticide stockpiles

and associated waste was not measurable. At appraisal, it was estimated that approximately 4,750 tonnes of obsolete pesticides were scattered across the ASP-P1 countries; inventories conducted during implementation in the ASP-P1 participating countries revealed approximately 800 additional tonnes of obsolete pesticides. However, it is difficult to assess whether further stockpiles were accumulated during program implementation or whether the difference resulted from inaccurate estimates used at appraisal (PDO Outcome Indicator #2 - N/A).

• The International Code of Conduct on pesticide distribution and use was

63 Excludes ASP-P1 Morocco and South Africa, as data was not available.

Table 8.1: Outputs by Original Components as presented in the PAD

26

adopted by all ASP-P1 participating countries during a FAO council session in 2002 (Results Indicator #5 – 100%).

c. Capacity building • See Technical Annexes.

d. Country project management and monitoring • See Technical Annexes.

Component 2 – Technical support: Achievements under this component are rated Moderately Unsatisfactory FAO hosted a TSU, which was tasked with coordinating the technical input to country projects, assuring the technical quality of country projects, and consolidating M&E data at the program level. a. Preparation of Country Operational

Manuals b. Preparation for disposal of obsolete

pesticides • Support the design of strategies for

the effective and safe disposal of obsolete pesticide stocks and associated waste, health and safety plans.

• Support the development of the CESAs including EMPs.

• Contract preparation – the TSU will support countries in preparing technical specifications and carrying out technical evaluations for project managers, consultants, and contractors for obsolete pesticide prevention and disposal activities.

• Advise on alternative disposal options, targeted to specific country or site conditions.

c. Prevention of accumulation • Support legislative and regulatory

reforms to enhance pesticide management.

• Support the identification of weaknesses in the enforcement of legislation and enhance enforcement measures.

• Support the design of enhanced pest management strategies to improve efficiencies and to reduce reliance on pesticides in various sectors.

The TSU was fully funded by two grants through FAO (Government of the Netherlands and GEF) and staffed; however, it exhausted its funds prior to the beginning of disposal activities (Results Indicator #6 – 100%). a. Preparation of Country Operational Manuals. Operational Manuals were

prepared by the ASP-P1 participating countries in collaboration with the TSU, with varying levels of quality.

b. Preparation for disposal of obsolete pesticides. • In collaboration with the World Bank, the TSU prepared three guidance

notes to help ASP-P1 participating countries implement the project. Guidance notes were prepared on planning and M&E; procurement of disposal services; and preparation and implementation of the CESAs.

• The TSU further supported, reviewed and technically approved the CESAs, EMPs, and disposal bidding documents, provided training on supervision of safeguarding and disposal operations, soil decontamination techniques, and operationalized FAO’s Pesticide Stock Management System (PSMS) in all ASP-P1 participating countries, except ASP-P1 South Africa. Training was also provided in pesticide safeguarding. However, the nature and level of support provided to participating countries by the TSU, varied across ASP-P1 participating countries (see Technical Annexes for more detail). To compensate for the insufficient level of technical support in some of the participating countries, the World Bank financed technical experts that provided temporary support in the areas of safeguarding, operations, and pesticide management.

• Guidance on alternative disposal options was included in the Environmental Management Tool Kit (EMTK - Volume 3). However, it was not clear, how relevant this guidance was to the participating countries since it had been agreed that all countries would dispose of their stocks at incineration facilities overseas.

• Noteworthy is also the provision of a TAD to almost all ASP-P1 participating countries. The TADs were financed by CLI, and proved to be very value adding and crucial in preparing for and overseeing the disposal of obsolete pesticides.

c. Prevention of accumulation • The TSU did not support prevention activities at the country-level to the

extent outlined in the PAD. d. Monitoring and Evaluation • Project launch workshops were successfully supported and held in all ASP-

P1 participating countries (Tunisia – November 2005; South Africa - June 2006; Tanzania - October 2006; Morocco – November 2006; Mali – December 2006; and Ethiopia –September 2007).

• Supervision support, however, was not provided as planned. Supervision

27

• Support the design and implementation of appropriate container management strategies.

d. Monitoring and Evaluation • Support country project-launch

workshops and supervision support with the World Bank at least twice per annum.

• Support countries to implement the detailed M&E system and enquiry system.

• Support annual workshops for the PMUs on lessons learned, and

• Support TSU management, including overall coordination with other ASP units, administration of TSU activities, including procurement, reporting and financial administration.

missions generally only took place once, instead of twice a year (missions conducted by the TSU were rarely announced and mission reports were not shared with ASP partners), and the collaboration between FAO and the World Bank was cumbersome, ultimately causing delays at the country-level. The TSU participated in monthly coordination conference calls and in-person meetings with the other ASP partners, and managed its procurement and financial administration. However, the TSU did not report on its activities to the other ASP partners. Rather, it provided limited input into some World Bank reports.

• Support on implementing the detailed M&E system was not delivered as planned. Country-specific M&E systems, intended to be finalized by the TSU during project launch and included in the Operational Manuals, were not provided until 2009. Eventually in 2009, the TSU provided some of the ASP-P1 countries with an M&E system; however, it was considered complex and ultimately not used by most ASP-P1 countries.

• Yearly ASP Stakeholder Forums took place as platform for stakeholders and ASP-P1 participating countries to share knowledge and experiences, and discuss questions and concerns; they were, however, organized by other partners (WWF, PAN-UK, and the World Bank).

Component 3 – Crosscutting activities: Achievements under this component are rated Moderately Satisfactory The objective of this component was to support ASP-P1 knowledge management, build the capacity of local NGOs in pesticide-related matters and identify alternative disposal technology and practices. This component was intended to be implemented by WWF and PAN-UK. a. Communications and knowledge

management (WWF). • The communication strategy will

be further refined during project implementation.

• A knowledge and information system will be designed and implemented to store and retrieve key documents and information.

• Treaty ratification and coordination with chemical convention activities: countries that had not yet signed or ratified the Stockholm Convention would be supported to do so.

• WWF would solicit private sector financial support for ASP in cooperation with the World Bank and FAO.

b. NGO capacity building (PAN-UK). • NGO capacity to address pesticide

Overall, NGO participation in project activities was achieved (Results Indicator #10 – 100%). a. Communications and knowledge management (WWF) • An overarching communication strategy was developed by WWF, together

with a communication strategy toolkit. On the basis of these tools and with support from WWF, national communication strategies were subsequently developed by all, however, only implemented by four ASP-P1 participating countries (Results Indicator #11 – 67%). WWF also coordinated the development of a program communication plan, which was partially implemented.

• WWF developed and maintained an ASP-P1 website during most of the program’s lifetime (2006-2011). Focal points were trained on the management and maintenance of their respective country pages hosted on the ASP website and trained in the use of an online platform that facilitated communication between representatives of participating countries.

• WWF published newsletters on the program’s activities, and produced a number of educational booklets for farmers and schoolchildren. It also developed radio public service advertisements and short videos that were aired on TV and radio in Ethiopia and Tanzania. Two workshops were held in Nigeria and Ethiopia in August 2007, to raise awareness for obsolete pesticides and the program, and to provide information on important safety issues.

• WWF participated in various conferences to promote ASP and pesticide policy advocacy. Alliances were developed with crop production boards (e.g., in the coffee sector and with academic and research institutions such as Innovations in Civic Participation (ICIPE)). Private sector financial support was not solicited by WWF; it was coordinated by CLI.

b. NGO capacity building (PAN-UK) • PAN-UK organized an NGO exchange event in Rabat in April 2007, with

28

management issues would be built through sharing lessons learnt at workshops, networking, and coordination of NGO activities.

• Preparing and disseminating operational guidelines for NGO involvement and training in ASP.

• Designing and implementing awareness raising activities and outreach program targeted at stakeholders.

c. Strategic studies (WWF). • Identifying alternative disposal

technologies for obsolete pesticides and remediation of contaminated soils and water resources.

• Identifying environmentally sound locust-control strategies and the safe maintenance of emergency pesticide stocks for the control of locust outbreaks, malaria, dengue fever, and other vector-borne diseases.

representatives from each ASP-P1 participating country; a regional agro-ecology training event for NGOs from West Africa (Mali, Morocco, and Tunisia) in 2007; and a capacity building training on fundraising approaches and strategies in 2008.

• PAN-UK finalized and disseminated a number of key publications: (i) An ASP NGO manual to present key issues associated with obsolete pesticides, and to provide information to help NGOs play a successful part in the implementation of national ASP-P1 projects; (ii) an ecological monitoring handbook and a guide to international conventions; and (iii) regular articles in PAN journals, various reports and publications on community monitoring, and translation of ASP posters and leaflets into local languages.

• CLI strengthened capacity building in both the public and private sector by promoting sustainable management of agrochemical waste. Support was provided to National CropLife offices in ASP-P1 participating countries, to become more closely engaged with the program. CLI further funded study tours to Kenya and Australia to provide ASP-P1 representatives from Ethiopia, Tanzania, and South Africa with information and first hand observations on private sector engagement in agrochemical waste management.

• PAN-UK supported, in collaboration with WWF, PAN-Africa and local NGO networks, the design of awareness raising activities in ASP-P1 participating countries to varying degrees. While support was provided to ASP-P1 Ethiopia, Mali, and Tanzania, PAN-UK was less active in Morocco, and Tunisia. Reasons were political environments in which local NGOs’ activities had been limited, and a lack of agreements determining the extent of responsibilities, roles and budgets between PMUs, and PAN-UK and WWF.

c. Strategic Studies (WWF) • To inform sustainable decision-making in the ASP-P1 participating

countries, a study on disposal technology options was prepared by WWF in June 2008. The study provided information on the treatment and disposal options of obsolete pesticides and associated waste (Results Indicator #9 – 100%).

• PAN-Africa conducted research on locust control operations in West Africa; the results were used to develop a toolkit for national NGOs to advocate and lobby for strengthening national policies and raising awareness for alternatives to chemical pesticides for locust control.

Component 4 – Project coordination: Achievements under this component are rated Moderately Unsatisfactory The objective of this component was to ensure overall coordination of ASP-P1, including the other three components and their activities. The PCU was planned to be transferred to an African organization – e.g., NEPAD. Components of the PCU included: a. Governance and coordination. • Providing secretarial support for

the ASP Steering Committee • Coordinating preparation of ASP-

The PCU intended to coordinate, monitor, and evaluate the implementation of ASP-P1 between the partners, and a Steering Committee (SC) intended to serve as advisory structure, were not established (see Section 2.2 for more details). Although NEPAD received funding to host the PCU, it did not assume this responsibility seemingly due to capacity issues (Results Indicator #13 – 0%). a. Governance and coordination • Instead, an unofficial governance body - the ASP-P1 Implementation

Committee (ASPIC) - was created to oversee the program’s implementation. Key partners met regularly using a combination of virtual and face-to-face meetings, and managed to establish a workable division of work and responsibilities. However, despite having been prepared, TORs for the ASPIC were never approved, and thus the committee was acting without

29

P2 • Ensuring effective management of

ASP financial resources and maintaining an audit regime in cooperation with the World Bank.

• Promoting ASP and liaising with donors and recipient countries.

b. Project Monitoring and Evaluation (M&E) • The PCU was intended to establish

and implement the overall ASP M&E system to ensure that the program would be successfully implemented to achieve the PDO and resources effectively and efficiently used. It was planned to consolidate individual Financial Management Reports (FMRs) and monitoring program progress against the program results framework and each components against its intended results, and to provide bi-annual consolidated ASP progress reports and work plans.

c. ASP-PCU management • Funds from the Multi-Donor Trust

Fund (MDTF) were to financially support the PCU for it to perform its central coordination functions. NEPAD was intended to be provided with a capacity development grant from the Development Grant Facility (DGF) to be prepared for taking on increasing responsibility throughout the life of ASP-P1.

formalized principles. • A proposal for ASP-P2 was prepared by FAO and the World Bank (in

collaboration with WWF, PAN-UK, and CLI), who jointly submitted a funding proposal to the GEF in 2009, for US$20.7 million. ASP-P2 was planned to be implemented in 22 additional African countries, and the proposal reflected the recommendations of the independent evaluation that had been undertaken for ASP-P1 (Results Indicator #8 - 100%) in 2008. ASP-P2 was not accepted by GEF due to a lack of resources at the end of a GEF replenishment cycle at the time, and poor performance of ASP-P1.; instead, FAO and the World Bank were encouraged to submit funding requests for individual country or regional projects once GEF funds became available.

• As the signatory of project and donor financial agreements, the World Bank was responsible for managing the program’s financial resources. Supervision was carried out through missions, audits, review of progress and financial reports, review of expenditures, and regular contact with governments and PMUs. Also, an independent program evaluation was commissioned by the World Bank at mid-term and was subsequently made available to all stakeholders in 2008.

• Yearly ASP Stakeholder Forums were organized by the ASP partners on a rotational basis, as platform for stakeholders and ASP-P1 participating countries to share knowledge and experiences, and discuss questions and concerns. The African Union, the Basel Convention Secretariat for francophone countries, NGO networks, and representatives from the donor community participated as well.

b. Project Monitoring and Evaluation (M&E) • An overall M&E system to systematically monitor progress against the

results framework and each component, was not established. However, overall progress was monitored through supervision missions, discussed during regular meetings of the ASPIC, and captured in yearly technical and financial reports prepared by the World Bank (Results Indicator #12 – 100%). Program-level Interim unaudited Financial Reports (IFRs) were not consolidated due to the multiplicity of funding resources and various project codes; however, the World Bank reported on and audited individual Trust Funds as part of its ongoing fiduciary responsibility.

• A survey to determine client satisfaction was not conducted (Results Indicator #7 – 0%).

c. ASP-PCU management • NEPAD was intended to host the PCU, which did not materialize due to a

lack in capacity. The PCU was subsequently initially taken on by the rotating chair of the ASPIC, then by a consultant hired by the World Bank and hosted by the TSU. In 2008 the PCU was to be placed in the World Bank office in Pretoria, South Africa. A recruitment effort however failed to identify a suitable candidate, and thus the PCU was never created.

30

8.3 ECONOMIC AND FINANCIAL ANALYSIS

2. As outlined in Section 3.4, an economic and financial analysis is not available.

8.4 WORLD BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS

3. Please refer to Section 8.4 in the Technical Annexes for an overview of Task Team Leaders (TTLs). Staff time and costs associated with program preparation and supervision was not possible to determine as a result of multiple coding for the program in the system.

8.5 BENEFICIARY SURVEY RESULTS

4. Beneficiary survey results are not available, as no survey was conducted.

8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS

5. A stakeholder workshop was not held at the program-level. For country-specific stakeholder workshop reports and results, please refer to Section 8.6 in the Technical Annexes.

8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR

6. The Federal Democratic Republic of Ethiopia, the Republic of Mali, the Republic of South Africa, the United Republic of Tanzania, and the Republic of Tunisia submitted completion reports; messages contained therein were integrated into this ICR, and associated Technical Annexes (see Technical Annexes for details). ASP-P1 Morocco did not submit a completion report.

7. Comments on the draft ICR were received from the Federal Democratic Republic of Ethiopia, the Republic of Mali, the United Republic of Tanzania, and the Republic of Tunisia, and incorporated/outlined accordingly. See Section 8.7 of the Technical Annexes for details.

8.8 COMMENTS OF CO-FINANCIERS AND OTHER PARTNERS/STAKEHOLDERS

8. The draft ICR was shared with all co-financiers and other partners/stakeholders. Comments on the program-level ICR were received from CLI, FAO, FFEM, WWF, and the European Commission. Comments and edits were incorporated accordingly and/or are outlined in the following. For comments received from FFEM, see Section 8.8 in the ASP-P1 Tunisia Technical Annex.

9. CLI: All suggested edits received from CLI were incorporated. 10. FAO: Comments received from FAO were incorporated. Additional comments included:

a. On the program’s design having given insufficient attention to specifying and defining the partnership arrangement, and the roles and responsibilities of each partners involved (paragraph 20): There was a TOR for the PCU developed but never approved by the World Bank – all other partners agreed.

b. On the collaboration between FAO and the World Bank (paragraph 25): There was a period when there was duplication of roles and advice to countries under one of the World Bank TTLs. This resulted in a breakdown of trust.

c. On the delayed financial support obtained from CLI (paragraph 25): FAO had to provide this role which was unbudgeted until these people were hired again blurring the lines of responsibility as

31

FAO was not supposed to provide this support as defined by WB in the discussions leading up to project implementation.

d. On the service delivery to ASP-P1 countries (paragraph 44): This statement is unacceptable and must be changed. The TSU was actively prevented from providing country-level support in the project design. We developed the guidance to countries for the CESA with the World Bank TTL and countries were trained in its developed although this was not foreseen in our project description. Training was also provided to countries on development of disposal tenders. The paragraph implies the TSU was supposed to do these activities when in fact we were told by the WB that this would be a conflict of interest and were actively prevented from providing this support. From the 2001 meeting in London we proposed the placement of chief technical advisors in each country as per FAO systems to manage the projects with the country teams. This was rejected. Crop Life offered to provide technical advisors and these took an age to materialize. The Bank sent in Safeguards consultants to provide this support and confused the countries as they were offering advice which was contradictory to FAO. This paragraph is not correct. The technical assistance provided by the TSU was as expected in our project document. If it did not reflect the support envisaged in the WB document then your document was incorrect. o World Bank response: Paragraph 44 was modified to partly reflect the above comment.

However, in general, the description of Component 2 outlined in Annex 4 of the PAD (Report No: 32232-MNA) was taken as basis for this ICR’s evaluation.

e. On no available data regarding FAO resources spent (paragraph 51): Again not accurate. Our reports were sent to GEF for our separate grant.

f. Overall: The main issue was that this was never a true regional project with the associated economies of scale or synergies. It was a multi-country project in a single region (or two regions in the case of the Bank). The issue was compounded by a lack of mutual acceptance of roles and responsibilities and a lack of understanding as to the mode of operation of implementation of WB country activities by FAO which made the operation of the TSU as designed impossible. This was never really understood and no point was any adjustment made to compensate.

11. WWF: Suggested edits received from WWF were incorporated. Additional comments included: a. Component 3 on knowledge management was not implemented to the end as the CCAME

effectively stopped being operational once funding for its activities stopped in December 2010. b. Inadequate resources were provided to WWF and PAN to execute the capacity building activities

for NGOs. 12. Development and Cooperation - European Commission: Comments received from the European

Commission included: a. The project closed in June 2013, addresses the issue of obsolete pesticides by developing and

implementing disposal and prevention activities through the establishment of a new partnership between African countries and their collaborating partners (including the World Bank, the United Nations Food and Agriculture Organization (FAO), the pesticide industry represented by CropLife International (CLI), the Pesticides Action Network (PAN) in the UK and Africa (PAN-UK and PAN-Africa, respectively) and the World Wide Fund for Nature (WWF)).

32

b. The project was co-funded among others from the Multi-Donor Trust Fund (established by the EU together with Denmark, Sweden, and Switzerland) but it is not clear to us which component was supported by the Multi-Donor Trust Fund (supported by EU) through parallel financing.

c. The report refers to a lot of results indicators (13) which are analyzed individually but not in correlation with each other’s.

d. The ICR says: “The elimination of obsolete pesticide stocks remains a development priority. Rural communities cannot hope to advance if the soil and water, upon which their livelihoods and health depend, are contaminated with pesticides, and urban populations cannot prosper if they are suffering from severe illnesses caused by pesticide poisoning”.

e. However, though technical support and strategic studies have been delivered and the NGOs participation was 100%, unfortunately the projects have shown unsatisfactory results concerning mainly the disposal implementation activities which are development priorities. Only 35% of the inventoried publicly held obsolete pesticides and associated waste were disposed in four (out of 6) ASP-P1 countries and the disposal contracts have been implemented only in three countries because country counter parts have not been found (the accomplishment of these targets is considered of 50%).

f. Therefore, we are very much concerned on the unsatisfactory results in implementation of activities on the field which is affecting not only the objective of the project, but the development of the concerned countries.

g. A deeper and correlated analysis of these indicators may reveal other aspects/conclusions that could be useful in future at least for the implementation of other projects, such as the effectiveness of technical support (including the quality of the studies), role and effectiveness of the participating NGOs, etc.

33

8.9 VERBATIM COMMENTS ON DRAFT ICR

36

TECHNICAL ANNEX I: ASP-P1 ETHIOPIA

37






CPD Crop Protection Department

EMP Environmental Management Plan

EMTK Environmental Management Tool Kit

EPA Environmental Protection Authority


FM Financial Management

GDP Gross Domestic Product



ICR Implementation Completion and Results Report

IP Implementation Performance


ISR Implementation Status and Results Report



MoARD Ministry of Agriculture and Rural Development


NIP National Implementation Plan

NSC National Steering Committee




PMT Project Management Team

PMU Project Management Unit


PRRP Pesticide Risk Reduction Program

38

PSMS Pesticide Stock Management System

QAE Quality at Entry

QAG Quality Assurance Group

QSA Quality of Supervision

TAD Technical Advisor for Disposal

TOR Terms of Reference


TTL Task Team Leader

USAID United States Agency for International Development


39

TABLE OF CONTENTS

1. PROJECT CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES AND DESIGN ........ 48 1.1 CONTEXT AT APPRAISAL ............................................................................................................................................. 48 1.2 ORIGINAL PDO AND INDICATORS ............................................................................................................................. 49 1.3 REVISED PDO AND INDICATORS ................................................................................................................................ 50 1.4 MAIN BENEFICIARIES ................................................................................................................................................... 51 1.5 ORIGINAL AND REVISED COMPONENTS .................................................................................................................. 51 1.6 OTHER SIGNIFICANT CHANGES ................................................................................................................................. 53

2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES ....................................................................... 54 2.1 PROJECT PREPARATION, DESIGN, AND QUALITY AT ENTRY ............................................................................. 54 2.2 IMPLEMENTATION ........................................................................................................................................................ 55 2.3 MONITORING AND EVALUATION (M&E) ................................................................................................................. 56 2.4 SAFEGUARD AND FIDUCIARY COMPLIANCE ......................................................................................................... 57 2.5 POST-COMPLETION OPEARTION/NEXT PHASE ....................................................................................................... 58

3. ASSESSMENT OF OUTCOMES ........................................................................................................................................... 59 3.1 RELEVANCE OF OBJECTIVES, DESIGN, AND IMPLEMENTATION ....................................................................... 59 3.2 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVE ................................................................................... 59 3.3 ACHIEVEMENT OF GLOBAL DEVELOPMENT OBJECTIVE .................................................................................... 62 3.4 EFFICIENCY ..................................................................................................................................................................... 62 3.5 JUSTIFICATION OF OVERALL OUTCOME RATING ................................................................................................. 63 3.6 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS ............................................................................. 63 3.7 SUMMARY OF FINDINGS OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS ..................... 64

4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME ............................................................................................ 64 5. ASSESSMENT OF WORLD BANK AND BORROWER PERFORMANCE .................................................................... 64

5.1 WORLD BANK PERFORMANCE ................................................................................................................................... 64 5.2 BORROWER PERFORMANCE ....................................................................................................................................... 65

6. LESSONS LEARNED .............................................................................................................................................................. 66 7. COMMENTS ON ISSUES RAISED BY BORROWER, IMPLEMENTING AGENCY, AND PARTNERS .................. 66 8. ANNEXES ................................................................................................................................................................................. 67

8.1 PROJECT COSTS AND FINANCING.............................................................................................................................. 67 8.2 OUTPUTS BY COMPONENT .......................................................................................................................................... 67 8.3 ECONOMIC AND FINANCIAL ANALYSIS .................................................................................................................. 71 8.4 WORLD BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS.................................. 71 8.5 BENEFICIARY SURVEY RESULTS ............................................................................................................................... 73 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS ............................................................................................ 73 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR .................................................................. 74 8.8 COMMENTS OF CO-FINANCIERS AND OTHER PARTNERS/STAKEHOLDERS ................................................... 75 8.9 VERBATIM COMMENTS ON THE DRAFT ICR ........................................................................................................... 76

40


Country: Ethiopia Project Name: Africa Stockpiles Program - Project 1 Project ID: P105711 L/C/TF Number: TF90543 ICR Date: 11/27/2013 ICR Type: Core ICR Lending Instrument: APL Borrower: Federal Democratic Republic of Ethiopia Original Total Commitment:

US$2.62 million

Disbursed Amount: US$2.23 million (06/30/2013)

Revised Amount: - Environmental Category: A Global Focal Area: CHEM Implementing Agency: Crop Protection Department Co-financiers and Other External Partners: Government of the Federal Democratic Republic of Ethiopia, Food and Agriculture Organization (FAO), CropLife International (CLI), World Wildlife Fund (WWF), and Pesticide Action Network (PAN-UK and PAN-Africa).

B. Key Dates

Process Date Process Original Date Revised / Actual Date(s) Concept Review: 07/17/2002 Effectiveness: 06/30/2007 07/13/2007 Appraisal: 04/19/2004 Restructuring: 06/30/2011 Approval: 06/19/2007 Mid-term Review: 06/30/2009 10/28/2009

Closing: 06/30/2011 12/31/2012; 05/31/2013; 06/28/2013

C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Global Environment Outcome: Moderate World Bank Performance: Moderately Unsatisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of World Bank and Borrower Performance World Bank Ratings Borrower Ratings Quality at Entry: Moderately Unsatisfactory Government: Moderately Satisfactory

Quality of Supervision: Moderately Unsatisfactory Implementing Agency/Agencies:


Overall World Bank Moderately Unsatisfactory Overall Borrower Moderately Satisfactory


41

Performance: Performance:

D. Sector and Theme Codes Original Actual Sector Code (as % of total World Bank financing) Sanitation 33.3 33 Health 33.3 33 General Agriculture 33.3 34 Theme Code (as % of total World Bank financing) Environmental policies and institutions 50 50 Pollution management and environmental health 50 50

E. World Bank Staff

Positions At ICR At Approval Vice President: Makhtar Diop Obiageli Katryn Ezekwesili Country Director: Guang Zhe Chen Mark D. Tomlinson Director (Regional Integration): Colin Bruce -

Sector Manager: Magda Lovei Mary Barton-Dock Project Team Leader: Dinesh Aryal Peter Kristensen/Denis Jordy ICR Team Leader: Dinesh Aryal ICR Primary Author: Veruschka Schmidt F. Results Framework Analysis - At project approval, no country-specific result framework existed; only seven Project Development Objective (PDO) Indicators were defined. Following a Level 1 Restructuring in June 2011, a country-specific results framework was introduced. The original seven PDO Indicators were revised and reduced to two PDO Indicators, and in addition, seven Intermediate Outcome Indicators were introduced.



-

Potential Problem Project at any time (Yes/No): Yes

Quality at Entry (QEA): -

Problem Project at any time (Yes/No): Yes

Quality of Supervision (QSA): -

PDO rating before Closing/Inactive status:


42

Project Development Objective (PDO) – as presented in the Global Environment Facility (GEF) Trust Fund Grant Agreement:65 To assist the recipient in: a) eliminating inventoried publicly held obsolete pesticide stocks and associated waste; and b) implementing measures to reduce and prevent future related risks. Revised PDO – as presented in the Level 1 Restructuring of June 2011 (Report No: 62794-ET): To assist the Federal Republic of Ethiopia to eliminate and/or safeguard inventoried publicly held obsolete pesticide stocks and associated waste, and develop a strategy for sustainable management of future accumulations. Global Environment Objectives (GEO) - as presented in the Project Appraisal Document (PAD (Report No: 39901-ET): Reduction of the effects of Persistent Organic Pollutants (POPs) on the global environment. Revised GEO – Not Applicable (a) PDO Indicators – Original PDO Indicators as presented in the GEF Trust Fund Grant Agreement are assessed against achievements realized at the time of the project’s Level 1 restructuring in June 2011; revised PDO Indicators are assessed against achievements realized at project closing in June 2013.




Formally Revised Target

Values

Actual Value Achieved at Completion or Target

Years

Original PDO Indicators

Indicator 1: Inventory database of publicly held obsolete pesticides stocks and associated waste in place and in use by the PMT and the recipient's staff66

Value (quantitative or qualitative)

No Yes - Yes

Date Achieved 04/19/2004 06/30/2011 - 05/31/2010


Target fully achieved (100%): An inventory was completed and validated in May 2010, including transfer of data into the FAO obsolete pesticide inventory database. The database was subsequently used for risk profiling of the stocks and preparation of the Country Environmental and Social Assessment (CESA).

Indicator 2: Completion of Country Environmental and Social Assessment (CESA), and implementation of the measures contained therein


No Yes - No

Date Achieved 04/19/2004 06/30/2011 - 06/30/2011

65 The PDO, indicators, and components that are stated in the Project Appraisal Document (PAD) slightly differ from those stated in the GEF Trust Fund Grant Agreement. This ICR bases its evaluation on the PDO, indicators, and components presented in the GEF Trust Fund Grant Agreement, which served as the principal guiding document for the project’s implementation activities. 66 PMT refers to Project Management Team. Hereafter referred to as Project Management Unit (PMU)

43


Target not achieved (0%): A CESA was not completed by the time of restructuring.

Indicator 3: Inventoried publicly held obsolete pesticides stocks disposed of, as a result of the completion of the disposal services contract(s) in accordance with national and international laws and regulations


0% 100% - 0%

Date Achieved 04/19/2004 06/30/2011 - 06/30/2011


Target not achieved (0%): The bidding document for the disposal tender was completed and cleared by the World Bank, but none of the publicly held obsolete pesticides was disposed of at restructuring, due to project implementation delays.

Indicator 4: Legal and regulatory framework for pesticide management improved, including measures to strengthen compliance with the Basel Convention and the Rotterdam Convention


No Yes - Partly (50%)

Date Achieved 04/19/2004 06/30/2011 - 06/30/2010


Target partly achieved (50%): A Pesticide Registration and Control Proclamation was approved in January 2010 by the Council of Ministers, and proclaimed by Parliament in June 2010. Measures to strengthen compliance with the Basel Convention and the Rotterdam Convention were not taken at restructuring.

Indicator 5: Pesticide management strategy is developed Value (quantitative or qualitative) No Yes - No

Date Achieved 04/19/2004 06/30/2011 - 06/30/2011 Comments (incl. % achievement)

Target not achieved (0%): A pesticide management strategy was not developed by the time of restructuring due to project implementation delays.

Indicator 6: The training program under Part D67 of the Project is implemented and the knowledge so acquired is being used by the PMT and recipient's staff


No Yes - Yes

Date Achieved 04/19/2004 06/30/2011 - 06/30/2011


Target fully achieved (100%): Project Management Unit (PMU) members and representatives from the ministry received training in the areas of pesticide management and utilization, pesticide store management, safeguarding, Financial Management (FM), and procurement.

Indicator 7: The Project Management Unit (PMU) is functional and its operation is satisfactory to the World Bank



67 See Section 1.5 for the project’s Component D (in the GEF Trust Fund Grant Agreement referred to as ‘Part D’)

44

Date Achieved 04/19/2004 06/30/2011 - 06/30/2011


Target partly achieved (50%): The PMU was functional, however, it was periodically missing key staff, and project implementation experienced significant delays particularly during the first half of the project.

Revised PDO Indicators as per the Level 1 Restructuring in June 2011

Indicator 1: Inventoried publicly held obsolete pesticides and associated waste disposed of and/or safeguarded by the end of the project


0 100 - Partly (35%)

Date Achieved 06/30/2011 06/30/2013 - 06/30/2013


Target partly achieved (35%): 450 tonnes (35%) of the inventoried 1300 tonnes of publicly held obsolete pesticides and associated waste were disposed of or safeguarded. These include 450 tonnes of obsolete pesticides (against 450 tonnes inventoried), and 0 tonnes of associated waste (against 850 tonnes inventoried).

Indicator 2: A strategy for sustainable management of future accumulations adopted by the Steering Committee by the end of the project


None Yes - Yes


Target fully achieved (100%): A national pesticide management strategy for Ethiopia was developed and endorsed by the Steering Committee on June 25, 2013.

(b) GEO Indicator(s) –The GEO was measured by the PDO and PDO indicators.

(c) Intermediate Outcome Indicator(s)



approval documents)

Formally Revised Target Values



Indicator 1: Percent of validated inventory data entered into PSMS Value (quantitative or qualitative)

100% - - 100%

Date Achieved 06/30/2011 - - 05/31/2010 Comments (incl. % achievement)

Indicator corresponds to original PDO Indicator #1 and was completed prior to restructuring. Therefore, the indicator is not taken into account for post-restructuring.

Indicator 2: Approved CESA available for use by the disposal company Value (quantitative or qualitative) No Yes - Yes

Date Achieved 06/30/2011 06/30/2013 - 12/26/2011 Comments (incl. % Target fully achieved (100%): Despite delays, a CESA was completed in February

45

achievement) 2011, found satisfactory by the World Bank on December 14, 2011, and publicly disclosed on February 29, 2012. Information contained therein was used by the disposal company.

Indicator 3: A final draft container management strategy submitted to the Steering Committee for approval


No Yes - Yes


Target fully achieved (100%): A container management strategy was endorsed by the Steering Committee and the Ministry of Agriculture on June 25, 2013.

Indicator 4: Two planned communication tools implemented Value (quantitative or qualitative)

0 2 - 2

Date Achieved 06/30/2011 06/30/2013 - 06/30/2013


Target fully achieved (100%): Posters and other printed materials were prepared to raise awareness on the harmful nature of pesticides to human health and the environment; and a one-year TV and radio program was conducted in three local languages to raise awareness on pesticides use.

Indicator 5: A final draft sustainable management plan for obsolete pesticides and associated wastes prepared and submitted to the Steering Committee for endorsement


No Yes - Yes

Date Achieved 06/30/2011 06/30/2013 - 05/31/2013


Target fully achieved (100%): A draft national pesticide management strategy for Ethiopia was developed and submitted to the Steering Committee at the end of May 2013. It was subsequently endorsed on June 25, 2013.

Indicator 6: A Pesticide Registration and Control Proclamation drafted for submission to Parliament


Yes - - Yes

Date Achieved 06/30/2011 - - 06/30/2010 Comments (incl. % achievement)

Indicator corresponds to original PDO Indicator #4 and was completed prior to restructuring. Therefore, the indicator is not taken into account for post-restructuring.

Indicator 7: Availability of one new storage warehouse for pesticide management Value (quantitative or qualitative)

0 1 - 0


Target not achieved (0%): The construction of a new storage warehouse for pesticide management was aborted as a result of time constraints.

46



1 05/30/2008 Moderately Unsatisfactory Moderately Unsatisfactory 0.25 2 12/01/2008 Unsatisfactory Highly Unsatisfactory 0.25 3 12/11/2008 Unsatisfactory Highly Unsatisfactory 0.25 4 05/29/2009 Unsatisfactory Unsatisfactory 0.28 5 12/23/2009 Unsatisfactory Unsatisfactory 0.29 6 06/22/2010 Unsatisfactory Moderately Satisfactory 0.38 7 03/27/2011 Unsatisfactory Moderately Satisfactory 0.51 8 07/13/2011 Moderately Satisfactory Moderately Satisfactory 0.63 9 01/08/2012 Moderately Satisfactory Moderately Satisfactory 0.67 10 10/08/2012 Moderately Satisfactory Moderately Satisfactory 0.74 11 05/29/2013 Moderately Satisfactory Moderately Satisfactory 1.25 06/30/2013 Final disbursement 2.23 H. Restructuring

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at Restructuring

in USD millions

Reason for Restructuring & Key Changes Made

PDO IP

08/01/2009 No Not Available Not Available

Not Available

• Reallocation of proceeds to make available funds for the construction of an improved pesticides storage facility

06/30/2011 Yes Unsatisfactory Moderately Satisfactory

0.5 • Revision of the PDO and indicators to reflect a shift from complete disposal to disposal and safeguarding of stocks

• Introduction of a results framework for better M&E

• Extension of closing date by 18 months to compensate for initial delays (06/30/2011 to 12/31/2012)

• Reallocation of proceeds to accommodate disposal and/or safeguarding of all the additional stocks

12/03/2012 No Moderately Satisfactory


Not available • Extension of closing date by five months to ensure disposal activity

G. Ratings of Project Performance in ISRs

47


Board Approved

PDO Change



in USD millions


PDO IP

completion (12/31/2012 to 05/31/2013) 05/24/2013 No Moderately

Satisfactory Moderately Satisfactory

1.3 • Extension of closing date by one month to ensure reception of disposal certificates (05/31/2013 to 06/30/2013)

If PDO and/or Key Outcome Targets were formally revised (approved by the original approving body), enter ratings below:

Outcome Rating Outcome Ratings

Against Original PDO targets Moderately Unsatisfactory Against formally revised PDO targets Moderately Satisfactory Overall (weighted) rating Moderately Satisfactory I. Disbursement Profile – A disbursement profile is not available.

48

1. PROJECT CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES AND DESIGN


1. Country background: Ethiopia’s economy is primarily based on agriculture. At project approval in June 2007, the sector contributed around 46% to Ethiopia’s GDP and employed over 80% of the labor force.68 Ethiopia’s highlands had been highly fertile, enabling the country to support a large population, but long-term output had been threatened by inefficient land tenure, soil degradation, and recurrent droughts. Pests and crop diseases further contributed to crop yield losses; at project appraisal, the average crop loss due to pests was estimated to be between 30% and 40%, annually.69

2. Sector background: Although pesticide use in Ethiopia had been historically poor due to low economic performance and the presence of smallholder agricultural systems, efforts to increase and diversify food production since the late 1970s had resulted in an increased use of pesticides in the agricultural sector. Of the total pesticides imported, commercial farmers were estimated to have used about 80%, mainly on cotton; the remaining 20% were used for small-scale farming, household and industrial purposes, and disease vector control. Malaria had been prevented and controlled for several decades with the application of pesticides such as Dichlorodiphenyltrichloroethane (DDT). In addition, Ethiopia was one of the eastern African countries that suffered from migratory pests such as desert locust and armyworm outbreaks; the ensuing mobilization of international assistance to combat the pests often resulted in excessive emergency prevention stocks.

3. The accumulation of obsolete pesticides was aggravated by inappropriate storage conditions caused by poor storage facilities, poor management capacities of the stocks, unavailability of required application equipment, and a lack of trained staff on pesticide stock management. Product bans, unsuitable products or packaging, donation or purchase in excess of need, further contributed to this accumulation. Various obsolete, banned, and unwanted pesticides were accumulated throughout Ethiopia carrying significant environmental and public health risks.

4. Institutional background: At appraisal, the Environmental Protection Authority (EPA) was the designated national authority for the Stockholm, Rotterdam, Basel, and Bamako Conventions, which had all been ratified by the Government of Ethiopia. However, by the nature of the mandate,70 EPA’s role and responsibilities had focused on policymaking, establishment of system-based standards and development of partnership with sector ministries. Consequently, the Ministry of Agriculture and Rural Development (MoARD), through its Crop Protection Department (CPD), had overall operational responsibility for cleanup and remediation operations.

5. Actions taken: Towards the end of the 1990s, a series of stockpile safeguarding and disposal operations was initiated, including an operation that was launched in 1999, and implemented with technical support of the Food and Agriculture Organization (FAO) and financial support of the United States Agency for International Development (USAID), the Netherlands and Sweden.71 1,500 tonnes of obsolete pesticides were successfully disposed of under the project’s first phase (FAO-I); the

68 The Economist Intelligence Unit, Country Profile Ethiopia, 2007 69 Country Environmental and Social Assessment (CESA) and Management Plan of the African Stockpiles Program Project-1 Ethiopia, MTS Consulting Engineers PLC, February 2011. 70 Outlined in the Proclamation No. 295 71 FAO project titled Prevention and Disposal of Obsolete Pesticide Stocks in Ethiopia

49

second phase (FAO-II) started in 2004, and allowed disposal of an additional 1,100 tonnes with funds provided by Belgium, Japan, Finland and CropLife International (CLI). Project activities further included development and enforcement of pesticide policies, implementation of Integrated Pest Management (IPM) and Integrated Vector Management (IVM), and capacity building. ASP-P1-Ethiopia intended to complement these operations by financing the disposal of remaining 400 tonnes of obsolete pesticides that had not been disposed of under FAO-II as originally planned, but stored in two locations in Addis Ababa.

6. In preparation of ASP-P1, the Government of Ethiopia had initiated preparatory activities under its National Implementation Plan (NIP).72 These activities included the completion of a preliminary inventory of obsolete pesticides across the country, which had identified over 900 sites with obsolete chemicals and contaminated soils. Very large quantities of empty containers and contaminated spray equipment were also found.

7. Rationale for World Bank assistance: Please refer to Section 1.1 in the program-level ICR. 8. Higher-level objectives: As noted in the Project Appraisal Document (PAD), the project intended to

contribute to Ethiopia’s national development strategy73 in the areas of agriculture and rural development, public health, poverty alleviation, and environmental protection. In particular, the project aimed to: (i) improve the quality of life in poor communities, by reducing environmental health risks; (ii) improve environmental protection; and (iii) enhance the capacity of the agricultural sector to better manage crop pests. Contributing to international efforts to eliminate Persistent Organic Pollutants (POPs), improving management of toxic chemicals, and promote alternatives to pesticide usage that include IPM and IVM represented further objectives.

1.2 ORIGINAL PDO AND INDICATORS

9. The Project Development Objective (PDO), indicators, and components that are described in the ASP-P1 Ethiopia PAD74 slightly differ from those described in the associated GEF Trust Fund Grant Agreement.75 This ICR bases its evaluation on the PDO, indicators, and components presented in the GEF Trust Fund Grant Agreement, which served as the principal guiding document for the project’s implementation activities; in addition and as outlined in Section 1.3 and 1.5, changes under a Level 1 restructuring were introduced to the GEF Trust Fund Grant Agreement.

10. ASP-P1 Ethiopia’s original PDO as presented in the GEF Trust Fund Grant Agreement:76 To assist the recipient in: a) eliminating inventoried publicly held obsolete pesticide stocks and associated waste; and b) implementing measures to reduce and prevent future related risks.

11. At appraisal, only an overall ASP-P1 results framework had been prepared (see program-level ICR); results frameworks for the individual ASP-P1 countries, including Ethiopia, were intended to be

72 Federal Democratic Republic of Ethiopia - National Implementation Plan for the Stockholm Convention, September 2006. The Plan had been prepared with the support of the United Nations Environment Program (UNEP) and the United Nations Industrial Development Organization (UNIDO). 73 Ethiopia: Sustainable Development and Poverty Reduction Program, Ministry of Finance and Economic Development (MOFED), July 2002 74 Ethiopia – Africa Stockpiles Program, (Report No: 39901-ET), June 5, 2007 75 GEF Trust Fund Grant Agreement between the Federal Democratic Republic of Ethiopia and the International Bank for Reconstruction and Development, (Number TF090543 – ET), July 13, 2007 76 ASP-P1 Ethiopia’s PDO as presented in the PAD: Eliminate inventoried publicly held obsolete pesticide stockpiles and associated waste, and implement measures to reduce and prevent future related risks.

50

Table 1.1: ASP-P1 Ethiopia Original PDO Indicators

finalized during the individual ASP-P1 country project launches. However, country-specific PDO indicators were developed at appraisal. For ASP-P1 Ethiopia, these were as follows:

Original PDO Indicators as presented in the PAD

Original PDO Indicators as presented in the GEF Trust Fund Grant Agreement

a. Inventory database of publicly held obsolete pesticide stocks and associated wastes in place and being used by the Project Management Team (PMT) and the stakeholders.

b. Completion of a Country Environmental and Social Assessment (CESA) and implementation of the measures contained therein.

c. Inventoried publicly held obsolete pesticides stocks disposed of, as a result of the completion of the disposal services contract(s) in accordance with national and international laws and regulations.

d. Legal and regulatory framework for pesticide management upgraded, including measures to strengthen compliance with the Conventions.

e. Pesticide management strategy developed. f. The training program under Component C of the

project is implemented and the knowledge so acquired is being used by the PMT and stakeholders.

g. The PMT is functional and its operation is satisfactory to the World Bank.

a. Inventory database of publicly held obsolete pesticides stocks and associated waste in place and in use by the PMT and the recipient's staff.

b. Completion of CESA, and implementation of the measures contained therein.


d. Legal and regulatory framework for pesticide management improved, including measures to strengthen compliance with the Basel Convention and the Rotterdam Convention.

e. Pesticide management strategy is developed. f. The training program under Part D of the project is

implemented and the knowledge so acquired is being used by the PMT and recipient's staff.77

g. The PMT is functional and its operation is satisfactory to the World Bank.

1.3 REVISED PDO AND INDICATORS

12. The PDO and indicators for ASP-P1 Ethiopia were revised through a Level I restructuring on June 30, 2011.78 The original PDO implied that all inventoried obsolete pesticides would be eliminated. With the discovery of higher than estimated stocks, including associated waste, it was decided to adopt a blended approach of disposal overseas and lower cost safeguarding in-country based on risk profiling of the inventoried pesticides and associated waste, and availability of satisfactory storage facilities (i.e. obsolete pesticides and higher risk stocks of heavily contaminated soils being disposed of overseas while lower risk stocks of associated waste being safeguarded in-country). In addition, the original PDO assumed no new accumulations of obsolete pesticides during the life of the project, which had been proven difficult to achieve in the context of agricultural intensification and food security concerns. Therefore, it was agreed that the project would instead focus on developing a strategy for management of future accumulations.

77 See Section 1.5 for the project’s Component D, here referred to as “Part D”. 78 The Africa Stockpiles Program – Project for Ethiopia, South Africa, Tanzania and Tunisia: Proposal to Restructure, (GEF/R2011-0022), June 22, 2011

51

13. ASP-P1 Ethiopia’s revised PDO: 79 To assist the Federal Republic of Ethiopia to eliminate and/or safeguard inventoried publicly held obsolete pesticide stocks and associated waste, and develop a strategy for sustainable management of future accumulations.

14. ASP-P1 Ethiopia PDO indicators were reduced in number and revised to focus on the revised project objective. In addition, a country-specific result framework was introduced for effective monitoring and evaluation of project progress (see Data Sheet).

15. ASP-P1 Ethiopia’s revised PDO Indicators: (i) Inventoried publicly held obsolete pesticides and associated waste disposed of and/or safeguarded by the end of the project; and (ii) a strategy for sustainable management of future accumulations adopted by the Steering Committee by the end of the project.


16. Please refer to Section 1.4 in the program-level ICR.

1.5 ORIGINAL AND REVISED COMPONENTS

17. The overall PDO of ASP-P1 Ethiopia was to be achieved through the implementation of four distinct components (outlined in Table 1.2). The table also shows revisions that were made through a Level I restructuring on June 30, 2011.80


Original Components as presented in the GEF Trust

Fund Grant Agreement

Revised Components (Level I Restructuring June 30,

2011)81 Component 1.A - Cleanup and disposal (Total: US$2.03 million; GEF US$1.03 million): a. To verify and validate the existing

obsolete pesticide data base, and conduct inventory on contaminated soils, buried pesticides and contaminated equipment, inventory and risk assessment including free standing pesticides, contaminated sites, soils, buried pesticides and contaminated containers and equipment;

b. To implement the World Bank-approved CESA for free standing obsolete pesticides and to produce a second CESA for the treatment of heavily

Part A - Obsolete pesticides inventory: a. Collection of data and

compilation of a data base; b. A risk-based prioritization of

the stocks identified in the data base; and

c. Preparation of the CESA.

Part A - Obsolete pesticides inventory: No changes were introduced at restructuring.

Part B – Disposal of obsolete pesticides: a. Carrying out of a technological

assessment of available treatment and disposal options;

b. Implementation of the treatment and/or disposal technology

Part B – Disposal and/or safeguarding of obsolete pesticides and associated waste: a. Carrying out a technological

assessment of available safeguarding and disposal options;

b. Implementation of the safeguarding

79 In the GEF Trust Fund Grant Agreement, the term ‘Performance Monitoring Indicators’ was used; the restructuring paper (The Africa Stockpiles Program – Project for Ethiopia, South Africa, Tanzania and Tunisia: Proposal for Restructure, (GEF/R2011-0022), June 22, 2011) used the term ‘project development objective level results indicators’. 80 The Africa Stockpiles Program – Project for Ethiopia, South Africa, Tanzania and Tunisia: Proposal to Restructure, (GEF/R2011-0022), June 22, 2011 81 Changes (outlined in more detail in the project’s restructuring paper, The Africa Stockpiles Program – Project for Ethiopia, South Africa, Tanzania and Tunisia: Proposal for Restructure, (GEF/R2011-0022), June 22, 2011) were introduced to the GEF Trust Fund Grant Agreement.

Table 1.2: ASP-P1 Ethiopia Original and Revised Components

52

contaminated soils, buried pesticides and contaminated equipment;

c. To safeguard obsolete pesticide stocks; and

d. To dispose of the identified publicly held obsolete pesticides.

selected; c. Improving the operations of

obsolete pesticide stocks collection centers, including store stabilization and safety measures at those centers; and

d. Container management.

and/or disposal technology selected; c. Improving the operations of obsolete

pesticide stocks collection centers, including store stabilization and safety measures at those centers; and

d. Container management. The changes reflected the addition of safeguarding as an outcome and clearly defined associated waste as an integral part of the stocks to be treated.

Component 1.B: Prevention of obsolete pesticide accumulation (Total: US$0.19 million; GEF: US$0.19 million): a. Strengthen the existing regulatory

system for pesticide control and the homologation system of the pesticides;

b. Promote alternatives to chemical pesticides through improvement of pest management strategies, with a particular attention to IPM for agriculture and IVM for health care;

c. Communication and awareness raising, and;

d. Strengthen the capacity of the warehouses in carrying out pesticide management.

Part C – Prevention of obsolete pesticide accumulation: a. Awareness raising activities on

the program, health, and environmental hazards of pesticides, and IPM;

b. A review and update of the legal and regulatory framework to control obsolete pesticide accumulation and strengthen enforcement mechanisms;

c. Preparation of a pesticide management strategy; and

d. Strengthening the pesticide management capacity of storage warehouses.

Part C - Reduction of obsolete pesticide and associated waste re-accumulation: a. Awareness raising activities on the

program, health and environmental hazards of pesticides;


c. Preparation of a sustainable pesticide management strategy;

d. Strengthening the pesticide management capacity of storage warehouses;

e. Provision of training to relevant staff of the recipient on health and safety issues, and environmental monitoring; and

f. Strengthening awareness on the pesticide management legal framework and homologation systems.

The changes reflected the integration of the capacity-building component in the revised prevention component as both aimed at improved pesticide management practices, and removal of the IPM activities where the potential impact was likely to be marginal.

Component 1.C: Capacity building (Total: US$0.30 million; GEF: US$0.20 million): a. Laboratory capacity including chemical

analysis; b. Environmental assessment and

monitoring and evaluation training for the EPA and other stakeholders; and

c. Health and safety training.

Part D – Capacity building: a. Provision of training to relevant

staff of the recipient on laboratory analysis, health and safety issues, and environmental monitoring;

b. Strengthening awareness on the pesticide management legal framework and homologation systems; and

Part D – Capacity building: No changes were introduced at restructuring.

53

c. Development and implementation of a pesticide storage and integrated pest management training program.

Component 1.D: Project management and M&E (Total: US$0.42 million; GEF: US$0.30 million): Activities included the establishment of a PMU in Ethiopia whose responsibility was to implement the country level activities, solicit external technical support, monitor and evaluate project activities and cooperate with the GEF-funded NIPs for the Stockholm Convention and similar activities. Country level management also included establishment of a National Steering Committee (NSC) representing key stakeholders and overseeing the work of the PMU.

Part E – Project management and monitoring: Supporting the PMT in carrying out project coordination, implementation, monitoring and evaluation, through the provision of technical advisory services, goods, and operational support.

Part E – Project management and monitoring: No changes were introduced at restructuring.

1.6 OTHER SIGNIFICANT CHANGES

Change Date Justification Approval

Extension of closing date Extension of closing date by 18 months from 06/30/2011 to 12/31/2012

06/30/2011 To compensate for initial delays, the disposal and safeguarding activities were expected to commence in 11/2011 and conclude in 12/2012.

Board

Extension of closing date by 5 months from 12/31/2012 to 05/31/2013

12/03/2012 To ensure the completion of the disposal and safeguarding contract, including reception of certificates of disposal from disposal facilities.

Regional Vice President

Extension of closing date by 1 month from 05/31/2013 to 06/30/2013

05/24/2013 To ensure reception of disposal certificates Country Director

Reallocation of Proceeds Reallocation of proceeds from the categories Goods (-US$10,000), Training and Workshops (-US$20,000), Operating Costs (-US$80,000) and Unallocated (-US$155,000) to the category Works (+US$265,000).

08/01/2009 To make funds available (US $265,000) for the construction of an improved pesticides storage facility as part of the prevention component. An improved facility was expected to reduce the amount of future obsolete pesticides through proper storage.

Country Director

Reallocation of proceeds from the categories Goods (-US$414,000), Consulting Services (-US $77,000), Operating Costs (-US $47,000) and Unallocated (-US $80,000) to the categories Disposal Services (+US $600,000) and Training and Workshops

06/30/2011 To accommodate the disposal and/or safeguarding of all the additional stocks and associated waste inventoried and provide additional related training to meet the first PDO indicator.

Board

Table 1.3: ASP-P1 Ethiopia Other Significant Changes

54

(+US $18,000)

Reallocation of US $310,000 from the Works category to the Disposal Services category.

12/03/2012 To further increase the budget for disposal activities due to larger than budgeted stocks, which is critical to meet the first PDO-level indicator. The request was made to reallocate the amount which had originally been allocated for the construction of a pesticide storage warehouse (see above), which was had not been constructed.


Redefinition of activities Redefinition of the term “Disposal Services” from services related to the treatment or disposal of obsolete pesticides and associated waste, to services related to the safeguarding and/or disposal of the same.

06/30/2011 Addition of in-country safeguarding as a possible cheaper end result for lower risk stocks; this was due to larger than budgeted stocks of obsolete pesticides and associated waste.

Board



18. ASP-P1 Ethiopia was prepared between 2001 and 2007, as part of preparatory activities at the program-level. It took more than five years from GEF eligibility in July 2001, and more than four years from concept note in July 2002 to project approval by the Board in June 2007.82

19. Soundness of background analysis: Project preparation was conducted in close collaboration between the government and the World Bank, FAO, and CLI, through joint missions, and knowledge exchange. Considerable background analysis was undertaken, including a procurement capacity assessment in 2004, which appropriately recommended a procurement capacity-enhancement plan and a comprehensive procurement–training program. A Financial Management (FM) capacity assessment was also undertaken at appraisal concluding that an FM system would need to be in place prior to effectiveness. Overall, the government had acquired substantial experience through implementation of the above-mentioned project;83 and was considered well prepared for pesticide disposal by the World Bank at appraisal in 2004, and by FAO in 2005.84

20. Key preparatory inputs such as the PAD and to a lesser extent the Operational Manual, however only broadly laid out the project’s plans; detailed country-specific information regarding Monitoring and Evaluation (M&E), risks and risk mitigation measures particularly regarding procurement, and FM, were missing. As noted above, the government had acquired substantial experience through implementation of a previous project, 85 but no lessons learned seem to have been integrated and used during preparation.

82 The long preparation time resulted from a lengthy partnership building/negotiation process at the program-level resulting from the complex implementation arrangement and project design (see Section 2.1 in the program-level ICR). 83 FAO project titled Prevention and Disposal of Obsolete Pesticide Stocks in Ethiopia 84 See: Federal Democratic Republic of Ethiopia – Africa Stockpiles Program – Appraisal Mission: Aide-Memoire (March 28 – April 2, 2004; and African Development Bank – The Federal Democratic Republic of Ethiopia – Africa Stockpiles Program – Project 1: Appraisal Mission (July 20-26, 2005) Aide-Memoire 85 FAO project titled Prevention and Disposal of Obsolete Pesticide Stocks in Ethiopia

55

21. Assessment of project design: The project’s design (as for all ASP-P1 participating countries) was preconfigured at the program-level, and did not reflect any country-specific modifications or lessons learned from previous projects executed in Ethiopia. Design weaknesses at the program-level that translated into implementation delays at the country level, including ASP-P1 Ethiopia, are outlined in Section 2.1 of the program-level ICR.

22. Government commitment: The Government of Ethiopia had shown significant commitment towards the objectives of the ASP, particularly through the implementation of a previous similar project. In addition, the government had already established the institutional structures for coordinating the implementation of ASP-P1. MoARD, CPD, had already established a PMU in 2003.

23. Assessment of risks: Program-level risks were adequately identified and outlined in the PAD. They were presented in the form of a matrix and represented a summary of risk assessments undertaken during identification, pre-appraisal and appraisal missions to ASP-P1 participating countries. Even though the assessment of risks was comprehensive and included a broad range of mitigation measures in generic terms, an assessment of risks and mitigation measures specific to Ethiopia was lacking. At the same time, it should be noted that it was difficult to assess country-specific risks in the absence of inventories of obsolete pesticides and CESAs, which were only planned to be undertaken during project implementation.

2.2 IMPLEMENTATION

24. Throughout implementation, the project experienced long delays, eventually resulting in three extensions for a total of 24 months, and a Level 1 restructuring in 2011.

25. Key factors affecting implementation and outcomes included: • Poor project management and procurement capacity affected project implementation. The PMU

staff that had managed the preceding pesticide project (FAO-II) was considered experienced and had therefore remained in place to take over the management of ASP-P1. However, FAO-II only closed in July 2008 and the transition period in which both projects were active was poorly managed; the PMU primarily focused on completing outstanding activities of FAO-II, and ASP-P1 start-up activities were delayed. Upon completion of FAO-II, key staff, including the project coordinator, unexpectedly left, resulting in significant additional delays as new staff had to be appointed and trained. The lack of adequate procurement staff coupled with cumbersome and lengthy procurement processes at the ministry-level, in particular, adversely affected the project’s progress throughout project implementation (see Section 2.4 for more details).

• Delayed and insufficient technical support affected implementation performance. The PMU was supposed to receive highly technical and specialized expertise (required for implementation, supervision and monitoring of project activities) from FAO’s Technical Support Unit (TSU); however, this support was not delivered to the extent originally planned.86 This was mainly due to poor collaboration between the World Bank and FAO, resulting from a complex and loosely defined stakeholder implementation arrangement at the program-level, and FAO having exhausted its GEF funds before the disposal activities began (see Section 2.2 in the program-level ICR). Urgently needed operational support from the World Bank was not sufficiently provided

86 In addition, equipment and vehicles intended to be provided by the TSU in the beginning of project implementation, were delayed by 5 months.

56

prior to mid-term review; however, increased supervision, operational and technical support thereafter, improved the level of support. For technical support during disposal activities, a Technical Advisor for Disposal (TAD) was eventually recruited and financed by CLI, which positively affected project implementation.

• Higher than estimated obsolete pesticide stocks caused a funding gap. The completion of the national obsolete pesticides inventory in May 2010 revealed additional 200 tonnes of obsolete pesticides and 850 tonnes of associated waste, which had not been budgeted. At restructuring, the project was therefore facing an estimated funding gap of US$4.1 million for the disposal of all inventoried stocks and associated waste. The World Bank and the government had agreed to reallocate GEF proceeds toward disposal, dispose of the highest risk stocks and safeguard stocks with a lower risk profile. A round table with possible donors to raise additional funds was organized in early 2012; however, at project closure no additional funds had been raised. Insufficient project funds eventually resulted in non-disposal and/or safeguarding of the inventoried associated waste including contaminated soils.

26. Restructuring: At mid-term review, achievement of the PDO was considered unlikely within the original timeframe and budget. The World Bank recommended revising the PDO and associated indicators, introducing a results framework, and extending the project’s closing date. However, it was agreed that the project would only be restructured once project implementation showed positive progress; the PMU was subsequently provided with increased technical and operational support by the World Bank.

27. In June 2011, following implementation progress, the project was restructured.87 Revisions reflected a shift from complete elimination of all inventoried publicly held obsolete stocks and associated waste to elimination overseas of the highest risk stocks and lower-cost in country safeguarding of the stocks. However, the revised PDO remained ambitious as project funds continued to be insufficient, even in view of revised targets.

2.3 MONITORING AND EVALUATION (M&E)

28. Overall M&E – Rating Moderately Unsatisfactory. 29. M&E design – Rating Unsatisfactory: Despite the lengthy preparation time, at appraisal, an M&E

system specific for Ethiopia was not developed. A results framework, which was intended to be finalized by the TSU during project launch and included in the project’s Operational Manual, did not materialize. Only a set of PDO indicators linked to the different project components was developed in the PAD and the GEF Trust Fund Grant Agreement, however, these did not allow for adequate monitoring of project implementation progress. The indicators focused on measuring outputs as opposed to progress towards project outcomes. In particular, results expected under the prevention component were not adequately considered by these indicators.

30. M&E implementation – Rating Moderately Unsatisfactory: Until mid-term review, there was no M&E system in place. Eventually, in 2009, the TSU provided the PMU with and trained it in using a Disposal-Project Tracking Workbook to monitor and evaluate project-related activities. However, the tool was complex, and therefore not used by the PMU. Increased support provided by the World Bank

87 Following the restructuring, two additional extensions were processed for a total of six months in order to compensate for longer than anticipated disposal activities, and in particular to ensure reception of incineration certificates before project closure.

57

in 2010 included assisting the PMU with an 18-months component-based work plan and a simplified implementation progress-monitoring plan. In 2011, following the restructuring of the project, a results framework specific to Ethiopia was finally introduced and the PDO and associated indicators were revised to focus on the two main project achievements (disposal and/or safeguarding and reduction of future accumulations) which strengthened project M&E, albeit late in the process.

31. M&E utilization – Rating Moderately Satisfactory: The inventory of obsolete pesticide stocks was conducted by using inventory forms and a manual provided by the FAO Environmental Management Toolkit (EMTK).88 The inventory data was entered into a database of the FAO web-based Pesticide Stock Management System (PSMS) to record and monitor the pesticides and manage their usage, and subsequently validated. The data was used by the PMU for risk profiling of the stocks,89 and to prepare the CESA and disposal bidding documents/contracts. FAO also provided training to the PMU in how to best use the database, which took place in 2010. The PSMS together with the EMTK proved to be very useful tools in managing obsolete pesticides stocks.

2.4 SAFEGUARD AND FIDUCIARY COMPLIANCE

32. Safeguard compliance – Rating Satisfactory: ASP-P1 triggered two safeguard policies: OP 4.01 on Environmental Assessment and OP 4.09 on Pest Management.

33. Compliance with OP 4.01 on Environmental Assessment: A CESA identifying risks and devising mitigation measures to address potential negative environmental and social impacts was completed to the World Bank’s satisfaction in December 2012, albeit with significant delays. The assessment concluded that MoARD had the capacity to safeguard medium and low risk pesticide stores, and included an Environmental Management Plan (EMP) together with a Health and Safety Plan for safeguarding obsolete pesticides before disposal. The CESA’s late completion resulted from: i) the late completion of the inventory of obsolete pesticide stocks (only finalized in May 2010) which was needed for the CESA; ii) a complicated and time-consuming consultant procurement process; and iii) a lengthy World Bank review and clearance process (which ultimately caused eight months of delay). Subsequently, project activities involving storage, transportation, disposal, and site remediation were implemented in compliance with the recommendations and requirements of the CESA, the associated EMP, and the Health and Safety Plan.

34. OP 4.09 on Pest Management: Since project activities themselves met requirements of a pest management plan (i.e., through pesticide management, IPM, and capacity building), individual country projects, including ASP-P1 Ethiopia, were not required to prepare self-standing documents. See Section 2.4 in the program-level for more details.

35. Financial Management compliance – Rating Moderately Satisfactory: During project implementation, quarterly Financial Monitoring Reports and annual audit reports were regularly prepared and submitted to the World Bank’s satisfaction, although the latter often delayed, particularly prior to 2010. The PMU encountered a shortage of staff, before in November 2009 a finance and administration officer and an accountant were recruited. Annual budgeting was being prepared, and its monitoring was satisfactory. Disbursement of total GEF funds was low throughout

88 The tool kit provides a practical set of methodologies to assist countries in the risk-based management of obsolete pesticides, including guidance on the management of collection/storage locations, based on international best practice. 89 Sites were evaluated by risk category (high, medium, and low).

58

implementation of the project, with only 26% in June 2011, when the project was restructured. Reasons were significant project implementation delays (often associated with procurement), and the project’s design having been back loaded in terms of expenditures with the more costly disposal activities programmed to take place only at the end of the project.

36. Procurement – Rating Unsatisfactory: Project implementation was significantly affected by continuous procurement-related delays, which were caused by low procurement capacity within the PMU and long authorization processes within the government. Following the departure of the initial procurement officer in June 2008, efforts were undertaken to recruit a replacement, however, with little success.90 At project closure, the position still had not been filled. Procurement support from MoARD was equally cumbersome. The World Bank eventually mitigated the PMU’s weak procurement capacity through temporary technical assistance.91

2.5 POST-COMPLETION OPEARTION/NEXT PHASE

37. Post-completion: To further strengthen the legal and regulatory framework, a Pesticide Registration and Control Regulation was developed in addition to and based on the Pesticide and Control Proclamation that was proclaimed in 2010 (Proclamation No. 674/2010). The new regulation was validated by a national workshop in April 2013, which was organized by a similar project with funds from FAO and the Netherlands.92 The regulation was subsequently approved by MoARD and is currently expected to be submitted to the Council of Ministers for further approval and final enactment. At the same time, the MoARD is determined to fully implementing Proclamation No. 674/2010.

38. Next phase: At project appraisal, the Ministry of Health was in possession of 450 tonnes of DDT that were stored near Addis Ababa. As Ethiopia was still relying on DDT for malaria control at appraisal, the stocks had not been declared obsolete and had therefore not been included in ASP-P1.93 In 2012 however, Ethiopia eventually suspended the use of DDT. The 450 tonnes of DDT near Addis Ababa were subsequently declared obsolete, albeit too late to be included in ASP-P1. The Ministry of Health is currently in the process of conducting a nation-wide preliminary inventory exercise, before the stocks are planned to be adequately inventoried through PSMS in collaboration with Ministry of Agriculture. Additionally, following the development of a container management strategy under ASP-P1 Ethiopia, the government is currently exploring the possibility of conducting a pilot project in selected areas with support from Crop Life Ethiopia.

90 The general paucity of trained procurement specialists in Ethiopia and competition between projects caused a lengthy and unsuccessful recruitment process In November 2009, an experienced procurement officer had declined the final offer due to an unattractive remuneration scheme, and the following candidate resigned two months after his appointment in June 2010. Procurement support received from the MoARD was also characterized by a lack of skilled staff. 91 The World Bank provided temporary assistance in April 2010 by funding an international consultant team that conducted five project management support missions between July 2010 and January 2011 to help the PMU update and refine project management tools, and finalizing the bidding document for the disposal. 92 The Pesticide Risk Reduction Program (PRRP) aims to contribute to a sustainable pesticide management system by focusing on capacitating pesticide registration and post registration. 93 DDT is being phased out internationally under the Stockholm Convention on POPs. It is no longer permitted for use in agriculture. However, several countries, including Ethiopia, still rely on DDT for the control of mosquito vectors of malaria.

59



39. Relevance – Rating Substantial: Despite having been too ambitious, the project’s original objectives, design and implementation were relevant when the project was prepared, and consistent with the World Bank’s Country Assistance Strategy for Ethiopia, which targeted to increase attention to environmental issues.94 Even though they remained ambitious after they were revised, the project’s objective, design, and implementation remained and continue to be relevant to the national development priorities today. The Government of Ethiopia places strong emphasis on preserving the environment, reducing pollution, and supporting efforts to improve the quality of life in cities and rural areas, as evident from the country’s climate-resilient green economy strategy which focuses on achieving economic development goals in a sustainable way.95 The project’s objective is also consistent with the World Bank’s Country Partnership Strategy for Ethiopia issued in 2012,96 which supports a multifaceted approach addressing the challenges posed by climate change, including but not limited to interventions in the agriculture sector; better management of natural resources such as land, forestry, groundwater; and promotion of green technologies.

40. The project supported the Government of Ethiopia in complying with the Stockholm Convention on POPs, and the Basel Convention on the control of trans-boundary movements of hazardous wastes and their disposal. In addition, the government has shown increasing commitment to eliminating obsolete pesticides and introducing measures to reduce future related risks, as demonstrated by its engagement in a similar project, the Pesticide Risk Reduction Program (PRRP – Ethiopia), which is planned to be completed by the end of 2013 (see Section 2.5).


41. Taking account of the project’s Level 1 restructuring, the project has been evaluated against its original and revised PDO by measuring achievements at the time of the project’s restructuring (June 2011) and project closing (June 2013), respectively.

42. Original PDO achievement – Rating Moderately Unsatisfactory: The assessment took the project’s seven PDO indicators into account, of which two were fully (100%), two partly (50%), and three not (0%) achieved. The three main project outcomes measured by indicators include: (i) sound disposal and management of obsolete pesticides; (ii) prevention of accumulation of new stockpiles of disposal and management of obsolete pesticides; and (iii) enhanced capacity and institutional strengthening of pesticide management.97

43. Sound disposal and management of obsolete pesticides. The level of risks emanating from publicly held obsolete pesticides and associated wastes to adjacent and wider communities or critical natural resources was determined and quantified through the completion of a national inventory in May 2010 (PDO Indicator #1 – 100%). This information (geo-location of the storage sites, whether the storage

94 Country Assistance Strategy for the Federal Democratic Republic of Ethiopia, (Report No. 25591-ET), March 24, 2003 95 Ethiopia’s Climate-Resilient Green Economy Strategy – The path to sustainable development, Federal Democratic Republic of Ethiopia, 2011 96 Country Partnership Strategy for the Federal Democratic Republic of Ethiopia, (Report No. 71884-ET), August 29, 2012 97 Achievements were identified based on intended outcomes presented in Section B.1 and Annex I of the PAD (Report No: 39901-ET).

60

sites were protected or not, identity of the contaminant, its quantity, and general condition of the stockpiles) allowed the government to risk profile the stocks and to prepare the CESA. However, at restructuring, the CESA was still being finalized and not yet completed (PDO Indicator #2 – 0%). Due to substantial project implementation delays, the disposal had not begun at restructuring. The bidding document for the disposal tender was completed and cleared by the World Bank, but no publicly held obsolete pesticides had been disposed of yet (PDO Indicator #3 – 0%).

44. Prevention of accumulation of new stockpiles of obsolete pesticides. Better regulation and control of the registration, production, storage, and sale of pesticides was achieved by strengthening the legal and regulatory frameworks for pesticide management. A Pesticide Registration and Control Proclamation was approved in January 2010 by the Council of Ministers, proclaimed by Parliament in June 2010, and published in August 2010 in a federal newspaper. However, measures to strengthen compliance with the Basel Convention and the Rotterdam Convention had not been undertaken (PDO Indicatory #4 – 50%). Important steps had been made towards preparing a national pesticide management strategy, aiming to enhance the sustainability of MoARD’s organizational and financial provisions to deal with obsolete pesticide and associated waste; however, at restructuring the strategy had not been finalized yet (PDO Indicator #5 – 0%). In collaboration with WWF and in consultation with various ministries, NGOs and other stakeholders, a communication strategy was finalized in March 2009.98 The strategy comprised an outreach program focusing on farmers and rural communities including a number of key events to support the prevention component. The project subsequently raised stakeholders’ awareness of the new proclamation, through a three-day workshop in May 2011. Awareness was not raised on the health and environmental hazards of pesticides and encouraging safe pesticide handling, as the envisaged radio, TV, and print media materials had not been distributed/aired by the time of restructuring.

45. Enhanced capacity and institutional strengthening of pesticide management. Between 2010 and 2011, PMU members and representatives from MoARD participated in several pesticide management trainings. Capacity was strengthened as became evident through successful completion of the inventory, and adequate preparation of safeguard instruments (PDO Indicator #6 – 100%). However, procurement capacity at the time of restructuring remained low (see Section 2.4), which continued to delay project implementation (PDO Indicator #7 – 50%).

46. Revised PDO achievement – Rating Moderately Satisfactory: The assessment took the project’s two revised PDO indicators into account, of which one was fully (100%) and the other partly (35%) achieved. In addition, seven Intermediate Outcome Indicators framed the evaluation. Two of these were not taken into account as they corresponded with Original PDO Indicators that had been achieved prior to restructuring; of those taken into account, four were fully and one was not achieved. Outcomes were slightly modified following the restructuring and included: (i) Sound disposal and management of obsolete pesticides and associated waste including safeguarding; (ii) reduction of re-accumulation of obsolete pesticides and associated waste through regulations, pest management, and stakeholder involvement; and (iii) enhanced capacity and institutional strengthening of pesticide management.

98 The communication strategy was developed during a workshop in May 2008 involving participants from various ministries, NGOs, and other stakeholders, and subsequently finalized during various consultative meetings.

61

47. Sound disposal and management of obsolete pesticides and associated waste, including safeguarding. The inventory had revealed a total of 449 tonnes of obsolete pesticides and 842 tonnes of associated waste, consisting of empty containers and contaminated material, equipment, and soil99 across 103 stores. Information obtained through the national inventory, was effectively used to inform the preparation of a CESA (Intermediate Outcome Indicator #2 - 100%).100

48. The project reduced risks emanating from publicly held obsolete pesticides to communities, natural resources, and the global environment through the sound disposal of all inventoried obsolete pesticides. To maximize risk reduction and by recognizing that funds would be insufficient to dispose of and/or safeguard all inventoried obsolete pesticides and associated waste, the government and the World Bank agreed to give priority to disposing of obsolete pesticides over safeguarding associated waste as these were considered posing the larger risk.

49. As a result, at project closing, all inventoried 450 tonnes of obsolete pesticides had been disposed of overseas;101 however, none of the 850 tonnes of associated waste had been disposed of or safeguarded (Revised PDO Indicator #1 – 35%).102 An additional 395 tonnes of obsolete pesticides were disposed of overseas in January 2012 through parallel funding by CLI.103 These were legacy stocks from the FAO-II project.

50. Reduction of re-accumulation of obsolete pesticides and associated waste through regulations, pest management, and stakeholder involvement. A national pesticide management strategy was developed and endorsed by the Steering Committee on June 25, 2013 (Revised PDO Indicator #2 – 100%; and Intermediate Outcome Indicator #5 – 100%). The strategy focused on sound pesticide management based on a holistic, integrated, and systematic approach. Equally endorsed by the Steering Committee on June 25, 2013 were action plans for a national container management strategy (Intermediate PDO Indicator #3 – 100%), and pesticide management. The project completed awareness raising activities as outlined in its communication strategy. In 2012 and 2013, posters and

99 Including 360 tonnes of contaminated building (contaminated concrete floor, contaminated walls), which required cleaning technologies not usable for contaminated soils. 100 The document was finalized on December 14, 2011, found satisfactory by the World Bank on February 21, 2012, and disclosed to the World Bank InfoShop on February 29, 2012. 101 Incineration certificates are dated to June 22, 2013. 102 450 tonnes (35%) of the inventoried 1,300 tonnes of publicly held obsolete pesticides and associated waste had been disposed of. 103 Incineration certificates are dated to January 20, 2012.

850

450450

850

449

842

0

100

200

300

400

500

600

700

800

900

ObsoletePesticides

Associated Waste(particularly

ContaminatedSoils)

Tonn

es

Inventoried 2010

Targets set atRestructuring June2011

Eliminated June2013

PartiallySafeguarded(reported/notverified) June 2013

Figure 3.1: Publicly Held Obsolete Pesticides and Associated Waste Inventoried and Eliminated/Safeguarded

62

similar materials were prepared and distributed to raise awareness on the harmful nature of pesticides to human health and the ecosystem; and a one-year TV and radio program was conducted in three local languages to raise awareness on pesticides use (Intermediate PDO Indicator #4 – 100%). The construction of a new storage warehouse for temporary storage of imported pesticides and thus better pesticide management, was not initiated as a result of time constraints (Intermediate PDO Indicator #7 – 0%).104

51. Enhanced capacity and institutional strengthening of pesticide management. In addition to the capacity building achieved prior to restructuring, the project did not achieve further institutional strengthening of pesticide management.

52. Weighted average PDO achievement – Rating Moderately Satisfactory: At restructuring in June 2011, the project had disbursed US$0.5 million, representing 21.5% of total disbursed funds of US$2.32 million. On this basis and as illustrated by the table below, project achievements against the original PDO are rated moderately unsatisfactory, whereas project achievements against the revised PDO are rated moderately satisfactory. The weighted average of the project’s PDO achievement is therefore rated moderately satisfactory.

Against Original PDO Against Revised PDO Overall

1. Rating Moderately Unsatisfactory Moderately Satisfactory -

2. Rating Value 3 4 -

3. Weight (% disbursed before/after PDO change) 21.5% 78.5% -

4. Weighted value 0.7 3.1 3.8

5. Final rating - - Moderately Satisfactory


53. The project aimed to directly contribute to the GEO of reducing the effects of POPs on the global environment through: (i) improving the quality of life in poor communities, by reducing environmental risks; (ii) improving environmental protection; and (iii) enhancing the capacity of the agricultural sector to better manage crop pests. GEO achievement is measured by PDO achievement and, thus, is also rated moderately satisfactory based on the assessment above (see Section 3.2).

3.4 EFFICIENCY

54. Efficiency – Rating Substantial: This rating represents a broad estimate, as financial and economic data to support this rating are scarce. At appraisal, an economic or financial analysis was not

104 Its construction required the development of an Environmental and Social Impact Assessment (ESIA) which was only completed in April 2011. The assessment’s late completion was caused by a lengthy public consultation process and data collection and analysis, a lengthy procurement process, and the World Bank’s late advice on developing such assessment. In May 2012, it was agreed by the World Bank and PMU to cancel the store’s construction as the remaining project time was considered insufficient.

Table 3.1: Weighted Average PDO Achievement

63

undertaken for ASP-P1 at either the program or at the country-level due to non-measurable benefits of reducing the risk of contaminating the environment.

55. Based on the fact that approximately 71% (US$1.70 million) of disbursed GEF funds (i.e., US$2.38 million) went towards disposal activities under ASP-P1 Ethiopia, the project’s efficiency was primarily assessed by its investment in the disposal of obsolete pesticides and associated waste.

56. During appraisal, the unit cost for disposal was estimated at US$3,400 per tonne (based on approximately US$15 million spent on the disposal of 3,240 tonnes of obsolete pesticides between 1990 and 2004 across Africa). (See Section 3.4 in the program-level ICR for details).

57. At project closing, 450 tonnes105 of obsolete pesticides and associated waste had been disposed of at the cost of approximately US$3,778 per ton, which represents a slightly higher cost than originally anticipated.106 Given the overall high cost involved, cleanup and safe disposal was prioritized on the basis of a detailed inventory of pesticide stockpiles and associated waste, determining the identity of the contaminant, its quantity, and its proximity to the people and environment. In comparison with the originally estimated US$3,400 per ton, the project achieved its goal with a slightly higher cost-benefit than originally estimated. However, the US$ depreciation since 2004 and an increase in the price of fuel, which has increased the cost of disposal in US$ terms, is not taken into account by this calculation; neither are varying distances from collection centers to harbors, varying distances of shipping routes, and responses to call of bid by different companies.


58. Overall outcome - Rating Moderately Satisfactory: On the basis of the project’s efficiency (substantial), relevance (substantial), and PDO achievement (moderately satisfactory), the overall outcome is rated moderately satisfactory. The rating takes into account that despite substantial implementation challenges prior to mid-term review (see Section 2), and a shortage of funds to dispose of and/or safeguard all inventoried obsolete pesticides and associated waste, the project successfully reduced risks to the environment and public health by disposing all inventoried obsolete pesticides. In addition, the project succeeded in significantly improving measures to reduce future re-accumulations by developing a national pesticide management strategy, including action plans for a national container management strategy, and pest management.


59. Poverty impacts, gender aspects, and social development: The disposal of obsolete pesticides and associated wastes had a direct positive impact on poor and farming communities, species and ecosystems, located in proximity to the sites carrying obsolete pesticide stocks and associated waste. Baseline data on affected populations, species, and ecosystems were not gathered at appraisal; exact impacts are therefore difficult to measure.

60. Institutional change/strengthening: The project strengthened the government’s institutional capacity on several levels: (i) The regulatory and legal framework for pest and pesticide management was improved; (ii) the logistics of hazardous waste transportation were internalized and the knowledge

105 Not including the additional 395 tonnes of obsolete pesticides that were disposed of through parallel funding by CLI. 106 Costs include costs associated with repackaging, transport, and incineration of obsolete pesticides and other related activities such as for example the inventory, safeguarding efforts, associated studies, and advisory services.

64

gained can be used to inform future disposal activities; (iii) the experience and skills gained in managing the project can be applied to other projects and subsequently contribute to success; and (iv) knowledge was also gained on sound pesticide management through the various training that were provided to governmental staff.

61. Other unintended outcomes and impacts: A positive outcome was the disposal of 395 tonnes stocks of obsolete pesticides under CLI funding in August 2011. The stocks represented legacy stocks from the earlier FAO-II project, and had been stored near Addis Ababa. An EMP and a Health and Safety Plan were developed, found satisfactory by the World Bank, and subsequently disclosed at the World Bank’s InfoShop and in country in April/May 2010. Disposal activities were subsequently implemented in compliance with World Bank safeguard policies, and in accordance with all necessary international protocols.


62. An ICR mission, including a stakeholder workshop was conducted in the beginning of May 2013 in Addis Ababa, Ethiopia. For a detailed summary of the findings please see Section 8.6. A beneficiary survey was not conducted.


63. Risk to the development outcome - Rating Moderate. The risk is confined to the re-accumulation of obsolete pesticides, and a potential weakening of government commitment to sustaining and building on the project’s outcomes, particularly with respect to prevention activities.

64. The project substantially strengthened pesticide legislation and national capacity, and together with the various activities agreed upon in the national strategy, the government is well positioned to sustainably manage and reduce future accumulations of obsolete pesticides. In addition, the government has shown continued commitment to reducing future related risks, as demonstrated by its engagement in a similar project, the PRRP, which is planned to be completed by the end of 2013 (see Section 2.5).



65. World Bank performance in ensuring quality at entry – Rating Moderately Unsatisfactory: Project preparation of ASP-P1 Ethiopia was conducted in conjunction with preparatory activities at the program-level, which were lengthy (see Section 2.1 in the program-level ICR). While considerable time was invested in overall preparation, key project preparation documents for ASP-P1 Ethiopia only broadly laid out project implementation plans, and lacked country-specific information. The project’s design was characterized by several weaknesses, which translated into substantial project implementation delays (see Section 2.1 in the program-level ICR). FM and procurement assessments adequately outlined potential risks, however, corresponding resulting mitigation measures were weak.

65

66. Quality of World Bank supervision – Rating Moderately Unsatisfactory: Until 2011, the project was supervised by the Task Team Leader (TTL) responsible for ASP at the program-level, who was based at the World Bank’s headquarters in Washington DC. In 2011, a co-TTL was assigned in Ethiopia. Supervision missions took place regularly, albeit only once a year (except in 2010 when a number of technical and operational support missions were conducted).

67. During the first three years, project implementation had been experiencing substantial delays; support from the World Bank was limited, and the project was affected by collaboration issues among partners, especially the World Bank and FAO, at the program-level (see Section 2.2 in the program-level ICR).107 Following the mid-term review, the World Bank team took corrective action by significantly increasing its support to more effectively address the project’s implementation delays. These included funding of an international consultant team that conducted five support missions between July 2010 and January 2011 to help the PMU update and refine project management tools, and finalize the disposal bidding document. In 2011, the World Bank prepared a Level 1 restructuring involving revision of the PDO to better respond to sector realities and country needs; however, the PDO remained ambitious. Final disposal activities were effectively supported through: (i) more frequent and ad-hoc project implementation support from the co-TTL; (ii) reallocation of GEF funds to accommodate the disposal of higher than estimated obsolete pesticides; and (iii) timely preparation of two additional extensions of the closing date to ensure the safe arrival of already shipped obsolete pesticide materials at their destination, PDO achievement, and project funding eligibility of all disposal-related activities. FM review missions were conducted regularly to ensure the project’s FM arrangements remained acceptable to the World Bank.

68. Overall World Bank performance – Rating Moderately Unsatisfactory: The project suffered from shortcomings in project preparation and design at the program-level. Following a resulting slow performance during the first half of the project, the World Bank team gradually improved its performance through corrective actions.


69. Government performance – Rating Moderately Satisfactory: Government commitment at entry towards the issue of obsolete pesticides was evident through the implementation of a similar previous project, described in Section 1.1. The government had therefore acquired substantial experience and was considered experienced and well prepared for pesticide disposal under ASP. In addition, the government had already established the institutional structures for coordinating the implementation of ASP-P1. MoARD through the CPD had established a PMU in 2003. However, during project implementation, MoARD’s cumbersome and long procurement processes significantly contributed to project implementation delays.

70. Implementing agency performance – Rating Moderately Unsatisfactory: Overall, the implementing agency’s performance was moderately unsatisfactory, with gradual improvement throughout project implementation. Significant implementation delays were caused by a poorly managed transition period between the preceding pesticide project (FAO-II) and ASP-P1, and key staff that had unexpectedly left upon completion of the FAO-II project, as new staff had to be appointed and trained

107 Significant delays were caused by lengthy World Bank review and clearance processes of project-related documents. It took eight months to review and clear the CESA TORs, as FAO and the World Bank struggled finding an agreement on a format that was in line with technical requirements; no objections were often provided with delays.

66

over time. Particular delays were caused by the lack of adequate procurement staff throughout project implementation (see Section 2.4 for more details). However, the agency’s performance improved gradually, and particularly during the last two years of project implementation, it demonstrated strong commitment towards achieving the PDO.

71. The agency complied with all World Bank loan covenants, and executed its fiduciary duties. Delays were noted in submitting Financial Monitoring Reports, however, no issues in view of FM or procurement were noted during project implementation. The agency equally ensured safeguard compliance.

6. LESSONS LEARNED

72. The most significant lessons learned are outlined in Section 6 of the program-level ICR, as the experience they are based on was often reflected across all country-specific projects. Lessons learned that were particularly relevant to ASP-P1 Ethiopia include:

• Both, the implementing agency and the World Bank team need to be staffed with appropriate technical expertise, which is crucial for sound preparation and implementation of projects associated with pesticide disposal. As was evident during implementation, the disposal of obsolete pesticides and associated waste required highly specialized technical expertise, particularly for the preparation of disposal activities (i.e., procuring the specialized services). The technical support the project received, particularly from the TAD proved to be crucial for the project’s progress toward PDO achievement.

• Knowledge exchange between participating countries of a regional project can be value adding. CLI had provided support to ASP-P1 Ethiopia and ASP-P1 Tanzania by financing a TAD. The TAD was represented by the same person, which automatically facilitated the knowledge exchange between the two countries with respect to disposal activities. Both countries confirmed the value added through this knowledge exchange.

• Possible low procurement capacity should be adequately reflected when assessing risks. Project implementation was significantly affected by continuous procurement-related delays. The general paucity of trained procurement specialists in Ethiopia and competition between projects for procurement staff should be taken into account during preparation.



67

8. ANNEXES


a. Project cost by component – All sources (in US$ million equivalent) – as of 06/30/2013

Component


equivalent)



Actual as % of appraisal estimate

1. Disposal of obsoletes pesticides 2.142 3.017 141%

2. Prevention of accumulation 0.194 0.3 155%

3. Capacity building 0.292 0.116 40%

4. Project management 0.421 0.675 160%

Total 3.048 4.108 135%

b. Financing – as of 06/30/2013

Sources of Funds Type of Co-financing


equivalent)

Actual/Latest Estimate (in US$ million equivalent)


Borrower In-kind 0.428 0.428 100%

Global Environment Facility (GEF) Grant 2.620 2.380 91%

Crop Life International Parallel Financing 0 1.300 -

Total 3.048 4.108 135%


1. Taking account of the project’s Level 1 restructuring in June 2011, the following table provides a comprehensive account of qualitative and quantitative information of outputs realized against both original (achievements at the time of restructuring in June 2011) and revised components (achievements at the time of project closing in June 2013).

Original Components as presented in the GEF Trust Fund Grant Agreement

Outputs achieved at the time of restructuring in June 2011

Part A – Obsolete pesticides inventory: Achievements under this component are rated Moderately Unsatisfactory

Table 8.1: Outputs by Original Components as presented in the GEF Trust Fund Grant Agreement

68

Carrying out of a detailed inventoried of the recipient’s publicly held obsolete pesticides stocks and associated waste through: a. Collection of data and compilation of a

database; b. A risk-based prioritization of the stocks

identified in the database; and c. Preparation of the CESA.

a. An inventory of publicly held obsolete pesticides was completed and validated in May 2010, including 100% data entry into an inventory database, which has been under the management and oversight of FAO, as part of their PSMS (Original PDO Indicator #1 – 100%).

b. The PMU and the government’s staff were trained by FAO in using the system (2010), and used the information on stocks identified in the database for a risk-based prioritization of the different sites, and as basis for the preparation of the CESA.

c. At restructuring, a CESA was still being finalized and not yet completed (Original PDO Indicator #2 – 0%).

Part B – Disposal of obsolete pesticides: Achievements under this component are rated Moderately Unsatisfactory Treatment and/or disposal of publicly held obsolete pesticides stocks and associated waste by: a. Carrying out of a technological assessment of

available treatment and disposal options; b. Implementation of the treatment and/or

disposal technology selected; c. Improving the operations of obsolete pesticide

stocks collection centers, including improving safety measures at those centers; and


a. A technological assessment of available treatment options was carried out by WWF in 2008.

b. Disposal activities, started at the time of restructuring, thus implementation of the treatment and/or disposal technology selected was not completed (Original PDO Indicator #3 – 0%).

c. Preparations for the construction of a new storage warehouse for temporary storage of imported pesticides and thus better pesticide management were initiated, including the development of an Environmental and Social Impact Assessment (ESIA) which was completed in April 2011.

d. Recycling of metal and/or plastic containers was addressed by a container management study, which was being conducted at the time of restructuring.

Part C – Prevention of obsolete pesticide accumulation: Achievements under this component are rated Moderately Unsatisfactory Carrying out of activities aimed at preventing obsolete pesticide accumulation, including: a. Awareness raising activities on the program,

health and environmental hazards of pesticides, and IPM;

b. A review and update of the legal and regulatory framework to control obsolete pesticide accumulation and enforcement mechanisms;

c. Preparation of pesticide management strategy; and

d. Strengthening the pesticide management capacity of storage warehouses.

a. In collaboration with WWF, a communication strategy was finalized in March 2009. The strategy comprised an outreach program focusing on farmers and rural communities including a number of key events to support the prevention component. The project subsequently raised stakeholders' awareness of the new proclamation, through a three-day workshop in May 2011. However, individual outreach activities envisaged under the communication strategy were not implemented at restructuring.

b. Better regulation and control of the registration, production, storage, and sale of pesticides was achieved by strengthening the legal and regulatory framework for pesticide management. A Pesticide Registration and Control Proclamation was approved in January 2010 by the Council of Ministers, proclaimed by Parliament in June 2010, and published in August 2010 in a federal newspaper. However, measures to strengthen compliance with the Basel Convention and the Rotterdam Convention had not been undertaken (PDO Indicatory #4 - 50%).

c. Important steps were made towards preparing a national pesticide management strategy, aiming to enhance the sustainability of MoARD's organizational and financial provisions to deal with obsolete pesticide and associated waste; however, at restructuring the strategy was not yet finalized (PDO Indicator #5 - 0%).

69

d. Preparations for the construction of a new storage warehouse for temporary storage of imported pesticides and thus better pesticide management were underway.

Part D – Capacity building: Achievements under this component are rated Moderately Satisfactory. Carrying out of capacity building activities, including: a. Provision of training to relevant staff of the

recipient on laboratory analysis, health and safety issues, and environmental monitoring.

b. Strengthening awareness on the pesticide management legal framework and homologation systems; and

c. Development and implementation of a pesticide storage and IPM training program.

a. Between 2010 and 2011, PMU members and representatives from MoARD participated in several trainings such as pesticide management (December 2010), safeguards and preparing EMPs, and PSMS. Capacity was strengthened as became evident though application of the acquired knowledge during the inventory, and preparation of safeguard instruments (PDO Indicator #6 – 100%). However, despite the strengthened capacity, the PMU was still facing low procurement capacity at the time of restructuring, which continued to delay project implementation (PDO Indicator #7 – 50%).

b. A Pesticide Registration and Control Proclamation was approved in January 2010 by the Council of Ministers, proclaimed by Parliament in June 2010, and published in August 2010 in a federal newspaper. Copies of the proclamation were also distributed to approximately 45 government representatives during an associated training.

c. Training in store keeping, pesticide management, and maintaining the PSMS was provided to the PMU and representatives of the ministries.

Part E – Project management: Achievements under this component are rated Moderately Unsatisfactory. Supporting the PMU in carrying out project coordination, implementation, monitoring, and evaluation, through the provision of technical advisory services, equipment, and operational support.

An unclear partnership arrangement at the program-level hampered timely technical support, particularly during the first three years. • The highly technical and specialized expertise required for

implementation, supervision and monitoring of project activities the PMU was supposed to receive from FAO’s TSU was not delivered to the extent planned. Valuable support was given in terms of equipment through the provision of the PSMS, and associated training. However, limited technical advisory services were given on the pre-qualification, bidding, and execution of the specialized services associated with procuring the selection processes of the CESA and disposal service providers, and M&E.

• The urgently needed operational support from the World Bank was not sufficiently provided prior to mid-term review; however, increased supervision, operational and technical support in 2010, and the subsequent deployment of an in-country co-TTL, improved the level of support.

70

Revised Components (Level 1 Restructuring June 30, 2011)108

Specific revisions are underlined

Outputs achieved at the time of project closing

Part A – Obsolete pesticidesinventory: Achievements under this component are rated Moderately Satisfactory




a. See outputs achieved by Original Part B. b. See outputs achieved by Original Part B. c. A CESA identifying mitigation measures to treat possible negative

environmental and social impacts that were considered likely to occur during implementation of the project, was completed in February 2012, found satisfactory by the World Bank in December, 2011, and disclosed to the World Bank InfoShop on February 29, 2012. (Intermediate Outcome Indicator #2 - 100%).

Part B – Disposal and/or safeguarding of obsolete pesticides and associated waste: Achievements under this component are rated Moderately Unsatisfactory.

Safeguarding and/or disposal of publicly held obsolete pesticides stocks and associated waste by: a. Carrying out a technological assessment of

available treatment and disposal options; b. Implementation of the safeguarding and/or

disposal technology selected; c. Improving the operations of obsolete pesticide

stocks collection centers, including improving safety measures at those centers; and


a. See outputs achieved by Original Part B. b. 450 tonnes (35%) of the inventoried 1,300 tonnes of publicly held

obsolete pesticides and associated waste had been disposed of overseas. These included 450 tonnes of obsolete pesticides (against 450 tonnes inventoried), but none of the 850 tonnes of associated waste due to exhausted project funds. Reportedly, the project safeguarded some of the associated waste in form of cleaning contaminated buildings, but no exact data was available/could be verified (Revised PDO Indicator #1 - 35%). Additional 395 tonnes of obsolete pesticides were disposed of overseas in January 2012 by a CLI contracted and funded UK company. The stocks represented legacy obsolete pesticides, which originally had not been included in ASP-P1 Ethiopia.

c. Preparations for the construction of a new storage warehouse for temporary storage of imported pesticides and thus better pesticide management were underway.

d. An action plan for a national container management strategy (Intermediate PDO Indicators #3 – 100%) was endorsed by the Steering Committee on June 25, 2013.

Part C – Reduction of obsolete pesticides and associated waste re-accumulation: Achievements under this component are rated Moderately Satisfactory. Carrying out activities aimed at reducing obsolete pesticide and associated waste re-accumulation, including: a. Awareness raising activities on the program,

health and environmental hazards of pesticides. b. A review and update of the legal and

regulatory framework to control obsolete pesticide accumulation and strengthen enforcement mechanisms;

a. The project completed awareness raising activities as outlined in its communication strategy. In 2012 and 2013, posters and similar materials were prepared and distributed to raise awareness on the harmful nature of pesticides to human health and the ecosystem; and a one-year TV and radio program was conducted in three local languages to raise awareness on pesticides use (Intermediate PDO Indicator #4 - 100%).

b. See outputs achieved by Original Part C. c. A national pesticide management strategy was developed and

108 Restructuring Paper on a Proposed Project Restructuring of Africa Stockpiles Program – Project 1 to the Republic of Tunisia, (Report No: 62797-TN), June 20, 2011

Table 8.2: Outputs by Revised Components as presented in the Level 1 Restructuring June 30, 2011

71

c. Preparation of a sustainable pesticide management strategy;

d. Strengthening the pesticide management capacity of storage warehouses;

e. Provision of training to relevant staff of the recipient on health and safety issues, and environmental monitoring; and

f. Strengthening awareness on the pesticide management legal framework and homologation systems.

endorsed by the Steering Committee on June 25, 2013 (Revised PDO Indicator #2 - 100%; and Intermediate Outcome Indicator #5 - 100%). The strategy focused on sound pesticide management based on a holistic, integrated, and systematic approach. Equally endorsed by the Steering Committee on June 25, 2013 were action plans for a national container management strategy (Intermediate PDO Indicator #3 - 100%), and pesticide management.

d. The construction of a new storage warehouse for temporary storage of imported pesticides and thus better pesticide management was aborted and not completed as a result of time constraints (Intermediate PDO Indicator #7 - 0%).

e. See outputs achieved by Original Part D. f. See outputs achieved by Original Part D.

Part D – Capacity building: Achievements under this component are rated Moderately Unsatisfactory. Carrying out of capacity building activities, including: a. Strengthening the recipient’s capacity to carry

out environmental assessments. b. Reviewing the recipient’s pesticide

management practices and providing training on improved methodologies.

c. Reviewing the implementation capacity of the PMU and providing requisite training.

a. See outcomes achieved under Original Part D. b. See outcomes achieved under Original Part D. c. See outcomes achieved under Original Part D.

Part E – Project management: Achievements under this component are rated Moderately Satisfactory Supporting the PMU in carrying out project coordination, implementation, monitoring, and evaluation, through the provision of technical advisory services, equipment, and operational support.

In addition to outputs achieved under Original Part D, the World Bank provided effective operational support during the final disposal activities. For technical support, a TAD was eventually recruited and financed by CLI, which positively affected project implementation.




a. Task Team Leaders Lending

(Task Team, specialization, and unit as outlined in the PAD)

Names Specialization Unit Responsibility/Specialty

Abushakra, Hadi Chief Legal Counsel LEGMS

Brackmann, Stefanie Environmental Specialist MNSRE

Chihuguyu, Evelyn Assistant AFTS4

Di Leva, Charles Chief Counsel LEGEN

Eshetu, Yimer Senior Financial Management Specialist AFTFM

72

Fissha, Azeb Extended Term Consultant AFTS2

Gorman, Steve Lead Environmental Specialist ENVGC

Jordy, Denis Senior Agricultural Specialist AFTS4 Co-TTL

Kaniaru, Muthoni Counsel LEGAF

Konare, Amadou Senior Environmental Specialist AFTS1

Kranz, Frederick Senior Procurement Specialist MNACS

Kristensen, Peter Senior Environmental Specialist AFTS4 TTL

Maber, Steven Senior Operations Officer MNSRE TTL

Ndiaye, Marie-Jeanne Language Program Assistant AFTS4

Permalpillai-Essex, Jeeva Lead Operations Officer AFTS3

Rajat, Narula Senior Finance Officer LOAG2

Temechew, Abiy Procurement Analyst AFTPC

Tran, Lucie Operations Officer AFTS4

Tynan, Ellen Senior Environmental Specialist ENVMP

Warner, Christopher Senior Environmental Specialist AFTS1 TTL

Waters, Warren Regional Environmental and Safeguards Advisor AFTSD

Supervision/ICR


Abate, Asferachew Environmental Specialist Co-TTL

Aryal, Dinesh Senior Operations Officer TTL

Badisso, Shimelis Woldehawariat Procurement Specialist

Belaye, Abiye Demissie Financial Management Specialist

Brackmann, Stefanie Environmental Specialist

Doucouré, Djibril Safeguards Specialist Consultant

Dwumfour, Edward Senior Natural Resource Management Specialist

Co-TTL

Fissha, Azeb Operations Analyst

Freminatos Abraham, Tafesse Financial Management Specialist

Fye, Serigne Omar Senior Environment Specialist

Guibert, Yves

Jordy, Denis Senior Environmental Specialist Co-TTL

Kristensen, Peter Program Manager TTL

Lagnaoui, Abdelaziz Senior Environmental Specialist

Peled, Ayala Environmental Economist

Prevoo, Dirk Senior Operations Officer TTL

73

Rechbauer, Gabriele Senior Environmental Operation Specialist

Said, Sophia Team Assistant

Schmidt, Veruschka Strategy Officer

Tadesse, Meron Financial Management Analyst

Temechew, Abiy Procurement Analyst

Tsegaye, Lakech Team Assistant

Tynan, Ellen Environmental Specialist TTL

b. Staff time and cost –Staff time and costs associated with program preparation and supervision was not possible to determine as a result of poor record keeping in the system.




4. An ICR mission including a stakeholder workshop was conducted in the beginning of May 2013 in Addis Ababa, Ethiopia. Stakeholders highlighted the following key points:

• The most important achievements that resulted from prevention activities included the preparation of a strategic plan, awareness raising among different target groups, the development of a legal framework that resulted in better utilization of the registration/purchasing system, and capacity building among government officials and users.

• The most important achievements that resulted from disposal activities included clearing up storage facilities, knowledge transfer from the international disposal company to national staff, identification of 850 tonnes of contaminated soils, and completion of a nation-wide inventory of obsolete pesticides.

• Key challenges included a shortage and high turnover of PMU staff, improper project management, weak procurement capacity, project design problems, project overlaps between the preceding and ASP project, slow response from different countries regarding the shipment, a lack of M&E, a lack of laboratories to determine the level of contamination, lack of awareness on dangers among end-users of pesticides, and lack of heavy trucks for transportation.

• The sustainability of project outcomes could be ensured through effective use of the gained opportunities, addressing the remaining associated waste, use of IPM and IVM, assessing alternative means for disposal, implementing improved legislation, increased and sustained government commitment, policies, strategies, and budget allocation, disposal and prevention strategies, continued awareness raising, and collaboration with the private sector.

• The institutional and partnership arrangement was lacking a harmonized implementation system, resulting in weak awareness and coordination among and between stakeholders.

• Key lessons learned were: (i) A pesticide management system needs to be in place; (ii) recruitment of local staff minimized costs; (iii) empty containers needed to be triple-rinsed

74

before they could be reused; (iii) knowledge sharing with other countries was value-adding; (iv) local stakeholders had to be involved from the beginning on; (v) emphasis on training was important; (vi) a pesticides poisoning/testing center should be in place; (vi) pesticides safety and management should be covered by the media; (vii) a system for container management needs to be in place; (viii) personal protective equipment needs to be available; and (ix) technical requirements per year need to be assessed in advance.


5. The borrower submitted a completion report; messages contained therein were integrated into this document.

6. Comments received on the draft ICR were equally incorporated. Additional comments included the following: a. Inventory: We have satisfactorily complete inventory and therefore we do not accept the rating

for inventory as Moderately Satisfactory. Please change this rating to Satisfactory.

• World Bank response: The data sheet notes that Original PDO Indicator #1 (Inventory database of publicly held obsolete pesticides stocks and associated waste in place and in use by the PMU and the recipient’s staff) was fully achieved (100%).

b. PDO Indicator 1: ASP-P1Ethiopia has disposed of all inventoried obsolete pesticides (450 tonnes) though the original target was 250 tonnes. Moreover, the project didn’t have any plan and resources to fully safeguard and or dispose all inventoried associated waste. Hence we strongly disagree on the downgraded rating of our performance of this PDO Indicator and recommend it to be updated as Satisfactory.

• World Bank response: The Revised PDO Indicator #1 (as per the Level 1 Restructuring in June 2011) was: “Inventoried publicly held obsolete pesticides and associated waste disposed of and/or safeguarded by the end of the project”. The inventory noted 450 tonnes of obsolete pesticides and 850 tonnes of associated waste. At restructuring, it was agreed to dispose of 250 tonnes of obsolete pesticides and safeguard all other inventoried obsolete pesticides and associated waste. However, it was recognized that the Revised PDO Indicator #1 continued to be ambitious and funds insufficient. Therefore, the government and the World Bank agreed to give priority to disposing of obsolete pesticides over safeguarding associated waste as these were considered posing the larger risk. The Overall Outcome rating took these circumstances into account, because even though the Revised PDO Indicator #1 was only achieved 35% (i.e., 450 tonnes of obsolete pesticides disposed of against inventoried 1,300 tonnes of obsolete pesticides and associated waste), the Overall Outcome was rated Moderately Satisfactory.

c. Efficiency: Our cost of disposal and safeguarding is about 3,011 US$/tonne but in the ICR it is stated that to be 3,778 US$/tonne. As our actual cost is less than the proxy (3,400 US$/tonne) we believe that ASP Ethiopia is cost effective and efficient.

• World Bank response: The ICR’s calculation took disbursed GEF funds that were spent towards disposal activities as basis. This included costs associated with repackaging, transport, and incineration of obsolete pesticides and other related activities.

75

d. Performance of Implementing Agency: We would be grateful if the downgraded rating (Moderately Unsatisfactory) be upgraded to be at least Moderately Satisfactory as all efforts were made by MOA/APHRD for the successful completion of ASP-P1 Ethiopia.

• World Bank response: The rating is based on the discussion outlined in Section 2.2 and 5.2, and found adequate.

e. Overall performance rating: We deserve to be evaluated as Satisfactory as we have disposed 450 tonnes (almost double of the original plan), prepared pesticide mgt strategy, proclaimed pesticide registration and control proclamation, drafted pesticide registration and control regulation, raised awareness on pesticide issues, disposed additional 395 tonnes through CLI funding and implementing PRRP Ethiopia project, which mainly focuses on pesticide registration and post registration capacity building etc.


7. No comments were received from co-financiers and other partners/stakeholders on the ASP-P1 Ethiopia draft ICR.

8. Comments of co-financiers and other partners/stakeholders on the program-level ICR are summarized in Section 8.8 in the program-level ICR.

76

8.9 VERBATIM COMMENTS ON THE DRAFT ICR

77

TECHNICAL ANNEX II: ASP-P1 MALI

78


AELP Africa Emergency Locust Project



CAS Country Assistance Strategy



CMDT Compagnie Malienne pour le Développement des Textiles

DNACPN Direction Nationale de l’Assainissement et du Contrôle des Pollutions et des Nuisances

EC European Commission


EMPRES Emergency Prevention System for Trans-boundary Animal and Plant Pests and Diseases







IFR Interim Unaudited Financial Report












79

PIF Project Identification Form







RAP Resettlement Action Plan

SDC Swiss Agency for Development and Cooperation

SIDA Swedish International Development Cooperation Agency






80

TABLE OF CONTENTS

1. PROJECT CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES AND DESIGN ........ 86 1.1 CONTEXT AT APPRAISAL .............................................................................................................................................. 86 1.2 ORIGINAL PROJECT DEVELOPMENT OBJECTIVE AND INDICATORS .................................................................. 87 1.3 REVISED PROJECT DEVELOPMENT OBJECTIVE AND INDICATORS .................................................................... 88 1.4 MAIN BENEFICIARIES..................................................................................................................................................... 88 1.5 ORIGINAL AND REVISED COMPONENTS ................................................................................................................... 88 1.6 OTHER SIGNIFICANT CHANGES ................................................................................................................................... 90

2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES ....................................................................... 91 2.1 PROJECT PREPARATION, DESIGN, AND QUALITY AT ENTRY .............................................................................. 91 2.2 IMPLEMENTATION .......................................................................................................................................................... 92 2.3 MONITORING AND EVALUATION (M&E) DESIGN, IMPLEMENTATION, AND UTLIZATION ........................... 93 2.4 SAFEGUARD AND FIDUCIARY COMPLIANCE ........................................................................................................... 94 2.5 POST-COMPLETION OPERATION/NEXT PHASE ........................................................................................................ 96

3. ASSESSMENT OF OUTCOMES ........................................................................................................................................... 96 3.1 RELEVANCE OF OBJECTIVES, DESIGN, AND IMPLEMENTATION ........................................................................ 96 3.2 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVE ..................................................................................... 96 3.3 ACHIEVEMENT OF GLOBAL DEVELOPMENT OBJECTIVE ..................................................................................... 98 3.4 EFFICIENCY ...................................................................................................................................................................... 98 3.5 JUSTIFICATION OF OVERALL OUTCOME RATING ................................................................................................... 98 3.6 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS .............................................................................. 99 3.7 SUMMARY OF FINDINGS OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS .................... 100

4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME .......................................................................................... 100 5. ASSESSMENT OF WORLD BANK AND BORROWER PERFORMANCE .................................................................. 100

5.1 WORLD BANK PERFORMANCE .................................................................................................................................. 100 5.2 BORROWER PERFORMANCE ....................................................................................................................................... 101

6. LESSONS LEARNED ............................................................................................................................................................ 101 7. COMMENTS ON ISSUES RAISED BY BORROWER, IMPLEMENTING AGENCY, AND PARTNERS ................ 102 8. ANNEXES ............................................................................................................................................................................... 102

8.1 PROJECT COSTS AND FINANCING ............................................................................................................................. 102 8.2 OUTPUTS BY COMPONENT ......................................................................................................................................... 103 8.3 ECONOMIC AND FINANCIAL ANALYSIS .................................................................................................................. 105 8.4 WORLD BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS ................................. 105 8.5 BENEFICIARY SURVEY RESULTS .............................................................................................................................. 105 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS ............................................................................................ 105 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR ................................................................. 105 8.8 COMMENTS OF CO-FINANCIERS AND OTHER PARTNERS/STAKEHOLDERS ................................................... 107 8.9 VERBATIM COMMENTS ON DRAFT ICR ................................................................................................................... 107

81


Country: Mali Project Name: Africa Stockpiles Program - Project 1 Project ID: P103189 L/C/TF Number: TF57678 ICR Date: 11/27/2013 ICR Type: Core ICR Lending Instrument: APL Borrower: Republic of Mali Original Total Commitment:

US$2.55 million


Revised Amount: - Environmental Category: A Global Focal Area: CHEM Implementing Agency: Direction Nationale de l’Assainissement et du Contrôle des Pollutions et des Nuisances (DNACPN) Co-financiers and Other External Partners: Government of Republic of Mali, CropLife International (CLI), CropLife-Mali, European Commission (EC), Swedish International Development Cooperation Agency (SIDA), Swiss Agency for Development and Cooperation (SDC) and Denmark Ministry of Foreign Affairs, French Global Environment Facility (FFEM), The Netherlands, UN Food and Agriculture Organization (FAO), Pesticide Action Network (PAN-UK and PAN-Africa) and World Wildlife Fund (WWF).

B. Key Dates

Process Date Process Original Date Revised / Actual Date(s) Concept Review: 07/17/2002 Effectiveness: 02/12/2007 12/26/2006 Appraisal: 04/19/2004 Restructuring: - - Approval: 12/05/2006 Mid-term Review: 02/11/2009 02/06/2009

Closing: 08/31/2010 12/31/2011, 06/30/2012,

12/31/2012 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Global Environment Outcome: High World Bank Performance: Moderately Unsatisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of World Bank and Borrower Performance World Bank Ratings Borrower Ratings


82

Quality at Entry: Moderately Unsatisfactory Government: Satisfactory

Quality of Supervision: Moderately Unsatisfactory Implementing Agency/Agencies: Moderately Satisfactory


Moderately Unsatisfactory Overall Borrower Performance:



E. World Bank Staff

Positions At ICR At Approval Vice President: Makhtar Diop Gobind T. Nankani Country Director: Ousmane Diagana Mark D. Tomlinson Director (Regional Integration):

Colin Bruce -

Sector Director - Michel Wormser Sector Manager: Magda Lovei Mary Barton-Dock Project Team Leader: Dinesh Aryal Peter Kristensen/Denis Jordy ICR Team Leader: Dinesh Aryal ICR Primary Author: Ayala Peled Ben Ari



-



-



PDO rating before Closing/Inactive status Moderately Satisfactory

83

Revised GEO – Not Applicable (a) PDO Indicators – Original PDO Indicators as presented in the GEF Trust Fund Grant Agreement are assessed against achievements realized at project closing in December 2012.





Values


Years


Indicator 1: The inventory database of publicly held obsolete pesticide stocks is in place and being used by the Project Management Unit (PMU) and the recipient's staff

Value (quantitative or qualitative) No Yes - Yes

Date Achieved 04/19/2004 12/31/2012 - 12/1/2009


Target fully achieved (100%): An inventory was completed and validated in 2007, including transfer of data into the FAO obsolete pesticide inventory database. The database was subsequently used for preparation of a Country Environmental and Social Assessment (CESA), safeguarding and soil cleaning operations, and to prepare disposal bidding documents.

Indicator 2: Inventoried publicly held obsolete pesticide stocks have been disposed of, as a result of the completion of the disposal services contract(s) in accordance with national and international laws and regulations


0% 100% - 0%


Target not achieved (0%): The inventoried stocks were not disposed of due to a long disposal procurement process and the closing of the project before disposal activities

110 The PDO, indicators, and components that are stated in the Project Appraisal Document (PAD) slightly differ from those stated in the GEF Trust Fund Grant Agreement. This ICR bases its evaluation on the PDO, indicators, and components presented in the GEF Trust Fund Grant Agreement, which served as the principal guiding document for the project’s implementation activities.

F. Results Framework Analysis - At project approval, a country-specific results framework was not presented; only six PDO Indicators were defined. Project Development Objective (PDO) – as presented in the GEF Trust Fund Grant Agreement:110 To assist the recipient in: a) eliminating inventoried publicly held obsolete pesticide stocks and associated waste and preparing measures for the future decontamination of soils in priority sites; and b) implementing measures to reduce and prevent future risks related to pesticides and waste. Revised PDO – Not Applicable Global Environment Objectives (GEO) – as presented in the PAD (Report No: 36273-AFR): Reduction of the effects of Persistent Organic Pollutants (POPs) on the global environment.

84

commenced.

Indicator 3: A plan, in form and substance satisfactory to the World Bank, for the decontamination of soils in priority sites has been prepared by the recipient


Date Achieved 04/19/2004 12/31/2012 - 12/31/2012


Target fully achieved (100%): The project carried out nation-wide soil contamination surveys in July 2007 and September 2009, and prepared an assessment of five contaminated sites in May 2009. The PMU subsequently prepared site-specific Environmental Management Plans (EMPs) for the decontamination of five priority sites.

Indicator 4: The legal and regulatory framework for pesticide management has been improved and includes measures to strengthen compliance with the Basel Convention and the Rotterdam Convention


No Yes - Yes

Date Achieved 04/19/2004 12/31/2012 - 12/31/2012


Target fully achieved (100%): The project conducted a legal review of Mali’s legislative and regulatory framework for pesticide management in 2007, and provided technical support for its improvement to various government entities. Consequently, project objectives and activities were integrated into national policies for hazardous waste management.

Indicator 5: The training program under Part B111 of the project has been implemented, and the knowledge so acquired is being used by the PMU and the recipient's staff


No Yes - Yes

Date Achieved 04/19/2004 12/31/2012 - 12/31/2012


Target fully achieved (100%): An extensive training program was implemented throughout implementation on inventory taking, data maintenance, site decontamination and pesticides management, reaching approximately 400 governmental and non-governmental agents, as well as pesticide retailers.

Indicator 6: The PMU is functional, and its operation is satisfactory to the World Bank Value (quantitative or qualitative)

No Yes - Yes


Target partly achieved (80%): PMU was functional and operated satisfactorily albeit procurement weaknesses.

(b) GEO Indicator(s) – The GEO was measured by the PDO and PDO indicators.

111 See Section 1.5 for the project’s Component B (in the GEF Trust Fund Grant Agreement referred to as ‘Part B’).

85

(c) Intermediate Outcome Indicator(s) – Not Applicable



1 10/31/2005 Not applicable as ASP-P1 Mali not yet launched 2 06/30/2006

3 12/27/2006 4 06/28/2007 Satisfactory Moderately Satisfactory - 5 12/18/2007 Moderately Satisfactory Moderately Satisfactory - 6 05/30/2008 - - - 7 12/18/2008 Moderately Satisfactory Satisfactory - 8 03/25/2009 Moderately Satisfactory Satisfactory 1.06 9 12/29/2009 Moderately Satisfactory Moderately Satisfactory - 10 06/29/2010 Moderately Satisfactory Moderately Satisfactory - 11 03/27/2011 - - 1.14 12 12/25/2011 Moderately Satisfactory Satisfactory 2.04 13 06/29/2012 Moderately Satisfactory Satisfactory 2.32 12/31/2012 Final disbursement 2.44 H. Restructuring


Board Approved

PDO Change



in USD millions


PDO IP

04/01/2010 No Not Available

Not Available

Not Available

• Extension of closing date by 16 months (08/31/2010 to 12/31/2011) • Reallocation of proceeds to accommodate delays in the signature of the ASP-Mali MDTF agreement due to internal processing


Not Available

2.04 • Extension of closing date by 6 months (12/31/2011 to 06/30/2012)

07/02/2012 No Moderately Satisfactory

Satisfactory 2.32 • Extension of closing by 6 months

(06/30/ 2012 to 12/31/2012) I. Disbursement Profile – A disbursement profile is not available (see explanation under G).

G. Ratings of Project Performance in ISRs – Some DO and IP ratings and actual disbursement data are not available because ISRs reported on ASP-P1 as a whole, and only occasionally on country-specific project performance, including disbursements.

86



1. Country background: At project approval in December 2006, agriculture and livestock husbandry had long been the backbone of the economy, accounting (with related activities) for around 30-35% of GDP, as well as providing the bulk of export revenue until the advent of large-scale gold production in 2000.112 Besides its importance in the overall economy, agriculture with main crops such as cereals and cotton had been the principal source of livelihood for a large segment of the Malian population.

2. Sector background: The agricultural sector had therefore accounted for the largest fraction of pesticides used in Mali, with pesticide imports having increased substantially during the 1990s.113 The use of chemical pesticides had been a crucial factor in controlling pests and improving agricultural productivity. About 80% of the total agricultural pesticide use in Mali had been in the cotton sector; of which 95% were insecticides, distributed by the parastatal cotton development agencies and the Ministry of Agriculture. Within the health sector, the Ministry of Health had been responsible for the execution of public health pest control (i.e., mosquito and black fly control). From 1990 to 1992, about 630 tonnes of pesticides per year were imported annually for public health pest control, representing approximately 19% of total pesticide use. From 1993, data on pesticide use in the public health sector have become unavailable as a result of decentralization efforts and a shift in pest control responsibilities to the private sector.

3. Obsolete pesticides accumulation and soil contamination: Mali was considered a priority country to participate in ASP-P1 because of its large estimated volume of obsolete pesticides.114 Accumulation of obsolete pesticides in Mali had been caused mainly by product bans due to their effects on public health and the environment, deterioration as a result of prolonged or poor storage, and poor procurement planning/uncoordinated donations of pesticides. Mali also had a number of sites with highly polluted soil as a result of spills and poor storage conditions.115

4. Institutional context: Obsolete pesticides management is handled by the National Department for Sanitation and Pollution Control (Direction Nationale de l'Assainissement et du Contrôle des Pollutions et des Nuisances - DNACPN) within the Ministry of Environment and Sanitation. In 2002, a National Pesticides Management Committee (Comité National de Gestion des Pesticides - CNGP) was formed under government decision as the executive implementing body of existing pesticides management regulation. CNGP recently underwent review and resumed activities with support from the project.

5. Actions taken: From 2005 to 2006, and in the course of preparing the project, the Government of Mali carried out an inventory of obsolete pesticides, with support from the UN Food and Agriculture Organization (FAO). Close to 2,000 tonnes of the total inventory were considered having potential for

112 The Economist Intelligence Unit, Country Profile Mali, 2006 113 Volumes grew from 1,800 tonnes in 1991 to 4,100 tonnes in 1999 114 See ESMF Executive Summary (October 28, 2004). Pre-2005 estimates indicated that Mali possessed 500 tonnes of obsolete pesticides spread in 76 storage sites, as well as 40,000 tonnes of contaminated soil. 115 An African approach for Risk Reduction of Soil Contaminated by Obsolete Pesticides. Wageningen University and Research Centre, Wageningen (The Netherlands), 2009

87

dangerous environmental impacts, with the majority of the stocks having been inappropriately stored.116

6. Rationale for World Bank assistance: Please refer to Section 1.1 in the program-level ICR. 7. Higher-level objectives: The project intended to contribute to the third pillar of Mali’s Country

Assistance Strategy (CAS)117, which focused on developing basic infrastructure and productive sectors. It aimed to support the development of the agriculture sector by reducing inputs of pesticides, to promote the use of Integrated Pest Management (IPM) in the cotton sector, to strengthen natural resource management by supporting the enhancement of pesticide management, and the conservation of scarce water supplies through supporting the cleanup of sites where underground water had been contaminated by pesticide spillage. The project further aimed to assist the government in developing strategies for cleaning up soils that had been contaminated with pesticides. Contributing to international efforts to eliminate Persistent Organic Pollutants (POPs), improving management of toxic chemicals, and promoting alternatives to pesticide usage that include IPM and Integrated Vector Management (IVM) represented further objectives.

1.2 ORIGINAL PROJECT DEVELOPMENT OBJECTIVE AND INDICATORS

8. The Project Development Objective (PDO), and components that are stated in the Project Appraisal Document (PAD),118 slightly differ from those stated in the GEF Trust Fund Grant Agreement.119 This ICR bases its evaluation on the PDO, indicators, and components outlined in the GEF Trust Fund Grant Agreement, which served as the principal guiding document for the project’s implementation activities.

Original PDO as presented in the PAD

Original PDO as presented in the GEF Trust Fund Grant Agreement

To assist the Government of Mali to dispose of publicly held obsolete pesticide stocks at priority-selected sites, to support the cleanup preparation of contaminated soils of priority sites, and to support actions to help prevent the future accumulation of new stocks of obsolete pesticides

To assist the recipient in: a) eliminating inventoried publicly held obsolete pesticide stocks and associated waste and preparing measures for the future decontamination of soils in priority sites; and b) implementing measures to reduce and prevent future risks related to pesticides and waste.

9. At appraisal, only an overall ASP-P1 results framework had been prepared (see program-level ASP ICR); results frameworks for the individual ASP-P1 countries, including Mali, were intended to be finalized during the individual ASP-P1 country project launches. However, country-specific PDO indicators were developed at appraisal. For ASP-P1 Mali, these were as follows:

116 Particularly the contaminated site Nanguila (Kati) in the Koulikoro region was identified as posing high environmental risks. 117 Country Assistance Strategy for the Republic of Mali (Report No. 25663), July 7, 2003 118 Project Appraisal Document for the Republic of Mali, the Kingdom of Morocco, and the United Republic of Tanzania in support of the Africa Stockpile Program – Technical Annex 1, (Report No. 36273-AFR), November 7, 2006. 119 GEF Trust Fund Grant Agreement between the Republic of Mali and the International Bank for Reconstruction and Development, (Number TF057678 – MLI), December 26, 2006

Table 1.1: ASP-P1 Mali Original PDO

88

Original PDO Indicators as presented in the PAD and the GEF Trust Fund Grant Agreement

a. The inventory database of publicly held obsolete pesticide stocks is in place and being used by the Project Management Unit's (PMU) and the recipient's government staff.

b. Inventoried publicly held obsolete pesticide stocks have been disposed of, as a result of the completion of the disposal services contract(s) in accordance with national and international laws and regulations.

c. A plan, in form and substance satisfactory to the World Bank, for the decontamination of soils in priority sites has been prepared by the recipient.

d. The legal and regulatory framework for pesticide management has been improved and includes measures to strengthen compliance with the Basel Convention and the Rotterdam Convention.

e. The training program under Part C of the project has been implemented, and the knowledge so acquired is being used by the PMU and the recipient's government staff.

f. The PMU is functional, and its operation is satisfactory to the World Bank.

1.3 REVISED PROJECT DEVELOPMENT OBJECTIVE AND INDICATORS

10. Similar to ASP-P1 Ethiopia, ASP-P1South Africa, and Tunisia, ASP-P1 Mali was planned to undergo a Level 1 Restructuring to ensure the PDO and associated indicators were achievable and measurable. However, the restructuring of ASP-P1 Mali did not materialize due to delays in the processing of an additional GEF Grant for the project and the March 2012 coup d'état, which resulted in the temporary suspension of World Bank operations in the country in accordance with OP/BP 7.30 (Dealing with De Facto Governments).




12. The PDO of ASP-P1 Mali was to be achieved through the implementation of three distinct components:

Original Components as presented in PAD


Component A – Country cleanup and disposal activities (Total: US$5.38 million; GEF: US$2.31 million): The objective of this component was to provide an inventory of obsolete pesticides, a risk assessment, a country environmental and social assessment (CESA), a minor upgrade of collection sites and the disposal of the obsolete pesticides. Main activities included: a. Completing the detailed inventory and risk assessment of

freestanding obsolete pesticides; b. Preparing CESA including an environmental management

plan (EMP); c. Preparing the disposal strategy, safeguarding obsolete

pesticides and undertaking the actual disposal of the

Part A – Cleaning and elimination of obsolete pesticides and associated waste a. Carrying out a detailed inventory of the recipient’s

publicly held obsolete pesticide stocks and associated waste through: a) provision of relevant training to the recipient; b) collection of data and compilation of a data base; c) a risk-based prioritization of the stocks and associated waste identified in the database; and d) preparation of the CESA.

b. Disposal of obsolete pesticides and associated waste through a) carrying out of a technological assessment of available treatment and disposal options; b) implementation of the treatment and/or disposal

Table 1.3: ASP-P1 Mali Original Components

Table 1.2: ASP-P1 Mali Original PDO Indicators

89

freestanding obsolete pesticides; d. Container assessment and disposal; e. Buried pesticides assessment and disposal; and f. Contaminated soil assessment and disposal.

technology selected; c) improving the operations of obsolete pesticide stocks collection centers, including improving safety measures at those centers.

Component B - Prevention of obsolete pesticides accumulation (Total: US$1.77 million; GEF: US$0.24 million): a. Strengthening the government’s capacity to manage

pesticides stocks through the design and the implementation of a program preventing the future accumulation of pesticides;

b. Promoting ongoing IPM and IVM efforts, particularly with small-scale farmers in sectors such as cotton.

c. Strengthening small-scale farmers’ capacity to use and manage pesticides;

d. Raising awareness on general pesticide issues using an Non-Governmental Organizations (NGOs) to support the program;

e. Developing a container management strategy; and f. Strengthening laboratory services.

Part B – Prevention of obsolete pesticide accumulation: a. Building capacity through the provision of training to

farmers, civil society groups and relevant staff of the recipient;


c. Promotion of improved pesticide management practices such as integrated pest management and integrated vector management, and improved management of empty pesticide containers through the design and implementation of container management strategy;

d. Awareness raising activities on the program, and the health and environmental hazards of pesticides, including carrying out of workshops on reducing reliance on pesticides in agricultural production, disease vector control and migratory pest control;

e. Development of a long-term funding strategy for prevention activities and a plan to prevent pesticide accumulation; and

f. Strengthening the recipient’s laboratory services.

Component C - Project management (Total: US$1.18 million; GEF US$0): a. Project coordinator position primarily financed by the

Government; b. Pesticide management expert including for Monitoring

and Evaluation (M&E). This position was planned to support the disposal operations. Therefore, the M&E function was included in its scope;

c. Prevention and communication staff to support the prevention component and awareness raising; and

d. Finance, procurement and administration staff to support the overall project.

Part C – Project management: Supporting the PMU in carrying out project coordination, implementation, monitoring and evaluation, through the provision of technical advisory services, goods and operational support.

90



Extension of closing date Extension of the closing date of the GEF Trust Fund Agreement by 16 months from 08/31/2010 to 12/31/2011.

04/01/2010 To compensate for start-up delays, and allow completion of the disposal contract, taking into account the potentially long lag time between obsolete pesticides being exported from the country and the date of actual incineration (up to 12 months).

Country Director

Extension of the closing date of the Multi-Donor Trust Fund (MDTF) child TF (TF096815) by 20 months from 04/30/2011 to 12/ 31/2012.

04/29/2011 Extension of the parent ASP MDTF as part of the overall restructuring of ASP-P1 projects.

Country Director

Extension of the closing date of the GEF Trust Fund Agreement by six months from 12/31/2011 to 06/30/2012.

12/12/2011 To allow time to finalize all remaining project activities by utilizing the project funds and the proposed additional GEF financing.

Country Director

Extension of the closing date of the GEF Trust Fund Agreement by six months from 06/30/2012 to 12/31/2012.

07/02/2012 a) Continued implementation delays, especially due to the needed preparation of a Resettlement Action Plan (RAP) and the long disposal procurement process; and b) the March 22, 2001 coup in Mali resulted in suspension of all operations in Mali in accordance with OP/BP 7.30. The extension was granted to allow for the possible resumption of operations after an assessment mission scheduled for June 2012.


Reallocation of proceeds Reallocation of proceeds from the categories Disposal Services (-$600,000), and unallocated (-$240,000) to categories Goods (+$180,000), Works (+$20,000), Consultant Services including Audits (+$125,000), Training and Workshops (+$150,000) and Operating Costs (+$365,000).

04/01/2010 Delays in the signature of the ASP-P1 Mali MDTF agreement due to internal processing resulted in the additional use of the GEF Trust Fund, causing an overdraw in almost all categories. The reallocation fixed the overdraw. The amount deducted from Disposal Services was later added to the Disposal category in the MDTF agreement.

Country Director

Table 1.5: ASP-P1 Mali Other Significant Changes

91



13. ASP-P1 Mali was prepared between 2001 and 2006, as part of preparatory activities at the program-level. It took more than five years from GEF eligibility in July 2001, and more than four years from concept note in July 2002 to project approval by the Board in December 2006.120

14. Soundness of the background analysis: Project preparation was conducted in close collaboration between the government and the ASP partners, and it resulted in intputs of varying levels of quality. The government, the World Bank, FAO, CropLife International (CLI), French Global Environment Facility (FFEM), World Wildlife Fund (WWF) and Pesticides Action Network (PAN)-UK collaborated via joint missions and workshops, where each partner carried out a distinct role.121 National stakeholders became involved around appraisal (2004), when a broad National Steering Committee (NSC) was created under a ministerial decree.122 The key preparatory outputs were the PAD and the Operational Manual: the PAD contained mostly ASP-P1 level information and plans with limited amount of country-specific information. Exception to this was the additional focus on contaminated soil, which had been recognized by all parties involved as a serious problem needing special attention. The Operational Manual, on the other hand, included considerable amount of background detail and plans specific to Mali, including a stakeholder analysis, an M&E framework, an analysis of the national legislative framework, environmental impact assessment procedures, an analysis of risks, and terms of reference for key tasks and budgets. There was a general agreement amongst the partners that the 2005-2006 inventory data might be inaccurate, and that the stocks of obsolete pesticides could range anywhere between 400 and 800 tonnes. Similarly, the lack of reliable data led to the estimate that soil treatment costs could range anywhere from US$2-6 million. Therefore, it was agreed that the inventory data would be validated early in implementation.

15. Assessment of project design: The project’s design (as for all ASP-P1 participating countries) was preconfigured at the program level, and did not reflect any country-specific modifications or lessons learned from pervious projects executed in Mali. Design weaknesses at the program-level that translated into implementation delays at the country level, including ASP-P1 Mali, are outlined in Section 2.1 of the program-level ICR. In the case of Mali, it was also agreed123 that additional attention would be given to the issue of contaminated soil as a result of pesticide spills and leaks. Consequently, ASP-P1’s generic PDO was slightly modified to include the preparation of measures for future decontamination of soils, and treatment of contaminated soil was included under component A (Country Cleanup and Disposal Activities). Studies were to be undertaken to assess the scale of soil contamination, identify the adequate technologies, and estimate the costs of treatment of the soils.

120 The long preparation time resulted from a lengthy partnership building/negotiation process at the program-level resulting from the complex implementation arrangement and project design (see Section 2.1 in the program-level ICR). 121 During preparation, FAO provided technical support to the PMU and sponsored the temporary hiring of a national expert in obsolete pesticides; WWF and PAN-UK conducted workshops for journalists and francophone NGOs, respectively; and the World Bank Task Team focused its support on institutional arrangements, safeguard requirements, risks analysis, budgeting and fund raising. CLI and FFEM participated in technical and budgetary discussions. 122 Decree No. 04 /1516 of August 2, 2004, modified on September 8, 2006. 123 See for example, the PDF-B completion report which summarized key country-specific points of focus.

92

16. Government commitment: DNACPN’s strong commitment at entry was evident from its engagement in the inventory of free-standing obsolete pesticides with the support of FAO and the interest in continuing pesticide cleanup activities. In 2004, the government established a PMU within the Ministry of Environment and Sanitation, and the NSC, which included broad representation of all key stakeholders. Another evidence of the government’s strong commitment was the concern it expressed (as reflected in the various mission reports) about the lengthy preparation time, especially in view of risks posed by some contaminated sites. The government committed to co-financing corresponding to standard disbursement percentages (SDP) and to financing two technical project staff.

17. Assessment of risks: The PAD and the Operational Manual presented brief and partial risk assessments, which focused mainly on crosscutting and program-level risks. The PAD included one risk specific to Mali, which was related to the government’s lack of capacity or experience in treating contaminated sites or soils. As mitigation, the PAD proposed to outsource these activities to professional contractors. The Operational Manual identified risks emanating from the health impact of pesticide handing, accidents during transport and public opposition to the project. Corresponding mitigation measures were proposed, which correctly highlighted the need to conduct accurate inventories, hire external expertise and continuously seek additional funding in case additional stocks were revealed. Neither the PAD nor the Operational Manual included risks of delays due to the sequential design of country projects and political instability. Appropriate mitigation of the sequential design could have significantly increased the project’s success, while the country’s political situation was outside the scope of the project.

2.2 IMPLEMENTATION

18. The project experienced long delays, mainly in the procurement of a disposal company. Positively affecting project implementation were government's strong commitment to the project and the recruitment of an experienced Technical Advisor for Disposal (TAD) by CLI in mid-2009.124

19. Key factors affecting implementation included: • Long disposal procurement process. The procurement of a disposal company took more than

three years to complete due to: (i) low procurement capacity within DNACPN, which meant that some steps took longer to complete or had to be repeated; (ii) lack of clarity within the World Bank about the disposal procurement process (see Section 2.2 of the program-level ICR); and (iii) the long time it took to finalize the CESA, which had to be completed before the scale of interventions could be determined and shared with potential bidders.

• A disposal-funding gap and delays in securing additional financing. Disposal activities were allocated a budget despite the uncertainty around the amount of stocks to be disposed of and the cost of soil treatment. An inventory validation exercise carried out in June 2007 found higher obsolete pesticide stocks (1,100 tonnes), indicating a disposal budget gap. While the exact scale of the gap was never fully identified by either the World Bank or the project, it was clear to all early in implementation that additional funds were needed in order to dispose of all stocks and

124 It should be noted, however, that the TAD was contracted with substantial delays (the ToRs were approved in September 2007 but the contract was signed in mid-2009) due to long discussions between CLI and the government about contracting procedures and mobilization of the funding within CLI. Given the TAD’s positive impact on the disposal contracting and PMU capacity, it is likely that an earlier recruitment could have shortened the disposal contracting period.

93

treat polluted soil. Additional financing was secured in March 2010 when the GEF Council approved a $3.19 million grant to ASP-P1 Mali, but this was only approved by the CEO in March 2012. Project/mission reports and discussions with World Bank task team members highlight the following reasons for this substantial delay:

o Unsuccessful initial fund raising efforts. Until additional funds were sought from the GEF, the project was unsuccessful in securing any funds. In fact, project and mission reports do not confirm whether fund raising efforts were made.

o Delayed submission of additional financing proposal to GEF. Following the February 2009 mid-term review and recognition that additional resources could not be secured by the government, the World Bank and the government decided to pursue additional financing from GEF. However, more than two and a half years had passed before the additional financing was approved (in March 2012).125 This was mainly the result of a World Bank TTL change, as well as the World Bank’s decision to merge the additional financing request with project restructuring and government’s reluctance to restructure the project.126

• Temporary unavailability of the MDTF funds. The US$3.98 million MDTF closed in July 2009 (with an undisbursed balance of US$3.1 million) due to delayed donor agreement on an extension and amendment of the parent MDTF agreement. The process of establishing a new MDTF with the balance took 19 months to complete (on February 1, 2011). This delay was the result of internal processing delays, resulting, according to the PMU, in the delay of some project activities. It is unclear, however, to what extent this delay affected the project since the GEF funds were used during this period as backup,127 and the disposal contract was only signed in October 2011, eight months after the new MDTF became available.

• The March 2012 coup d'état and suspension of activities: The Additional Financing/Level I Restructuring Paper was submitted for management decision in March 2012, days before the military coup, which halted all operations in the country.

2.3 MONITORING AND EVALUATION (M&E) DESIGN, IMPLEMENTATION, AND UTLIZATION

20. Overall M&E - Rating Moderately Unsatisfactory based on the following evaluation. 21. M&E design - Rating Unsatisfactory: The project’s M&E framework lacked the most important

element – a project-level results framework. Although the appraisal mission developed a project-level results framework128, it was not reflected in the PAD and the Operational Manual, both presenting only the overall ASP-P1 results framework (see program-level ASP ICR) and six country-specific

125 The additional financing was titled “Africa Stockpiles Programme - Project 1 – Supplemental Funds for Disposal and Prevention” (P118630). The Project Identification Form (PIF) included both Mali and Tunisia, while the CEO Memo included Mali alone after Tunisia had decided to withdraw from the request. 126 Discussion with the task team indicate the following two key reasons: (i) concerns about public criticism, and (ii) the Technical Support Unit’s (TSU’s) conflicting opinion that a restructuring was not necessary if some disposal tasks were carried by local staff, an option which was considered too risky and unacceptable to the World Bank. 127 In April 2010, the World Bank authorized a reallocation of GEF proceeds between categories in order to facilitate this. 128 See Africa Stockpiles Programme (ASP) Mali Country Project -Appraisal mission Aide Memoire (April 19 to 27, 2004), July 1, 2004

94

Performance Monitoring Indicators. These indicators did not permit adequate monitoring of project implementation progress because they focused on measuring outputs as opposed to progress towards project outcomes. Expected results under the prevention component, in particular, were not adequately considered by the indicators.

22. M&E implementation - Rating Moderately Satisfactory: The project was implemented without a country-specific results framework until the mid-term-review in February 2009, more than three years after effectiveness. Discussions with the PMU and the World Bank team indicate that the delays were caused by delayed recruitment of an M&E consultant and the dissatisfaction of the World Bank with the consultancy’s results. The PMU with assistance from a World Bank M&E specialist eventually developed an M&E manual and a project-specific results framework based on FAO guidelines and in agreement with World Bank instructions. The PMU was effective in collecting data and producing satisfactory progress reports. Performance evaluation was carried out internally by the disposal specialist who was also in charge of M&E, while external evaluation was carried by external consultants and an FFEM-assigned committee according to specific terms of reference.129

23. M&E utilization - Rating Satisfactory: The inventory validation exercise and the national database that was created using this information were the key mechanisms for risk evaluation, site cleanup prioritization and resource allocation. The inventory data were also used to assess the disposal funding gap and dictated which safeguard instruments should be prepared and followed by the project. In order to enhance the usefulness of the national database, the project provided training on data management to key holders of pesticide stocks. Since the database was, and still is, hosted by FAO’s online Pesticide Stock Management System (PSMS), the government received training on data management offline to be periodically up-loaded back to FAO’s server.


24. Safeguard compliance - Rating Satisfactory: ASP-P1 triggered two safeguard policies: OP 4.01 on Environmental Assessment and OP 4.09 on Pest Management.

25. ASP-P1 Mali complied with OP 4.01 on Environmental Assessment: The CESA was completed in December 2009 and disclosed in the World Bank InfoShop on April 8, 2010, more than three years after effectiveness. The long process was due to weak procurement capacity, lack of clarity of the PMU and the consultant on the scope of work, poor quality of the first drafts and a long authorization process within the World Bank. Stakeholders and concerned communities were consulted twice during the preparation of the CESA. The report outlined mitigation measures for the safe handling of hazardous substances during preparatory cleanup works, repackaging, intermediary storage, transport, and site rehabilitation.130 The CESA highlighted the need to prepare separate EMPs for a number of highly contaminated sites and a RAP for one site where residents had to be temporarily relocated. While the CESA was never implemented (the disposal work had not commenced by project closing), the site-specific EMPs were implemented satisfactorily during site cleanup operations, as confirmed by World Bank reports prepared by a World Bank-hired safeguard specialist.

129 The committee comprised representatives from MEDD France, French Ministry of Foreign Affairs, French Development Agency, French Ministry of Agriculture and Forestry, PAN-Europe and Union of Plant Protection Industries. It oversaw FFEM-funded activities. Agricultural Research for Development and two disposal companies (SYPRED and ADIVALOR) provided technical expertise to the committee. 130 The CESA did not include mitigation measures for the final disposal since incineration was to take place in an overseas licensed facility, and therefore no impact was foreseen in Mali.

95

26. OP 4.09 on Pest Management: Since project activities themselves met requirements of a pest management plan (i.e., through pesticide management, Integrated Pest Management (IPM), and capacity building), individual country projects, including ASP-P1 Mali, were not required to prepare self-standing documents. See Section 2.4 in the program-level for more details.

27. ASP-P1 Mali complied with OP 4.12 on Involuntary Resettlement to a satisfactory extent: A RAP was prepared for planned safeguarding and soil remediation works in Nanguila (Koulikoro region), which involved the temporary relocation of inhabitants during the cleanup operation. The RAP was approved, disclosed in November 2010, and subsequently implemented in full compliance with safeguard requirements, as confirmed by various World Bank implementation support missions.

28. Financial Management compliance - Rating Satisfactory: The project had in place appropriate financial management staffing131 and software, and accounts were maintained properly. The project submitted Interim unaudited Financial Report (IFRs) and independent audits in a timely and overall satisfactory manner. Any ineligible expenditure was reconciled. Highlighted weaknesses were unclear distribution of financial management tasks between the Administrative and Finance Officer and the Accountant, and some deficiencies in record keeping, internal control system management and fixed asset management. The project undertook the necessary corrective measures. GEF disbursement was low during the first half of the project (14% at mid-term compared to a 66% disbursement estimate), but increased substantially during the second half of the project (95% disbursement at closing). The low disbursement rate at mid-term is mainly attributable to the project's design having been back loaded in terms of expenditures with the more costly disposal activities (over 66% of the GEF grant budget) programmed to take place only at the end of the project.

29. Procurement - Rating Unsatisfactory: Procurement undertaken by DNACPN suffered from substantial delays, particularly in the purchasing of personal protective equipment for the disposal and cleanup operations, selecting a consulting firm to conduct the CESA and selecting a disposal company. The delays were attributable to a number of factors, the main one being DNACPN’s weak procurement capacity and the lack of a dedicated procurement staff132, combined with unclear guidance from the World Bank. The PMU also used a procurement plan, which had not been approved by the World Bank, and finalized contracts without the World Bank’s prior to approval. World Bank suggestions for increasing the procurement capacity of the PMU, by either hiring a procurement officer or providing training to other PMU staff, were not accepted by the PMU, highlighting the lack of qualified specialists in the country, and the already overloaded work program of the PMU members. Procurement received some reinforcement in July 2011 when the Africa Emergency Locus Project’s (AELP’s) procurement specialist began providing cross-support to the project under a coordination agreement between the two projects.

131 With the exception of a temporary absence of the administrative and financial manager (between March and June 2010) due to personnel exchange 132 Early in implementation the dual functioning of financial management and procurement was judged by the World Bank to be problematic because it both slowed down procurement and constituted a conflict of interest within the PMU where purchasing and payments were carried out by the same person. The World Bank repeatedly recommended hiring a procurement specialist but none was hired due to lack of qualified candidates in Mali.

96


30. Post-completion: During implementation, inventoried low risk stocks at the various sites were repacked in metal containers and centralized in a store 35 km outside Bamako by trained local staff. Higher risk stocks have remained untreated since the 2005-2006 inventory in more than ten stores across the country, including two stores in Gao and one store in Timbuktu. The condition of the stocks is overall unknown; however, discussions with the PMU indicate that it is likely that some stores are in poor condition, leaking and accessible by the public. Despite this situation, no post completion activities have taken place since project closing.

31. Next phase: Discussions between the World Bank and the GEF Secretariat confirmed that the approved US$3.19 million additional financing could be used in a follow-up disposal and prevention project in Mali, and a letter from government dated March 20, 2013 confirmed the government’s interest in pursuing this option. In April 2013, the World Bank conducted an identification mission, where it also sought additional sources of financing. The World Bank task team confirmed that if a follow-up project would be launched, most of the proceeds would be used to pay for the disposal and/or safeguarding of the 1,100 tonnes of obsolete pesticide stocks and treatment of sites with highly contaminated soil.

32. DNACPN had offered to create a permanent ASP unit, which would be responsible for updating the national obsolete pesticide and waste inventory, prepare for the disposal operation, oversee decontamination of polluted sites, and conduct various prevention activities. It is unclear at this point whether this unit will be formed by the government.



33. Relevance – Rating Substantial: Pesticide management, and obsolete pesticide and site cleanup were and continue to be relevant to the country's current transitional and development priorities, but are not key priorities. The Plan for the Sustainable Recovery of Mali for 2013-2014 (May 2013) only includes a pesticide cleanup operation in Gao.133 The Plan is part of the country’s Transitional Roadmap of January 29th 2013 and the Strategic Framework for Growth and Poverty Development 2012-2017 of December 2011. Obsolete pesticide management is relevant to the Interim Strategy Note (ISN) for Mali for the period FY14-15,134 and its objective of increasing agriculture performance and improving resilience to weather-related shocks as a measure for resuming economic activity. The project supports the Government of Mali in complying with the Stockholm Convention on POPs, the Basel Convention on Control of Trans-boundary Movements of Hazardous Waste and their Disposal, and the Rotterdam Convention on the Prior Informed Consent Procedure for Certain Hazardous Chemicals and Pesticides in International Trade.


34. The project has been evaluated against its PDO by measuring achievements at the time of the project closing (December 31, 2012).

133 The document is presented in the IMF Poverty Reduction Strategy Paper (PRSP), (Country Report No. 13/111). 134 Report No. 76233-ML, May 20, 2013

97

35. PDO achievement – Rating Moderately Unsatisfactory: The assessment took the project’s six PDO indicators into account, of which four were fully achieved (100%), one was partly achieved (80%) and one was not achieved (0%). Indicator #2 (disposal of the stocks), which signifies the most important project achievement, received a higher weight in the rating calculation. The three main project outcomes measured by indicators include: 135 (i) Sound disposal and management of obsolete pesticides and contaminated sites; (ii) prevention of accumulation of new stockpiles of obsolete pesticides; and (iii) enhanced capacity and institutional strengthening of pesticide management.

36. Sound disposal and management of obsolete pesticides and contaminated sites. The level of risks emanating from publicly held obsolete pesticides and associated waste to adjacent and wider communities or critical natural resources was determined and quantified through the completion of a national inventory of publicly held obsolete pesticides in 2005-2006 and its validation in June 2007 (PDO Indicator #1 – 100%). This information was used to prepare a CESA, conduct an assessment of contaminated sites (PDO Indicator #3 – 100%), safeguard low risk stocks, clean up a number of priority sites and contract a disposal company. It is estimated that over four million people (half of them are women) living in vicinity to the contaminated sites benefitted from soil cleanup operations and obsolete pesticide safeguarding activities.136 Inventoried stocks were not disposed of under this contract due to a very long disposal procurement process and the closing of the project before the disposal work commenced (PDO Indicator #2 - 0%).

37. Prevention of accumulation of new stockpiles of obsolete pesticides. The project helped improve the legal and regulatory framework for pesticide management (PDO Indicator #4 - 100%) by conducting a legal review of Mali's legislative and regulatory framework for pesticide management, in 2007, followed by technical support for its improvement based on the results. The project supported the National Directorate of Agriculture in the development of procedures of obtaining approval for the importation/export and the sale of pesticides. It also supported DNACPN in the development of laws for empty container management and management of seized pesticides. Project objectives and activities were integrated into national policies for hazardous waste management (such as the Politique Nationale d'Assainissement, endorsed by the Council of Ministers in 2009), where an entire chapter was devoted to obsolete pesticides management at both the public and private sectors. Awareness raising and sensitization efforts were made to educate relevant governmental agents about the introduced changes. New accumulations of associated waste were addressed by the project by developing and implementing an empty container strategy in 29 rural communities from 2009 to 2012 in the cotton zones of the Malian Cotton Company (Compagnie Malienne pour le Développement des Textiles - CMDT). The project also increased government and public awareness with respect to the harmful effects of pesticide mishandling and contact with obsolete pesticides. It developed, validated and implemented a communication plan, which included various communication tools, such as films, radio, newspapers, as well as through workshops for NGOs. It is estimated that over 1.4 million people were affected by project communication, either via face-to-face workshops or through radio programs.

135 Achievements were identified based on intended outcomes presented in Section B.1 and Annex I of the PAD (Report No: 36273-AFR) 136 Source : Nombre de bénéficiaires du PASP-Mali (2007 à 2012), ASP-Mali Project Management Unit, 4/29/2013

98

38. Enhanced capacity and institutional strengthening of pesticide management. A total of 4,451 governmental and non-governmental staff received training on various topics related to pesticide and waste management, communication and M&E (Original PDO Indicator #5 - 100%). These skills were used to validate the pesticide inventory, enter the data into the PSMS data base, and participate in successful safeguarding and site cleanup operations. The project also invested resources in reactivating the CNGP, the executive pesticides regulating body; however, it is unclear whether this strengthened the institutional settings for pesticide management.


39. The project aimed to directly contribute to the GEO of reducing the effects of POPs on the global environment through: (i) improving the quality of life in poor communities, by reducing environmental risks; (ii) improving environmental protection; and (iii) enhancing the capacity of the agricultural sector to better manage crop pests. GEO achievement is measured by PDO achievement and, thus, is also rated moderately unsatisfactory (see Section 3.2).

3.4 EFFICIENCY

40. Efficiency – Rating Low: This rating represents a broad estimate, as financial and economic data to support this rating are scarce. At appraisal, an economic or financial analysis was neither undertaken for ASP-P1 at the program level nor at the country level due to non-measurable benefits of reducing the risk of contaminating the environment. A lack of reliable baseline data in combination with controversial aspects related to the valuation of human life and natural resources did not permit a cost-benefit analysis. In addition, efficiency of capacity building and prevention activities is difficult to quantify due to the temporal disconnect between project support and actual results. For GEF purposes, the ASP-P1 underwent an incremental cost analysis to demonstrate the incremental benefit associated with the participation of GEF in the program. The analysis demonstrated that GEF support would lead to the channeling of considerable additional resources to the program ($35 million in addition to the $25 million from GEF), as well as important domestic (safeguarding local populations and environments and having in place meaningful prevention mechanisms) and global benefits (removal of trans-boundary pollution threats).

41. Although 77% of the entire project budget was disbursed at the end of the project, it was not used to dispose of any obsolete pesticide stocks, demonstrating low project disposal efficiency. In view of safeguarding and site cleanup activities, the project was successful in decontaminated five polluted sites using low cost techniques (e.g., land farming) and employing trained local staff, as well as collecting low risk stocks from numerous sites, also by local staff at relatively low cost. . These cost savings, coupled with the fact that the project achieved substantial prevention and capacity building results, alleviate the overall project efficiency rating.


42. Overall Outcome - Rating Moderately Unsatisfactory: On the basis of the project's efficiency (low), relevance (substantial), and PDO achievements (moderately unsatisfactory), the overall outcome is rated moderately unsatisfactory. The overall outcome rating takes into account that even though the key result – disposal - did not take place, important environmental and health impacts expected of the project in view of POPs-related illnesses which are likely to have been prevented in some of the most

99

polluted sites (through safeguarding and soil clean up). Prevention (improved legislation, awareness raising and improved container management practices) and capacity building activities conducted by the project have the potential of sustaining these achievements.


43. Poverty impacts, gender aspects, and social development: The safeguarding and site cleanup operations had potentially a direct impact on nearby poor and farming communities, species and ecosystems. Since baseline data on affected populations, species and ecosystems was not conducted at appraisal, exact health and environmental impacts associated with the removal of POPs from the environment, are difficult to measure. However, a project beneficiaries report calculated that close to 1.5 million people were reached by the project’s communication and awareness campaigns, and that over 4 million people live in vicinity to the contaminated sites. These people’s health is likely to be positively affected by the removal of POPs from their environments (water, soil, crops, fish, livestock, etc.) through the site cleanup and safeguarding activities.137

44. Institutional change/strengthening: The project strengthened the government’s institutional capacity on several levels: (i) the regulatory and legal framework for pesticide management was strengthened; (ii) the government gained experience in the preparation of social and environmental impact assessments and management plans; (iii) the experience and skills gained in carrying out inventory, creating and utilizing a database, participating in safeguarding and soil remediation activities, and managing a multi-stakeholder project, can be applied to other projects and subsequently contribute to success; and (iv) knowledge gained on sound pesticide management through the various training and courses, which were provided to governmental and non-governmental staff (the beneficiary report calculated a total of 4,451 governmental and non-governmental staff receiving training on various topics related to pesticide and waste management, communication and M&E).

45. Other unintended outcomes and impacts: ASP-P1 Mali had an additional focus on soil treatment due to a number of sites known for having been polluted by pesticide spills and leaks. This provided an opportunity for the program to explore the issue of soil remediation and offer solutions by conducting assessments and carrying out pilot cleanup operations. In July 2008, a site in the town of Molodo, which contained large quantities of highly toxic obsolete pesticides contaminating local ground water supplies, was successfully cleaned up using a 'landfarming' technique for the first time in the country.138 The operation was carried out with strong support from the TSU and Wageningen University. The results of the Molodo operation and similar operations in Dialakoroba, Niogoméra, Sévaré, Gao and Nara were analyzed and disseminated to other ASP countries in a workshop organized by FAO in Mali in 2010 and in 2009 at an international soil decontamination conference in Baltimore, Maryland.

137 Source : Nombre de bénéficiaires du PASP-Mali (2007 à 2012), ASP-Mali Project Management Unit, April 29, 2013 138 Land farming is a soil decontamination technique which involves taking polluted soil and encouraging the biodegradation activity of soil microbes

100


46. A specific completion, and/or ICR mission was not carried out, and thus no stakeholder workshop was held. The project submitted a draft completion report in March 2013,139 which according to the PMU and the World Bank team, had not been shared with national stakeholders. The report identified the following key factors as having a negative impact on project implementation: (i) delays in mobilizing the MDTF, FFEM and additional GEF funds, resulting in implementation delays and a disposal funding gap; (ii) inability of the project to disburse funds due to a three months absence (March to June 2010) of a financial manager to sign transactions; and (iii) complex World Bank safeguards and procurement procedures.


47. Risk to the development outcome - Rating High: Given Mali's post-conflict and transitional situation and the relatively low priority obsolete stocks cleanup is receiving in the country's recovery plans, it is unlikely that the obsolete pesticide stocks will be disposed of by the country without a donor-funded disposal project. Furthermore, Mali has a number of identified polluted sites which need cleaning up, but the government does not have the budget or skills to carry out such operations independently. In terms of prevention of new accumulations of obsolete pesticide stocks, the improved legislation and regulation is unlikely to have significant on-the-ground effect unless the government dedicates adequate resources to enforce them, which is also questionable. Mali's strong focus on agricultural intensification as a measure for economic development is likely to overpower any such efforts in the long run.



48. World Bank performance in ensuring quality at entry - Rating Moderately Unsatisfactory: While World Bank performance during preparation was adequate, program-related factors negatively affected quality at entry. Considerable efforts were invested by the World Bank in preparing the project, which involved agreements on project scope, budget, implementation arrangements, results, M&E arrangements, project work plans, FM and procurement. The World Bank also invested considerable efforts in securing project co-financing by meeting with potential donors and seeking collaboration with ongoing projects. Nonetheless, the preparation of the project was conducted in conjunction with preparatory activities at the program-level, which were lengthy (see Section 2.1 in the program-level ICR). In addition, while some of the fund raising efforts were successful, the delays during preparation resulted in some co-financing becoming unavailable.140

49. Quality of World Bank supervision - Rating Moderately Unsatisfactory: The project was supervised by the program co-TTL responsible for francophone countries (Mali, Morocco and Tunisia) who was

139 MEA/DNACPN’s draft progress report for the period June 2006-December 2012 140 Funds were secured from the Netherlands, FFEM and MDTF. Co-financing from PASAOP was no longer available since the project had closed at the time ASP-P1 Mali was approved (on December 31, 2006).

101

based in Washington DC and later in Dakar, and a Mali-based co-TTL. The two co-TTLs ensured knowledge exchange between Mali and other countries on one hand, and contact with stakeholders on the other. Supervision missions by both TTLs took place regularly, albeit only once a year, with additional technical support and fiduciary missions in between. Aide Memoires were detailed, providing candid description of issues and risks. However, significant delays by the World Bank in the processing of the restructuring/additional GEF financing and MDTF funds as explained above and the unclear guidance on the disposal procurement process negatively affected project performance.

50. Overall World Bank performance – Rating Moderately Unsatisfactory: Based on the above-presented evaluation, the overall borrower performance is rated moderately unsatisfactory.


51. Government performance - Rating Satisfactory: The government of Mali continued to show commitment to the project and supported it throughout implementation. Commitment was evident from a) the integration of project objectives and activities in national policies; b) the timely and adequate provision of counterpart financing; and c) the close collaboration between the PMU and government officials during implementation, which facilitated overall project management. A wide variety of government representatives participated in the NSC, the CNGP, and thematic groups created for the project141, training sessions and safeguarding operations (e.g., a soil contamination forum). The government initial reluctance to restructure the project contributed to the delays in obtaining the additional financing. However, this was not the result of lack of commitment to the project but rather from an aspiration to accomplish the original, more ambitions PDO.

52. Implementing Agency performance - Rating Moderately Satisfactory: DNACPN demonstrated high level of performance in the areas of stakeholder mobilization, coordination of activities, awareness raising, safeguard implementation and FM. M&E was properly conducted and progress reporting was done in a satisfactory manner. DNACPN's performance was hampered by weak procurement capacity and the refusal to receive offered World Bank support, as well as the low efficiency in the use of resources. DNACPN complied with all World Bank legal covenants, and executed its fiduciary duties.

53. Overall Borrower performance – Rating Moderately Satisfactory: Based on the above-presented evaluation, the overall borrower performance is rated moderately satisfactory.

6. LESSONS LEARNED

54. The most significant lessons learned are drawn at the program-level, as the experience they are based on was often reflected across all country-specific projects; they are therefore discussed in the program-level ICR. Lessons learned that were particularly relevant to ASP-P1 Mali include:142

141 Such as the soil decontamination group, which comprised eleven focal points of the main relevant institutions including DNACPN, Crop Protection Office, National Center for Locust Control, Health Directorate, Food Safety Agency, the Air Force, Laboratory of Toxicology and Environmental Quality control and the Central Veterinary Laboratory; this group provided information, technical support and advice to the PMU on issues related to soil decontamination. In addition, a prevention working group comprised 16 focal points of the main institutions involved in pesticides management in Mali. The group provided information related to pesticides management to the PMU and met on a semi-annual basis to review and evaluate progress made under the prevention component, including the National Prevention Plan developed by the project. 142 The lessons learned are summarized from the ASP-P1 draft progress report for June 2006-December 2012.

102

• Soil decontamination activities should be tailor-made for each site. ASP-P1 Mali's unique experience in soil decontamination demonstrated that each site has its own characteristics and requires different implementation considerations. Land farming or sealing contaminated soil in concrete are affordable solutions if conditions permit.

• National strategies for container management should be developed through a community-based approach. Such an approach could guarantee the full involvement of associations and producer groups, administrative and municipal authorities, decentralized technical services and NGOs. Agreements should be signed on container centralization and collection, which clearly define roles and responsibilities.

• Developing and implementing a pesticide management communication plan is a prerequisite for the implementation of sound pesticide management. The audience of the communication plan should be defined by stakeholders and target groups.

• The importance of joint development of a national prevention plan. A participatory process could ensure the involvement of all stakeholders in the prevention of new accumulations of obsolete pesticide stocks.



8. ANNEXES


a. Project Cost by Component – All sources (in US$ million equivalent) – as of 12/31/2012

Component


equivalent)







Total 8.33 6.47 78%




equivalent)



Borrower Counterpart Funding/In-kind 0.75 0.98 130%

103


Multi-Donor Trust Fund (MDTF) Grant 3.98 1.89 18%

French Global Environment Facility (FFEM)

Parallel Co- financing/Grant 0.75 0.69 92%

Netherlands Parallel Co- financing/Grant 0.22 0 -

UN Food and Agriculture Organization (FAO)

Parallel Co- financing/Grant

and In-kind 0.08 0.18 225%

CropLife International (CLI) Parallel Co-

financing/ In-kind

- 0.29 -

Total 8.33 6.47 78%


1. The following table provides a comprehensive account of qualitative and quantitative information of outputs realized against original components (achievements at the time of project closing on December 31, 2012).


Outputs achieved at the time of closing in December 2012

Part A – Cleaning and elimination of obsolete pesticides and associated waste: Achievements under this component are rated Moderately Unsatisfactory Carrying out of a detailed inventory of the recipient’s publicly held obsolete pesticide stocks and associated waste through: a. provision of relevant training to the recipient; b. collection of data and compilation of a data

base; c. a risk-based prioritization of the stocks and

associated waste identified in the database; and d. preparation of the CESA.

Treatment and/or disposal of obsolete pesticides stocks and associated waste through: a. carrying out of a technological assessment of

available treatment and disposal options; b. implementation of the treatment and/or

disposal technology selected; and c. improving the operations of obsolete pesticide

stocks collection centers, including improving safety measures at those centers.

a. 40 government and non-governmental agents received training in conducting an inventory of obsolete pesticides and associated waste in 2007.

b. Data were collected, validated (on June 12, 2007) and compiled into the inventory database which had been under the management and oversight of FAO, as part of their Pesticide Stock Management System (PSMS).

c. Risk-based prioritization of the different sites was carried out to determine which sites require emergency clean up and safeguarding. The data were also used as basis for the preparation of the CESA (Original PDO Indicator #1 – 100%).

d. A CESA, identifying mitigation measures to treat possible negative environmental and social impacts that were considered likely to occur during implementation of the project, was completed in December 2009, and found satisfactory by the World Bank.

e. National soil contamination surveys were carried out in July 2007 and September 2009, and an assessment of available soil treatment options was carried out in May 2009 with the support of the TSU and University of Wageningen (Original PDO Indicator #3 – 100%). Treatment and disposal options for obsolete pesticide stocks were assessed as part of the CESA and site-specific EMPs.


104

f. The obsolete stocks (1,100 tonnes), were not collected or disposed of at the time of project closing (Original PDO Indicator #2 – 0%).

g. The following cleanup/safeguarding operations took place: inventoried low risk stocks at the various sites were repacked in metal containers and centralized in a store 35km outside Bamako; Elimination of 65 tonnes of obsolete pesticides and waste and soil decontamination in Gao; Safeguarding of obsolete pesticides, veterinary products and empty containers in Koutiala, Dialakoroba, Ménaka, Molodo, Niogoméra, Sikasso, Tombouctou, Sévaré, Kerbaye, Baguineda, Kéniéba, Kayes and Yélimané and in 7 rural communities in Koutiala and Kita; Safeguarding of obsolete pesticides and decontamination in the industrial park of Bamako and a site in Bla; Decontamination of highly polluted sites in Molodo, Dialakoroba and Niogoméra (through landfarming) and in Sévaré and Nara.

Part B – Prevention of obsolete pesticide accumulation: Achievements under this component are rated Satisfactory a. Building capacity through the provision of

training to farmers, civil society groups and relevant staff of the recipient;

b. Review and update of the legal and regulatory framework to control obsolete pesticide accumulation and strengthen enforcement mechanisms;

c. Promotion of improved pesticide management practices such as integrated pest management and integrated vector management, and improved management of empty pesticide containers through the design and implementation of container management strategy;

d. Awareness raising activities on the program, and the health and environmental hazards of pesticides, including carrying out of workshops on reducing reliance on pesticides in agricultural production, disease vector control and migratory pest control;

e. Development of a long-term funding strategy for prevention activities and a plan to prevent pesticide accumulation; and

f. Strengthening the recipient’s laboratory services.

a. A total of 4,451 governmental and non-governmental staff received training on various topics related to pesticide and waste management, communication and M&E (Original PDO Indicator #5 – 100%)

b. Reviewing and updating legal and regulatory framework to control re-accumulation and strengthen enforcement (Original PDO Indicator #4 – 100%): • Technical support was provided to the National Directorate of

Agriculture in the review of the Decree on pesticides management; in the development of procedures for obtaining import/export and sale authorizations; and in the development of permits and certificates for shops, storage facilities and transport of pesticides of control;

• Project objectives and activities were integrated into national policies for hazardous waste management (such as the Politique Nationale d’Assainissement, endorsed by the Council of Ministers in 2009), where an entire chapter was devoted to obsolete pesticides management at both the public and private sectors;

• Technical support was provided to DNACPN in the development of laws for empty container management and management of seized pesticides; in the development of laws for notification of seizure and detention of obsolete pesticides; and in the development of a Decree on Principle Pollutant Payer;

• Sensitization of Plant Protection agents and representatives from the Police Force and Customs on laws related to pesticides control.

c. Development and implementation of an empty container strategy for 29 rural communities in the CMDT cotton zones.

d. Development, validation, and implementation of a project communication plan via various communication tools to raise awareness of project activities (project logo, documentary films, sketches, reports, microprograms, folders, calendars, etc.); broadcasting of project activities via the radio and newspapers; workshop for 40 NGOs on pesticide management through PAN-Mali as a means to encourage NGO participation in project

105

activities. e. Development of a national prevention plan: supporting the National

Health Directorate in the development of a strategic prevention plan for chemical poisoning; establishment of a Prevention Working Group, comprising 24 members from national entities, associations and NGOs; dynamization of the National Pesticides Management Committee;

f. The project supported the Central Veterinary Laboratory in the development of a plan for the Laboratory of Toxicology and Environmental Quality Control.

Part C – Project management: Achievements under this component are rated Moderately Unsatisfactory Supporting the PMU in carrying out project coordination, implementation, monitoring and evaluation, through the provision of technical advisory services, goods and operational support.

The PMU received technical advisory services from the TSU (on inventory taking, data management, M&R, soil assessment and remediation techniques in partnership with the University of Wageningen, and safeguarding of low risk stocks), CLI (on disposal through the CLI-funded TAD), WWF (on communication), PAN-UK/Africa (on civil society engagement in project activities) and the World Bank (on procurement, financial management, safeguards and M&E). Project funds were used to purchase needed goods and other consultant services. The support to the PMU was hampered by: (i) technical disagreements between the TSU and the World Bank on the content of the M&E system, and the need for restructuring; (ii) significant delays in recruiting the TAD due to long negotiations between CLI and the government and delays within CLI in mobilizing the funds, and (iii) inconsistent procurement advice from the World Bank about the disposal contract.







4. A specific completion, and/or ICR mission was not carried out, and thus no stakeholder workshop was held.



106

6. Comments and edits received on the draft ICR were equally incorporated. Additional comments included suggested lessons learned:

• A national inventory should be a preliminary step during project preparation, particularly in the technical and financial arrangements for the component associated with the removal and cleaning. This could avoid the financing gaps and the necessity to seek additional funds for the elimination of additional stocks of obsolete pesticides and related waste.

• If the PMU consists of specialists who are directly recruited by the implementing agency, as was the case for ASP-Mali, it is useful and necessary to add counterparts to capitalize on experiences and to strengthen the capacity of these structures. The selection criteria of these counterparts should be based on the profile and long-term availability of applicants.

• National teams composed of technical pesticide management officers, who are well trained and equipped with adequate facilities and equipment, are able to secure a certain category of obsolete pesticides and related waste. This category includes stocks of small amounts (less than 500 kg) and low risk (not belonging to Classes Ia and Ib WHO). Using national teams can strengthen national capacities in securing stockpiles of obsolete pesticides and related waste. Once these capabilities are integrated into the missions and activities of the departments concerned, they could perform similar operations without input and external funding after the closure of the project. Using national teams can also shorten lead times and reduce the costs of security operations and disposal.

• With regard to decontaminated soils, each site has its own characteristics. The history of the sites and nature of contaminants determine the technical options available for cleanup operations. Land-farming and confinement techniques in the contaminated concrete layer allow for solving soil decontamination problems with local human resources and at lower costs.

• A national empty container management strategy using community-based approaches guarantees the involvement of associations and producer groups, administrative and municipal authorities, structured and decentralized technical services and NGOs. It also enhances sustainability. The signing of agreements on local collection/grouping of empty packaging, which define the roles and responsibilities of each other, would demonstrate the commitment of those concerned. It is also a prerequisite for a community management strategy targeting empty containers.

• Development and implementation of a specific communication plan on pesticides management is a prerequisite for the implementation of sound management of pesticides. This plan must take internal and external communication into account. The external communication plan should be defined by stakeholders and target groups.

• The national prevention plan represents a road map for implementing activities targeted at preventing the accumulation of obsolete pesticide stocks. Its development should be based on a coordinated and responsible approach involving all stakeholders.

• The working group that focuses on implementing the national prevention plan should include all stakeholders. Group members should be nominated focal points by their respective stakeholder group. As such, they would be responsible for collecting information and data relating to their respective stakeholder group and contribute the same to the group work in order to contribute to a comprehensive overview of national pesticide management.

107

• Involving NGOs in project implementation allows for training and raising the awareness of local producers. Priority activities should relate to: (i) the development of, translation into local languages, and dissemination of informational guides on the hazards and risks and good management of pesticides; (ii) the design and dissemination of micro programs on good management practices, pesticides and empty containers; (iii) community monitoring of hazards related to the use of pesticides; (iv) promoting Integrated Pest Management (IPM); and (v) independent monitoring of operations for disposal of obsolete pesticides and related waste.


7. No comments were received from co-financiers and other partners/stakeholders on the ASP-P1 Mali draft ICR.



115

TECHNICAL ANNEX III: ASP-P1 MOROCCO

116







CPS Country Partnership Strategy

DELM Direction de l’Epidémiologie et de la lutte contre les Maladies

DPVCTRF Direction de la Protection des Végétaux des Contrôles Techniques et de la Répression des Fraudes

DSPR Direction de la Surveillance et de la Prévention des Risques

DSSPA Direction de la Sécurité Sanitaire des Produits Alimentaires






IFR Interim Unaudited Financial Report





MADRPM Ministère de l’Agriculture, du Développement Rural et des Pêches Maritimes

MATEE Ministère de l’Aménagement du Territoire, de l’Eau et de l’Environnement




ONSSA Office National de la Santé Sanitaire des Aliments




117







SAICM Strategic Approach to International Chemicals Management





UNDP United Nations Development Program


WHO World Health Organization


118

TABLE OF CONTENTS

1. PROJECT CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES AND DESIGN ...... 124 1.1 CONTEXT AT APPRAISAL ............................................................................................................................................ 124 1.2 ORIGINAL PROJECT DEVELOPMENT OBJECTIVE AND INDICATORS ................................................................ 125 1.3 REVISED PROJECT DEVELOPMENT OBJECTIVE AND INDICATORS .................................................................. 125 1.4 MAIN BENEFICIARIES................................................................................................................................................... 125 1.5 ORIGINAL AND REVISED COMPONENTS ................................................................................................................. 126 1.6 OTHER SIGNIFICANT CHANGES ................................................................................................................................. 127

2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES ..................................................................... 127 2.2 IMPLEMENTATION ........................................................................................................................................................ 128 2.3 MONITORING AND EVALUATION (M&E) DESIGN, IMPLEMENTATION, AND UTLIZATION ......................... 130 2.4 SAFEGUARD AND FIDUCIARY COMPLIANCE ......................................................................................................... 130 2.5 POST-COMPLETION OPERATION/NEXT PHASE ...................................................................................................... 131

3. ASSESSMENT OF OUTCOMES ......................................................................................................................................... 132 3.1 RELEVANCE OF OBJECTIVES, DESIGN, AND IMPLEMENTATION ...................................................................... 132 3.2 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVE ................................................................................... 132 3.3 ACHIEVEMENT OF GLOBAL DEVELOPMENT OBJECTIVE ................................................................................... 133 3.4 EFFICIENCY .................................................................................................................................................................... 133 3.5 JUSTIFICATION OF OVERALL OUTCOME RATING ................................................................................................. 134 3.6 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS ............................................................................ 134 3.7 SUMMARY OF FINDINGS OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS .................... 134




8.1 PROJECT COSTS AND FINANCING ............................................................................................................................. 136 8.2 OUTPUTS BY COMPONENT ......................................................................................................................................... 137 8.3 ECONOMIC AND FINANCIAL ANALYSIS .................................................................................................................. 138 8.4 WORLD BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS ................................. 138 8.5 BENEFICIARY SURVEY RESULTS .............................................................................................................................. 138 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS ............................................................................................ 138 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR ................................................................. 138 8.8 COMMENTS OF COFINANCIERS AND OTHER PARTNERS/STAKEHOLDERS .................................................... 139

119


Country: Morocco Project Name: Africa Stockpiles Program - Project 1 Project ID: P103189 L/C/TF Number: TF57778 ICR Date: 11/27/2013 ICR Type: Core ICR Lending Instrument: APL Borrower: Kingdom of Morocco Original Total Commitment: US$4.0 million Disbursed Amount: US$0.29 million (06/30/2010)

Revised Amount: - Environmental Category: A Global Focal Area: CHEM Implementing Agency: Direction de la Protection des Végétaux des Contrôles Techniques et de la Répression des Fraudes (DPVCTRF) Co-financiers and Other External Partners: Government of the Kingdom of Morocco, CropLife International (CLI), UN Food and Agriculture Organization (FAO), CropLife Morocco, Pesticide Action Network in the United Kingdom (PAN-UK), PAN-Africa, and World Wildlife Fund (WWF).

B. Key Dates

Process Date Process Original Date Revised / Actual Date(s) Concept Review: 07/17/2002 Effectiveness: 02/12/2007 02/05/2007 Appraisal: 04/19/2004 Restructuring: - - Approval: 12/05/2006 Mid-term Review: 11/30/2008 12/11/2009 Closing: 06/30/2010 06/30/2010 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Unsatisfactory Risk to Global Environment Outcome: High World Bank Performance: Moderately Unsatisfactory Borrower Performance: Unsatisfactory C.2 Detailed Ratings of World Bank and Borrower Performance World Bank Ratings Borrower Ratings Quality at Entry: Moderately Unsatisfactory Government: Moderately Unsatisfactory

Quality of Supervision: Moderately Satisfactory Implementing Agency/Agencies: Unsatisfactory



Unsatisfactory


120


E. World Bank Staff

Positions At ICR At Approval Vice President: Inger Andersen Daniela Gressani/Gobind T. Nankani Country Director: Neil Simon M. Gray Theodore O. Ahlers Director (Regional Integration): Colin Bruce -

Sector Director: - Inger Andersen/Michel Wormser Sector Manager: Charles Cormier Vijay Jagannathan Project Team Leader: Dinesh Aryal Peter Kristensen/Denis Jordy ICR Team Leader: Dinesh Aryal ICR Primary Author: Ayala Peled Ben Ari F. Results Framework Analysis - At project approval, a country-specific results framework was not presented; only six PDO Indicators were defined.



-






Unsatisfactory

121

Project Development Objective (PDO) – as presented in the GEF Trust Fund Grant Agreement:144 To assist the recipient, in the project area, in: (a) eliminating inventoried publicly held obsolete pesticide stocks and associated waste; and (b) implementing measures to reduce and prevent future related risks. Revised PDO – Not Applicable Global Environment Objectives (GEO) - as presented in the PAD (Report No: 36273-AFR): Reduction of the effects of Persistent Organic Pollutants (POPs) on the global environment. Revised GEO – Not Applicable (a) PDO Indicators – Original PDO Indicators as presented in the GEF Trust Fund Grant Agreement are assessed against achievements realized at project closing on June 30, 2010.





Values


Years


Indicator 1: Inventory database of publicly held obsolete pesticide stocks in place and being used by the Project Management Unit (PMU) and stakeholders


No Yes - Yes


Target fully achieved (100%): An inventory was completed and validated in 2010. The database was subsequently used to prepare a draft disposal tender.

Indicator 2: Completion of the County Environmental and Social Assessment (CESA) and implementation of the measures contained therein


No Yes - No

Date Achieved 04/19/2004 06/30/2010 - 06/30/2010


Target not achieved (0%): The procurement process for a consulting firm to conduct the CESA began in February 2008 with the preparation of TORs. By project closing, a consulting firm was selected but not contracted.

Indicator 3: Inventoried publicly held obsolete pesticide stocks disposed of, as a result of the completion of the disposal services contract(s) in accordance with national and international laws and regulations


0% 100% - 0%

Date Achieved 04/19/2004 06/30/2010 - 06/30/2010

144 The Project Development Objective (PDO) and components that are stated in the Project Appraisal Document (PAD) slightly differ from those stated in the GEF Trust Fund Grant Agreement. This ICR bases its evaluation on the PDO, indicators, and components presented in the GEF Trust Fund Grant Agreement, which served as the principal guiding document for the project’s implementation activities.

122


Target not achieved (0%): Morocco adopted Tunisia’s short list of pre-qualified disposal firms, and a first draft of the tender document was prepared by the project. However, a disposal firm was not selected and no disposal activity took place by project closing.

Indicator 4: Legal and regulatory framework for pesticide management upgraded, including measures to strengthen compliance with the Conventions


No Yes - Yes


Target fully achieved (100%): Several pesticide/waste management-related legal texts and regulations were improved.

Indicator 5: The training program under Part D145 of the project is implemented and the knowledge so acquired is being used by PMU and stakeholders


Date Achieved 04/19/2004 06/30/2010 - 05/01/2009


Target partly achieved (50%): In 2007 and 2008 stakeholders participated in inventory and date management training. PMU staff participated in two World Bank fiduciary training workshops in 2008 and in 2009. The training was not entirely effective in increasing the PMU and stakeholders’ capacities in the subject matters.

Indicator 6: PMU is functional and its operation is satisfactory to the World Bank Value (quantitative or qualitative)

No Yes - Yes

Date Achieved 04/19/2004 06/30/2010 - 06/30/2010


Target partly achieved (50%): The PMU was staffed as needed throughout project implementation; however, it lacked capacity in procurement, financial management, M&E, planning and keeping up with timelines. Reporting was also delayed.


(c) Intermediate Outcome Indicator(s) – Not Applicable G. Ratings of Project Performance in ISRs – Some DO and IP ratings and actual disbursement data are not available because ISRs reported on ASP-P1 as a whole, and only occasionally on country-specific project performance, including disbursement.



1 10/31/2005 Not applicable as ASP-P1 Morocco not yet launched

2 06/30/2006

145 See Section 1.5 for the project’s Component D (in the GEF Trust Fund Grant Agreement referred to as ‘Part D’)

123

3 12/27/2006 4 06/28/2007 Moderately Satisfactory Moderately Satisfactory - 5 12/18/2007 Moderately Satisfactory Moderately Unsatisfactory - 6 05/30/2008 - - - 7 12/18/2008 Moderately Unsatisfactory Unsatisfactory 0.23 8 03/25/2009 Unsatisfactory Highly Unsatisfactory 0.28 9 12/29/2009 Unsatisfactory Highly Unsatisfactory 0.28 10 06/29/2010 Unsatisfactory Highly Unsatisfactory 0.29 06/30/2010 Final disbursement 0.29 H. Restructuring – Not Applicable I. Disbursement Profile – A disbursement profile is not available (see explanation under G).

124



1. Country background: After expanding significantly in the 1980s, agricultural production stagnated in the 1990s because of the increased incidence of drought, which damaged output of the drought-sensitive cereals that dominated Moroccan agriculture. The agricultural sector had been considered inefficient; modern, large-scale farms characterized certain parts of the country, elsewhere, however, farming suffered from the fragmentation of the land, low rates of mechanization and fertilizer use, and limited use of selected seeds. Locusts had been occasionally a problem in the far south, as in 2003-04, although crop damage had been small.

2. Sector background: At project approval, Morocco had imported approximately 9000 tonnes of pesticides per year, with the majority having been insecticides and fungicides. Large-scale farmers primarily used chemicals in controlling pests and diseases, whereas small farmers often used alternatives such as applying soil management or crop rotation.

3. Actions taken: Recognizing the potential hazard obsolete stocks of pesticides were posing to people, animals and to the environment, the Government of Morocco had developed legislative measures as well as qualified structures (laboratories and services) to handle pesticide issues. Since 2002, Morocco had taken a series of measures allowing the country to become eligible for international donations to reduce and eliminate pesticide stockpile and toxic waste. The Government of Morocco had initiated activities under its National Implementation Plan (NIP).146 In preparation of ASP-P1, a preliminary inventory had been conducted and identified 700 tons of obsolete stocks at approximately 225 sites. These stockpiles had been largely stocked for combating locust infestations and were held largely by the Ministries of Agriculture and Health. Only a small portion of the stocks was held by private companies.

4. Rationale for World Bank assistance: Please refer to Section 1.1 in the program-level ICR. 5. Higher-level objectives: The project intended to contribute to Morocco’s national development

strategy, in the areas of public health, poverty alleviation, environmental protection, and strengthening the agricultural sector. In particular, the ASP-P1 Morocco aimed to (i) improve the quality of life in poor communities by reducing environmental health risks, (ii) improve environmental protection, and (iii) enhance the capacity of the agricultural sector to better manage crop pests. In addition, the project aimed to consolidate various approaches to obsolete pesticide cleanup, generate practical lessons and produce practical tools that would facilitate its replication in other countries and regions. Contributing to international efforts to eliminate Persistent Organic Pollutants (POPs), improving management of toxic chemicals, and promoting alternatives to pesticide usage that include Integrated Pest Management (IPM) and Integrated Vector Management (IVM), represented further objectives.

146 National Implementation Plan for Stockholm Convention on Persistent Organic Pollutants (POPs), May 2006, Kingdom of Morocco – Ministry of Territory Management, Water and Environment.

125


6. The Project Development Objective (PDO), indicators, and components that are stated in the Project Appraisal Document (PAD)147 slightly differ from those stated in the GEF Trust Fund Grant Agreement.148 This ICR bases its evaluation on the PDO, indicators, and components presented in the GEF Trust Fund Grant Agreement, which served as the principal guiding document for the project’s implementation activities.

7. ASP-P1 Morocco’s original PDO as presented in the GEF Trust Fund Grant Agreement:149 To assist the recipient in: a) eliminating inventoried publicly held obsolete pesticide stocks and associated waste; and b) implementing measures to reduce and prevent future related risks.

8. At appraisal, only an overall ASP-P1 results framework had been prepared (see program-level ASP ICR); results frameworks for the individual ASP-P1 countries, including Morocco, were intended to be finalized during the individual ASP-P1 country project launches. However, country-specific PDO indicators were developed at appraisal. For ASP-P1 Morocco, these were as follows:

Original PDO Indicators as presented in the PAD and GEF Trust Fund Grant Agreement

a. Inventory database of publicly held obsolete pesticide stocks in place and being used by the Project Management Unit (PMU) and stakeholders.

b. Completion of Country Environmental and Social Assessment (CESA) and implementation of the measures contained therein.


d. Legal and regulatory framework for pesticide management upgraded, including measures to strengthen compliance with the Conventions.

e. The training program under Part D of the project is implemented and the knowledge acquired is being used by the PMU and stakeholders.150

f. The PMU is functional and its operation is satisfactory to the Bank.


9. Not Applicable.



147 Project Appraisal Document for the Republic of Mali, the Kingdom of Morocco, and the United Republic of Tanzania in support of the Africa Stockpile Program – Technical Annex 2, (Report No. 36273-AFR), November 7, 2006. 148 GEF Trust Fund Grant Agreement between the Kingdom of Morocco and the International Bank for Reconstruction and Development, (Number TF057778-MOR), February 5, 2007. 149 ASP-P1 Morocco PDO as presented in the PAD was: To eliminate inventoried publicly held obsolete pesticide stockpiles and associated waste, and implement measures to reduce and prevent future related risks. 150 See Section 1.5 for the project’s Component D, here referred to as “Part D”.

Table 1.1: ASP-P1 Morocco Original PDO Indicators

126


11. The PDO of ASP-P1 Morocco was to be achieved through the implementation of four distinct components (outlined in Table 1.2).



Component 1.A – Disposal of obsolete pesticides (Total: US$2.51 million; GEF: US$2.26 million): Country cleanup and disposal activities were intended to be based on a comprehensive and detailed inventory of pesticide stockpiles and related contamination, determining the level of risk to adjacent and wider communities or critical natural resources (such as water supplies). The cleanup was intended to include the repackaging of obsolete pesticides, transport and destruction of pesticides based on chemical analysis, needs assessment and technical capacity.

Part A – Obsolete pesticide inventory: Carrying out a detailed inventory of the recipient's publicly held obsolete pesticides stocks including: a. Collection of data and compilation of a database; b. A risk-based prioritization of the stocks identified in the

database; and c. Preparation of the CESA. Part B – Disposal of obsolete pesticides: Disposal of publicly held obsolete pesticides stocks and associated waste on the basis of an adequate disposal technology, and establishment, as needed, of obsolete pesticide stocks collection centers.

Component 1.B - Prevention of obsolete pesticides accumulation (Total: US$0.78 million; GEF: US$0.55 million): Activities in this component aimed at strengthening the pesticide management through improving pesticide registration, licensing, enforcement of import controls, stock management, waste management, and formulation of effective procurement strategies, as well as promotion of alternatives to chemical pesticides through improvement of pest control strategies with particular attention to IPM for agriculture and IVM for health care. Activities further included the development of awareness raising programs.

Part C – Prevention of obsolete pesticide accumulation: Carrying out activities aimed at preventing obsolete pesticide accumulation, including: a. Awareness raising activities on the health and

environmental hazards of pesticides; b. A review of the legal and regulatory framework to control

obsolete pesticide accumulation and enforcement mechanisms with a view to upgrading said framework and mechanisms;

c. Preparation and implementation of an IPM strategy; and d. A review of pesticide management practices and

preparation and implementation of a training plan.

Component 1.C - Capacity building (Total: US$0.27 million; GEF: US$0.27 million): Capacity building activities included the implementation of a training program and awareness raising campaigns. Environmental monitoring was to be supported through the provision of equipment and staff training for effective pesticide and other chemical management practices.

Part D – Capacity building: Provision of technical assistance, training and equipment to strengthen the capacity of PMU and the staff of relevant ministries and other stakeholders, in environmental monitoring, pesticide and other chemical management practices, and in carrying out awareness raising campaigns.

Component 1.D - Project management and Monitoring and Evaluation (Total: US$1.84 million; GEF: US$0.35 million): Activities included the establishment of a PMU: its responsibility was to implement the country level activities, solicit external technical support, monitor and evaluate project activities and coordinate their implementation with the GEF-funded NIPs for the Stockholm Convention and similar initiatives. Furthermore, the establishment of a steering committee representing key stakeholders, such as Non-Governmental Organizations (NGOs) and the private sector, and overseeing the work of the respective PMU.

Part E – Project management and evaluation: Supporting the PMU in carrying out project coordination, implementation, monitoring and evaluation, through the provision of technical advisory services, equipment and operational support.

Table 1.2: ASP-P1 Morocco Original Components

127


12. No changes were made to ASP-P1 Morocco. The project closed as originally scheduled on June 30, 2010.


13. Project preparation took place between 2001-2006, as part of preparatory activities at the program-level. It took more than five years from GEF eligibility in July 2001, and more than four years from concept note in July 2002 to have the project approved by the World Bank Board in December 2006.151

14. Soundness of the background analysis: Preparation was conducted in close collaboration between the World Bank and the Food and Agriculture Organization (FAO), with CropLife International (CLI) joining around appraisal (April 2004); mission reports do not indicate the participation of the World Wildlife Fund (WWF) and the Pesticides Action Network (PAN-UK/PAN-Africa) during preparation. The key preparatory inputs were the PAD and the Operational Manual with varying levels of quality: the PAD contained mostly ASP-P1 level information and plans with very little country-specific information, while the Operational Manual presented a considerable amount of country information.152 It was based on country background analyses, which had been carried out by national working groups. The World Bank carried out comprehensive procurement and Financial Management (FM) assessments in September 2004. While it was recognized by the World Bank, FAO, CLI and the government that the inventory carried out in 2002 had to be updated in order to determine the disposal budget,153 the new inventory was postponed to the implementation phase, most likely due to budgetary constraints. Consequently, a tentative disposal budget was agreed on. 154

15. Assessment of project design: The project’s design (as for all ASP-P1 participating countries) was preconfigured at the program level, and did not reflect any country-specific modifications or lessons learned from pervious projects executed in Morocco. Design weaknesses at the program level that translated into implementation delays at the country level, including ASP-P1 Morocco, are outlined in Section 2.1 of the program-level ICR. At the project level, design weaknesses were inherent in the complex implementation arrangements, where overall project management was with the Ministry of Agriculture, Rural Development and Fisheries (Ministère de l’Agriculture, du Développement Rural et des Pêches Maritimes - MADRPM), but it was one of at least two identified executing entities, with unclear definition of responsibilities and budget allocations.155 Another key weakness was the

151 The long preparation time resulted from a lengthy partnership building/negotiation process at the program-level resulting from the complex implementation arrangement and project design (see Section 2.1 in the program-level ICR). 152 Information included considerable detail onthe legislative framework for pesticide management, stakeholder analysis, national disposal options, processes for establishing national prevention and empty containers plans, procurement and financial management procedures, health and safety procedures and an M&E framework. 153 See Royaume du Maroc - Programme Africain Relatif aux stocks de pesticides obsolètes - Première phase, Mission de préparation, Aide-mémoire, December 2003 154 US$2.265 million for component A, of which US$165,000 for inventory and US$2.1 million for disposal. 155 Within MADRPM, the Direction de la Protection des Végétaux, du Contrôle technique et de la Répression des Fraudes (DPVCTRF) was the main executing agency and host of the PMU; the Direction de la Surveillance et de la Prévention des Risques (DSPR) within MATEE was the second executing agency. The Operational Manual also includes the Direction de l’Epidémiologie et de la lutte contre les Maladies (DELM) within the Ministry of Health (Ministère de Sante) as a third executing agency; however this is not mentioned in the PAD.

128

agreed disbursement arrangements, which relied on government’s central budgeting and disbursement system (see more under Section 2.2).

16. Government commitment: During preparation, the World Bank reported on strong government commitment as was evident from its proactivity in starting to analyze the baseline situation before it was formally agreed on.156 In addition, in 2003, the government established multi-stakeholder thematic working groups and a steering committee. Finally, the government took the necessary actions to process the ratification of the Stockholm Convention on POPs, which was a condition for joining the program. The process was finalized in June 2004.

17. Assessment of risks: The PAD and the Operational Manual presented brief and partial risk assessments, which focused mainly on crosscutting and program-level risks; corresponding mitigation measures were proposed. The World Bank procurement and FM/disbursement assessments carried out during preparation identified respective risks, which were specific to ASP-P1 Morocco, and offered mitigation measures. The risks, which were also reflected in the PAD and the Operational Manual, correctly highlighted possible future delays in commitments and payment as a result of Morocco’s pre-financing system;157 weak FM capacity of the implementing agency; and the joint responsibility of the executing agencies to produce Interim Unaudited Financial Reports (IFRs). The appraisal mission in April 2004 identified weak inter-ministerial coordination of prevention activities as a project risk; however, this risk was not included in the PAD or the Operational Manual.

2.2 IMPLEMENTATION

18. The implementation of ASP-P1 Morocco experienced significant delays throughout implementation, which ended with the closing of the project on June 30, 2010 as scheduled (with no extensions). During the December 2009 mid-term review, and in view of unsatisfactory project performance, the World Bank and the government agreed on an action plan of corrective measures; its timely implementation served as a minimum condition for a closing date extension. However, by June 30, 2010 the action plan had only been partially implemented, leading to a World Bank decision to close the project.

19. Key factors that affected implementation included: • Institutional challenges most notably an internal government restructuring, halted project

implementation. Project management was substantially affected by an internal government restructuring process, which resulted in a lack of institutional leadership, a need to redefine a number of project aspects (such as disbursement arrangements and counterpart financing), and a need for repeated training. 158 In addition, project management was affected by (i) an unclear

156 See Royaume du Maroc - Programme Africain Relatif aux stocks de pesticides obsolètes - Première phase. Mission de préparation, Aide-mémoire, December 2003 157 Whereby payments are made directly to suppliers, regardless of the source of funding, by the Moroccan Treasury (Trésorerie Générale du Royaume) from the Budget funds upon submission of supporting documentation and payment orders by the project unit. Reimbursement is made directly from the grant account. 158 MADRPM underwent a significant restructuring process. In May 2009, the PMU was temporarily transferred from DPVCTRF to the Department of Health Safety of Food Products (Direction de la Sécurité Sanitaire des Produits Alimentaires - DSSPA) under MADRPM. In January 2010 the PMU was again transferred to the newly established ONSSA, which became the agency responsible for pesticide management and for project implementation. In July 2009, the PMU was mainly composed of former DPVCTRF staff who were assigned to DSSPA until ONSAA was established. These changes impacted the PMU because it had

129

definition of roles and responsibilities of PMU members from both MADRPM and the Ministry of Spatial Planning, Water and Environment (Ministère de l’Aménagement du Territoire, de l’Eau et de l’Environnement – MATEE); (ii) a lack of full time staff159 and the staff’s weak fiduciary and project management capacity; (iii) ineffective collaboration between stakeholders (most likely due to unclear definition of roles, responsibilities and budgets); and (iv) low-level representation in the Steering Committee. Challenging disbursement arrangements and a lack of flexibility in budget planning further affected project implementation.

• Lengthy procedures to mobilize GEF funds in absence of a project special account: The project was dependent on the government’s budget because designated/special accounts were largely not allowed in Morocco; in order to access the GEF funds, each executing agency had to develop and submit annual budgets to the government, to be approved as part of the national budget approval process, and once approved, changes to the budget were difficult to make. The lengthy process also caused project activities to only begin nine months after effectiveness. In addition, in order to effect payments, each contract had to go through the Ministry of Finance’s control in addition to the World Bank approval process, which took considerable time.160

• Higher than estimated amounts of obsolete pesticides caused delays: It took seven months for the project to complete an inventory (December 2007-July 2008), and 22 months to enter all data into FAO’s Pesticide Stock Management System (PSMS) and have it validated (March 2008-January 2010; originally 15 months were allocated for data validation). A data validation workshop took place in February 2010. The underlying reasons for these delays were: (i) increase in the number of sites (340 instead of 225 originally planned); (ii) difficulties in carrying out an inventory during the summer and Ramadan; (iii) repeated visits to some sites due to removal of stocks and poor quality of the collected data/photos as a result of insufficient training; and (iv) the large number of pesticides found at the sites, resulting in multiple data sheets to be entered into the PSMS.161

• Insufficient and delayed technical support affected implementation performance: FAO’s Technical Support Unit (TSU) support was insufficient (e.g., inventory, PSMS management and M&E). In addition, while the recruitment of the Technical Advisor for Disposal (TAD) was initiated in January 2007, she was only recruited by CLI in January 2010. The valuable support, however, came too late and was not used by the PMU before project closing.

to relocate to a new facility with unreliable internet connection, which was necessary to maintain the PSMS and overall project management. 159 The coordinator, because of his other responsibilities as the Head of Division within the DPVCTRF, could not commit all of his time to the project; the four technical officers were similarly involved in other activities in DPVCTRF 160 Any budget allocation request, which had not been specifically included in the GEF Grant Agreement, such as MATEE’s and NGOs’ budgets, was not approved, and activities were consequently not implemented. Sources: Programme Africain Relatif aux Stocks de Pesticides Périmés, Mission de Supervision au Maroc 14-17 Mai, 2007, Aide-Mémoire and Back-to-Office Report : Morocco - Africa Stockpiles Programme – Implementation Support Mission (P0103189 / TF057778). 161 Sources: Programme Africain relatif aux Stocks des Pesticides Périmés, PASP – Maroc : État d’avancement, July 2009 (PowerPoint Presentation); and Programme Africain Relatif aux Stocks de Pesticides Périmés (Phase 1) - Mission d’achèvement du 7 au 9 juin 2010, Aide Mémoire (Draft), July 25, 2010.

130

2.3 MONITORING AND EVALUATION (M&E) DESIGN, IMPLEMENTATION, AND UTLIZATION

20. Overall M&E - Rating Unsatisfactory. 21. M&E design – Rating Unsatisfactory: The design of the M&E framework, as presented in the

Operational Manual, was generally appropriate but lacked the most important element – a project-level results framework. Although the appraisal mission developed a project-level results framework,162 it was not reflected in the PAD and the Operational Manual, both presenting only the overall ASP-P1 results framework (see program-level ASP ICR) and six country-specific Performance Monitoring Indicators.163 These indicators did not allow for adequate monitoring of project implementation progress because they focused on measuring outputs as opposed to progress towards project outcomes. Expected results under the prevention component, in particular, were not adequately considered by the indicators. The PAD further indicated that a country-specific framework would be finalized during project launch. Other key M&E elements, such as strategies and responsibilities, were detailed in the Operations Manual. The inventory was expected to provide baseline and target values for disposal, and overall M&E was expected to be carried out using standardized methodologies for data collection and quality assurance. The responsibility for data collection rested with the PMU with data provided by the various project stakeholders, assisted by the TSU and external consultants.

22. M&E implementation and utilization – Rating Unsatisfactory: A project-specific results framework was developed with the government during the February 2008 World Bank mission; however, it was not officially introduced to the project (through a restructuring or addition to the Operations Manual), and it is unclear whether the PMU used it to produce progress reports. The late completion of the national database (in January 2010) left little time to use these data for a CESA and disposal activities.


23. Safeguard compliance – Rating Moderately Satisfactory: ASP-P1 triggered two safeguard policies: OP 4.01 on Environmental Assessment and OP 4.09 on Pest Management.

24. ASP-P1 Morocco complied with OP 4.01 on Environmental Assessment: The project did not begin to prepare the CESA by the project closing due to considerable delays in the completion of data validation of the inventory as mentioned above, coupled with weak PMU procurement capacity. In terms of on-the-ground activities’, the World Bank observed at mid-term that some health and safety procedures had not been properly applied during the inventory process by the trained teams. No other field activities took place, which necessitated consideration of OP 4.01. Note that the safeguard rating applies to activities that took place before the project closed in June 2010. Activities not conducted by the project closing (i.e., preparation of a CESA and EMP) were not factored into the rating.

162 See Africa Stockpiles Program (ASP) - Morocco - Appraisal mission Aide Memoire (April 22 to 27, 2004). 163 Lengthy discussions between the partners during project preparation on whether to have an M&E system at the program-level alone, or also at the country-level, may have contributed to the absence of an M&E system at the country-level at project launch. However, no information was available on whether it was a deliberate decision not to have an M&E system in place at the country-level.

131

25. OP 4.09 on Pest Management: Since project activities themselves met requirements of a pest management plan (i.e., through pesticide management, IPM and capacity building), individual country projects, including ASP-P1 Morocco, were not required to prepare self-standing documents. See Section 2.4 in the program-level for more details.

26. Financial Management Compliance – Rating Unsatisfactory: Despite training sessions provided by the World Bank, the PMU’s financial management capacity was insufficient to ensure smooth management of funds and timely preparation of reports. Financial management was further weakened by the above-mentioned disbursement arrangements and institutional changes. In December 2007, following considerable procurement delays, the PMU recruited an administrative, financial and procurement consultant, who was tasked to strengthen the PMU’s fiduciary capacity and to develop a financial management manual for the project. Poor performance of the consultant, partly caused by delayed payments by the government’s treasury department, resulted in the termination of the contract in June 2009. In December 2009, another fiduciary consultant was identified; however, the recruitment was not finalized due to the institutional changes and proximity to the closing date.

27. GEF disbursement was low throughout the project’s lifetime. The first disbursement was recorded in December 2008 (5.9%), two years after project approval, mainly on inventory equipment, inventory training, a vehicle and operating costs. In May 2009, disbursement increased to 7.4% and remained at this level until project closing. The low level of disbursement is linked to the above-mentioned implementation delays (the CESA and disposal contract, which together were allocated over 56% of the GEF grant amount, were not implemented by project closing), and cumbersome disbursement procedures.

28. Procurement – Rating Unsatisfactory: Project procurement was considerably delayed due to the PMU’s weak capacity and the centralized contract awarding system. A number of training sessions and the above-mentioned recruitment of a consultant did not strengthen procurement capacity. Despite the World Bank authorization to use ASP-Tunisia’s list of pre-qualified disposal firms, the disposal procurement process did not advance as expected, and at project closing, only a first draft of the tender document was available.


29. Post completion: After the project closed, two related activities were carried out by the government with donor support: (i) a national team composed of ten technical staff from the Ministry of Public Health received training by the Strategic Approach to International Chemicals Management (SAICM) project, led by the Ministry of Public Health. Consequently, 50 tonnes of Dichlorodiphenyltrichloroethane (DDT) were packed and stored in Oued Zem. This DDT stock is expected to be disposed of under a GEF project titled 'Alternatives to DDT in the Middle East and North Africa', led by the World Health Organization (WHO) and the United Nations Environment Program (UNEP); (ii) in 2011, at the request of the government, FAO implemented a small project, which identified and qualified priorities for a national program on sound POPs and pesticides management. The following priorities were identified: (i) revising pesticide legislation; (ii) establishing a registration system for pesticides used in agriculture, public and animal health; (iii)

132

technical and analytical support for pesticide quality control; and (iv) extension services and farmer field schools.164

30. Next phase: In January 2012, the GEF approved a concept for a follow-up disposal and prevention project titled ‘Disposal of Obsolete Pesticides including POPs and Implementation of Pesticides Management Programme’, to be managed by FAO.165 The project is designed to address the above-mentioned priorities, in addition to (i) disposing of the stocks and remediation of contaminated soil identified by ASP-P1 Morocco, as well as stocks of a recently banned pesticide (endosulfan); (ii) elimination of legacy empty containers and development of a container management system; and (iii) development of a network for information exchange within and among pesticide using sectors. The full project proposal is currently being developed by FAO.



31. Relevance – Rating High: Even though the project’s objectives were ambitious, they were and they continue to be highly relevant to Morocco, especially in view of the implementation of the Green Morocco Plan, and its focus on increasing agricultural productivity through intensification of production systems. The project is similarly relevant to the most recent Country Partnership Strategy (CPS)166 Program Area 2.6 (Agricultural Sector Reform), which looks to increase competitiveness and diversification of the agricultural sector; and Program Area 3.3 (Solid Waste Management) to support disposal practices complying with international social and environmental standards. Project objectives and components also support the Government of Morocco in complying with the Stockholm Convention on POPs, the Basel Convention on Control of Trans boundary Movements of Hazardous Waste and their Disposal, and the Rotterdam Convention on the Prior Informed Consent Procedure for Certain Hazardous Chemicals and Pesticides in International Trade.


32. The project has been evaluated against its PDO by measuring achievements at the time of the project closing (June 30, 2010).

33. PDO achievement – Rating Unsatisfactory: The assessment took the project’s six PDO indicators into account, of which two were fully achieved (100%), two were partly achieved (50%) and two were not achieved (0%). The three main project outcomes measured by the indicators include:167 (i) sound disposal and management of obsolete pesticides; (ii) prevention of accumulation of new stockpiles of obsolete pesticides; and (iii) enhanced capacity and institutional strengthening of pesticide management.

164 Source: Disposal of Obsolete Pesticides including POPs and Implementation of Pesticides Management Programme, GEF Project Identification Form (FAO), January 5, 2012. 165 See http://www.thegef.org/gef/project_detail?projID=4738 for the project concept. 166 Country Partnership Strategy for the Kingdom of Morocco for the Period FY10-13, (Report No. 50316-MA), December 30, 2009. 167 Achievements were identified based on intended outcomes presented in Section B.1 and Annex I of the PAD (Report No: 36273-AFR).

133

34. Sound disposal and management of obsolete pesticides and contaminated sites. The level of risks emanating from publicly held obsolete pesticides and associated waste to adjacent and wider communities or critical natural resources was determined and quantified through the completion of a national inventory of publicly held obsolete pesticides in 2007-2008 and its validation in 2008-2010 (PDO Indicator #1 - 100%). However, this information was not used to prepare a CESA (PDO Indicator #2 - 0%) or contract a disposal company by project closing. Inventoried stocks were not disposed of under a disposal contract due to considerable delays in the inventory and validation activities, and the lack of a CESA as a guiding document (PDO Indicator #3 - 0%).

35. Prevention of accumulation of new stockpiles of obsolete pesticides. Project and parallel government-led activities taking place during the project lifetime helped improve the legal and regulatory framework for pesticide management (PDO Indicator #4 - 100%) by improving several pesticide/waste management-related legal texts and regulations. In 2008, the ASP NGO network conducted three workshops on pesticide and empty container management. In June 2009, both PAN-Morocco (established in February 2009) and the ASP NGO network collaborated to develop and disseminate a brochure on the proper management of pesticides. Other awareness raising tools were to be developed by the project but did not materialize due to procurement delays.

36. Enhanced capacity and institutional strengthening of pesticide management. In October 2007, 21 plant protection agents received inventory training from the TSU; and in March 2008, ten plant protection agents received data management training using the PSMS from the TSU. PMU staff participated in two World Bank fiduciary training workshops in 2008 and in 2009. The training sessions were not entirely effective in increasing the PMU and stakeholders’ capacities in the subject matters (Original PDO Indicator #5 – 50%).


37. The project aimed to directly contribute to the GEO of reducing the effects of POPs on the global environment through: (i) improving the quality of life in poor communities by reducing environmental risks; (ii) improving environmental protection; and (iii) enhancing the capacity of the agricultural sector to better manage crop pests. GEO achievement is measured by PDO achievement (see Section 3.2).

3.4 EFFICIENCY

38. Efficiency – Rating Low: This rating represents a broad estimate, as financial and economic data to support this rating are scarce. At appraisal, an economic or financial analysis was neither undertaken for ASP-P1 at the program- nor at the country-level due to non-measurable benefits of reducing the risk of contaminating the environment. A lack of reliable baseline data in combination with controversial aspects related to the valuation of human life and natural resources neither allowed for a cost-benefit analysis. In addition, efficiency of capacity building and prevention activities is difficult to quantify due to time disconnect between project support and actual results. For GEF purposes, the ASP-P1 underwent incremental cost analysis of achieving global benefits to address the issue of obsolete pesticides in the participating countries.

39. The project was effective for over three years (February 2007 to June 2010) but in fact, implementation took place for about two and a half years (November 2007-June 2010). Within this time period little on the ground achievements were recorded although a fully staffed PMU was in

134

place. Of the 7.4% total disbursement at closing, 56% were spent on operating costs while the rest was spent mainly on inventory equipment and a project vehicle, demonstrating low project efficiency.


40. Overall Outcome – Rating Unsatisfactory: On the basis of the project’s efficiency (low), relevance (high), and PDO achievements (unsatisfactory), the overall outcome is rated unsatisfactory. The overall outcome rating takes into account the few project achievements and lack of any tangible environmental and health impacts.


41. Not Applicable


42. A three days mission took place shortly before project closing (June 7-9, 2010); however, it did not include a stakeholder workshop. The government did not prepare a completion report.


43. Risk to the development outcome - Rating High: The government of Morocco is aware of the risks posed by obsolete pesticides, but lacks the capacity and resources to safeguard and dispose of the stocks. According to FAO,168 without a donor-supported project, inventory and disposal would not be completed, and heavily contaminates sites could not be cleaned up. Pesticides management will continue to be addressed periodically without coordination and connection to a national strategic framework. FAO’s planned follow up project could fill these gaps.



44. World Bank performance in ensuring quality at entry – Rating Moderately Unsatisfactory: The preparation of the project was conducted in conjunction with preparatory activities at the program-level, which were lengthy (see Section 2.1 in the program-level ICR). The World Bank made considerable preparation efforts, which involved agreements on project scope, implementation arrangements, work plans, FM and procurement. However, it was not effective in clearly defining project implementation arrangements and embedding clear responsibilities and budgets in either the PAD or the GEF Trust Fund Grant Agreement, causing significant implementation problems later on. In addition, the World Bank did not agree with the government on simpler disbursement arrangements. This worsened quality at entry, which was already adversely affected by problematic program-related design factors.

168 See Disposal of Obsolete Pesticides including POPs and Implementation of Pesticides Management Programme, GEF Project Identification Form (FAO), January 5, 2012.

135

45. Quality of World Bank supervision – Rating Moderately Satisfactory: The project was supervised by the program co-TTL responsible for francophone countries (Mali, Morocco and Tunisia) and a country co-TTL, both based in Washington DC. Five supervision and/or technical support missions (including the mid-term review mission) and a closing mission were conducted between February 2007 and June 2010, averaging twice a year. Aide Memoires were detailed, providing candid description of issues and risks. Expertise brought in these missions (operations, pesticide management, procurement, FM and safeguards) were adjusted to meet the perceived project needs. In addition, World Bank fiduciary staff based in the Rabat office provided additional support to the PMU. However, the negative effect of repeated TTLs changes (four in total) is noted. In addition, the World Bank was not proactive in offering adjustments to the project, such as restructuring, in a timely manner.

46. Overall World Bank performance – Rating Moderately Unsatisfactory: Based on the above-presented evaluation, the overall borrower performance is rated moderately unsatisfactory.


47. Government performance – Rating Moderately Unsatisfactory: The government of Morocco showed commitment to the issue of pesticide management by working to improve a number of legal and regulatory texts during the life of the project. However, the government did not provide the necessary high-level oversight to the project via the steering committee and dictated cumbersome disbursement procedures. While the national authorities were unanimous in confirming their high-level commitment to the World Bank,169 this was clearly disconnected from project implementation performance.

48. Implementing Agency performance – Rating Unsatisfactory: The PMU’s management weaknesses were exacerbated by the restructuring of MADRPM and the shifting of the unit between directorates/agencies. Although the PMU demonstrated adequate technical capacity, its procurement, FM and planning capacities did not improve despite training and other support it received.

49. Overall Borrower performance – Rating Unsatisfactory: Based on the above-presented evaluation and the project’s unsatisfactory outcome rating, the overall borrower performance is rated unsatisfactory.

6. LESSONS LEARNED

50. The most significant lessons learned are drawn at the program level, as the experience they are based on was often reflected across all country-specific projects; they are therefore discussed in the program-level ICR. Lessons learned that were particularly relevant to ASP-P1 Morocco include:

• Activities and disbursement schedules should be realistic. Projects involving a number of executing agencies are inherently complex and should dictate realistic operational and disbursement timetables accordingly.

• Steering committees should ensure the sustained participation of decision-making staff. The participation of lower level staff in meetings greatly limits the effectiveness of such a committee

169 See Back-to-Office Report: Africa Stockpiles Program-Morocco (P103189/GEF TF057778) – Mid-Term Review, Dec 7-11, 2009

136

to fulfill its role as a conduit between government priorities, operational progress and results achieved; further, it greatly limits its ability to dictate corrective measures.



8. ANNEXES



Component


equivalent)




4. Disposal of obsoletes pesticides 2.51 N/A170 N/A




8. Contingencies 0.57 N/A N/A

Total 5.98 N/A N/A




equivalent)



Borrower Counterpart Funding 1.98 N/A N/A

Global Environment Facility (GEF) Grant 4.0 0.29 7.3%

Total 5.98 0.29 4.8%


137


1. The following table provides a comprehensive account of qualitative and quantitative information of outputs realized against original components (achievements at the time of project closing in June 2010).


Outputs achieved at the time of closing in June 2010

Part A – Obsolete pesticide inventory: Achievements under this component are rated Moderately Unsatisfactory Carrying out a detailed inventory of the recipient's publicly held obsolete pesticides stocks including: a. Collection of data and compilation of a



a. Inventory began in December 2007 and was completed in July 2008. The inventory covered 340 sites, revealing 480 tonnes of obsolete pesticides plus 230m3 (February 2009 estimate: 756 tonnes); 600 m3 of soil; 800 m3 of equipment; 670 m3 of material; 730 m3 plus 22,240 containers/drums; and 904m3 of building material. Data entry began in March 2008, and was fully entered and validated in January 2010. The project conducted a validation workshop in February 2010.

b. N/A c. The CESA’s procurement process began in February 2008 with the

preparation of TORs. By project closing a consulting firm was selected but not contracted.

Part B – Disposal of obsolete pesticides: Achievements under this component are rated Unsatisfactory Disposal of publicly held obsolete pesticides stocks and associated waste on the basis of an adequate disposal technology, and establishment, as needed, of obsolete pesticide stocks collection centers.

No concrete results at project closing: Tunisia’s short list of pre-qualified disposal firms was adopted ad a first draft of the tender document was prepared. No collection centers were established.

Part C – Prevention of obsolete pesticide accumulation: Achievements under this component are rated Moderately Unsatisfactory Carrying out activities aimed at preventing obsolete pesticide accumulation, including: a. Awareness raising activities on the health and

environmental hazards of pesticides; b. Review of the legal and regulatory framework

to control obsolete pesticide accumulation and enforcement mechanisms with a view to upgrading said framework and mechanisms;

c. Preparation and implementation of an IPM strategy; and

d. Review of pesticide management practices and preparation and implementation of a training plan.

a. In 2008, the PMU organized communication material for posters and pamphlets about the project and good plant protection practices (not printed and distributed). In November 2008, TORs for the production of two television spots on pesticide storage/empty containers management and good plant protection practices were prepared and received no objection from the World Bank. This did not materialize due to procurement difficulties. In 2008, the ASP NGO network with the financial support of PAN and other donors conducted thee workshops on pesticide and empty container management. In June 2009, both PAN-Morocco (established in February 2009) and the ASP NGO network collaborated to develop and disseminate a brochure on the proper management of pesticides in external funding to conduct training sessions for farmers.

b. Improvement of several pesticide/waste management-related legal texts and regulations as a result of project and a number of government entities’ actions.

c. In February 2009, TORs were prepared for a diagnostic study of the


138

management of pesticides, which was to inform the preparation of a national prevention plan leading to a national strategy. The procurement process was not successful since only one firm showed interest despite three successive publications.

d. For IPM and training activities, see Part D below.

Part D – Capacity building: Achievements under this component are rated Moderately Unsatisfactory Provision of technical assistance, training and equipment to strengthen the capacity of the PMU and the staff of relevant ministries and other stakeholders, in environmental monitoring, pesticide and other chemical management practices, and in carrying out awareness raising campaigns.

In October 2007, 21 plant protection agents received inventory training from the TSU; and in March 2008, ten plant protection agents received data management training using the PSMS from the TSU. A communication strategy was developed and validated by WWF in June 2007 and by the World Bank in February 2008. It included a number of outreach activities and training sessions on the impact of pesticides on the environment, human health, and also initiatives to promote IPM with small farmers. An international TAD was hired by CLI in January 2010. In the short period of her work, the TAD supported the preparation of the disposal tender.

Part E - Project Management and Evaluation: Achievements under this component are rated Moderately Unsatisfactory Supporting the PMU in carrying out project coordination, implementation, monitoring, and evaluation, through the provision of technical advisory services, equipment, and operational support.

Participation of PMU staff in two World Bank fiduciary training workshops: Bamako in May 2008 and Rabat in May 2009. Limited M&E support from the TSU at project onset. Operational and pesticide management support from the World Bank in July 2009.










6. The borrower did not submit a completion report. 7. The draft ICR was shared with the borrower; however, no comments were received.

139

8.8 COMMENTS OF COFINANCIERS AND OTHER PARTNERS/STAKEHOLDERS

8. No comments were received from co-financiers and other partners/stakeholders on the ASP-P1 Morocco draft ICR.


140

TECHNICAL ANNEX IV: ASP-P1 SOUTH AFRICA

141



AVCASA Association of Veterinary and Crops Associations of South Africa




DAFF Department of Agriculture, Forestry, and Fisheries

DEA Department of Water and Environmental Affairs

DDT Dichlorodiphenyltrichloroethane

EA Environmental Assessment


EMTK Environmental Management Toolkit






IFR Interim Financial Report



IWMP Integrated Waste Management Plan





NRP National Retrieval Project





POPs Persistent Organic Pollutants

142


QEA Quality at Entry Assessment







143

TABLE OF CONTENTS

1. PROJECT CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES AND DESIGN ...... 152 1.1 CONTEXT AT APPRAISAL ............................................................................................................................................ 152 1.2 ORIGINAL PDO AND INDICATORS ............................................................................................................................. 153 1.3 REVISED PDO AND INDICATORS ............................................................................................................................... 154 1.4 MAIN BENEFICIARIES................................................................................................................................................... 154 1.5 ORIGINAL AND REVISED COMPONENTS ................................................................................................................. 155 1.6. OTHER SIGINIFICANT CHANGES .............................................................................................................................. 156

2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES ..................................................................... 157 2.1 PROJECT PREPARATION, DESIGN, AND QUALITY AT ENTRY ............................................................................ 157 2.2 IMPLEMENTATION ........................................................................................................................................................ 159 2.3 MONITORING AND EVALUATION (M&E) DESIGN, IMPLEMENTATION, AND UTILIZATION ........................ 161 2.4 SAFEGUARD AND FIDUCIARY COMPLIANCE ......................................................................................................... 162 2.5 POST-COMPLETION OPERATION/NEXT PHASE ...................................................................................................... 163


4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME .......................................................................................... 168 5. ASSESSMENT OF BANK AND BORROWER PERFORMANCE .................................................................................. 169



8.1 PROJECT COSTS AND FINANCING ............................................................................................................................. 171 8.2 OUTPUTS BY COMPONENT ......................................................................................................................................... 172 8.3 ECONOMIC AND FINANCIAL ANALYSIS .................................................................................................................. 174 8.4 WORLD BANK LENDINGA AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS .............................. 175 8.5 BENEFICIARY SURVEY RESULTS .............................................................................................................................. 176 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS ............................................................................................ 176 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR ................................................................. 176 8.8 COMMENTS OF COFINANCIERS AND OTHER PARTNERS/STAKEHOLDERS .................................................... 176

144


Country: South Africa Project Name: Africa Stockpiles Program - Project 1 Project ID: P075776 L/C/TF Number: TF55763 ICR Date: TBC ICR Type: Core ICR Lending Instrument: APL Borrower: Republic of South Africa

Original Total Commitment: US$1.7 million Disbursed Amount: US$0.71 million (06/30/2012)

Revised Amount: - Environmental Category: A GEF Focal Area: CHEM Implementing Agency: Ministry of Water and Environmental Affairs - Department of Environmental Affairs (DEA) Co-financiers and Other External Partners: Government of Republic of South Africa, CropLife International (CLI), Pesticide Action Network (PAN-UK and PAN-Africa), and World Wildlife Fund (WWF).

171 Information presented in the datasheet was inserted manually. Information provided in the system was incomplete resulting from coding issues experienced during implementation; these were not resolved prior project closing.

B. Key Dates

Process Date Process Original Date Revised / Actual Date(s) Concept Review: 07/17/2002 Effectiveness: 09/30/2005 04/04/2006

Appraisal: 03/28/2004 Restructurings: 11/25/2009, 05/03/2011, 06/30/2011

Approval: 09/08/2005 Mid-term Review: 03/2008 01/23/2009 Closing: 09/30/2009 06/30/2012

C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Global Environment Outcome: Moderate World Bank Performance: Moderately Unsatisfactory Borrower Performance: Moderately Satisfactory

C.2 Detailed Ratings of World Bank and Borrower Performance World Bank Ratings Borrower Ratings Quality at Entry: Moderately Government: Satisfactory

145


E. World Bank Staff

Positions At ICR At Approval

Vice President: Makhtar Diop Christiaan J. Poortman/Gobind T. Nankani

Country Director: Asad Alam Theodore O. Ahlers Director (Regional Integration): Colin Bruce Mark Tomlinson Sector Director: - Inger Andersen Sector Manager: Jonathan S. Kamkwalala Vijay Jagannathan Project Team Leader: Dinesh Aryal Steven P. Maber ICR Team Leader: Dinesh Aryal ICR Primary Author: Veruschka Schmidt

Unsatisfactory

Quality of Supervision: Moderately Unsatisfactory Implementing Agency:




Overall Borrower Performance:



QAG Assessments (if any) -



-

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

-



146

F. Results Framework Analysis - At project approval, no country-specific result framework existed; only six PDO Indicators were defined. Following a Level 1 Restructuring in June 2011, a country-specific results framework was introduced. The original six PDO Indicators were revised and reduced to two PDO Indicators, and in addition, four Intermediate Outcome Indicators were introduced. Project Development Objective (PDO) - as presented in the GEF Trust Fund Grant Agreement:172 To assist the recipient in: (a) eliminating inventoried publicly held obsolete pesticide stocks and associated waste; and (b) implementing measures to reduce and prevent future related risks. Revised PDO - as presented in the Level 1 restructuring of June 2011 (Report No: 62795-ZA): To assist the Recipient in the development and piloting of a sustainable system to identify, collect and dispose of obsolete pesticides and associated waste. Global Environment Objectives (GEO) - as presented in the PAD (Report No: 32232-MNA): Reduction of the effects of Persistent Pollutants (POPs) on the global environment. Revised GEO – Not Applicable

(a) PDO Indicator(s) - Original PDO Indicators as presented in the GEF Trust Fund Grant Agreement are assessed against achievements realized at the time of the project’s Level 1 restructuring in June 2011; revised PDO Indicators are assessed against achievements realized at project closing in June 2012.



approval documents)


Values




Indicator 1: Inventory database of publicly held obsolete pesticides stocks in place and being used by the PMU and the recipient’s government staff


No Yes - No

Date Achieved 04/19/2004 09/30/2009 - 06/30/2011


Target not achieved (0%): In 2007, it was recognized that the ASP-P1 country implementation approach was not applicable in South Africa. Carrying out an inventory was considered cost-ineffective in an environment where most stocks were held privately; in addition, the government was already using a national register to maintain pesticide data.

Indicator 2: Completion of CESA, and implementation of the measures contained therein

Value (quantitative or qualitative) No Yes - No


147

Date Achieved 04/19/2004 09/30/2009 - 06/30/2011


Target not achieved (0%): In 2007, it was recognized that the project implementation approach used in other ASP-P1 countries was not applicable in South Africa. A CESA was subsequently not prepared.

Indicator 3: Inventoried publicly held obsolete pesticides stocks disposed of, as a result of the completion of the disposal services contract(s) in accordance with national and international laws and regulations

Value (quantitative or qualitative) 0 250 - Approx. 17

Date Achieved 04/19/2004 09/30/2009 - 06/30/2011


Target partly achieved (6.8%): 102.5 tonnes of obsolete pesticides were collected and stored in Limpopo province in 2006. Of those, approximately 17 tonnes were publicly held non-hazardous stocks and disposed of locally in 2009 using GEF funds.



No Yes - Yes

Date Achieved 04/19/2004 09/30/2009 - 06/30/2011


Target fully achieved (100%): A national Pesticides Management Policy, which supported the objectives of the conventions, was adopted and published by DAFF in 2010.

Indicator 5: The training program under Part D173 of the project is implemented and the knowledge so acquired is being used by the PMU and recipient’s government staff

Value (quantitative or qualitative) No Yes - Partly (80%)

Date Achieved 04/19/2004 09/30/2009 - 06/30/2011


Target partly achieved (80%): A training program was not developed. However, training in financial management, procurement, project management, CESA preparation, and Pesticide Stock Management System (PSMS) application was received, and a study tour to Australia was conducted; additional envisaged procurement training was not received.

Indicator 6: The PMU is functional and its operation is satisfactory to the World Bank Value (quantitative or qualitative) No Yes - Partly (80%)


Target partly achieved (80%): PMU was functional and performing moderately satisfactorily. However, procurement was conducted without following World

173 See Section 1.5 for the project’s Component D (in the GEF Trust Fund Grant Agreement referred to as ‘Part D’).

148

Bank guidelines in a couple of instances, subsequently resulting in a failure to procure a disposal contractor, and financial management staff did not allocate sufficient time to the project.


Indicator 1: A system for identifying, collecting and disposing of private and public obsolete pesticides stocks and associated waste developed through collaboration with the private sector association


No system in place

Private and public obsolete pesticides stocks and associated waste identified, collected and disposed of (using the booking line)

-

Private and public obsolete pesticides stocks and associated waste identified in three provinces (100%), collected (33%), and disposed of in one province (33%)

Date Achieved 06/30/2011 06/30/2012 - 06/30/2012


Target partly achieved (55%): A booking line to facilitate identification, collection, and disposal of privately and publicly held stocks was developed in collaboration with the private sector, tested in two provinces, and subsequently introduced to all other provinces in South Africa. Stocks were identified in three provinces (100%), collected (33%), and disposed of in one province (33%).

Indicator 2: Three pilot safeguarding and/or disposal operations implemented Value (quantitative or qualitative) 0 3 - 1

Date Achieved 06/30/2011 06/30/2012 - 06/30/2012


Target partly achieved (33%): At Limpopo province, 100 % of obsolete pesticides were disposed of using CLI. At Western Cape and Free State provinces, obsolete pesticides were identified, but not collected, safeguarded and/or disposed of within the implementation period.

(b) GEO Indicator(s) –The GEO was measured by the PDO and PDO indicators. (c) Intermediate Outcome Indicator(s)

Indicator Baseline Value Original Target Values


Formally Revised

Target Values



Indicator 1: Booking line developed, tested and maintained Value (quantitative or qualitative)

No booking line in place

Booking line developed, tested and -

Booking line developed, tested,

149

maintained and maintained Date Achieved 06/30/2011 06/30/2012 - 06/30/2012


Target fully achieved (100%): A booking line was developed in collaboration with the private sector, tested in two provinces, subsequently introduced to all other provinces in South Africa, and maintained by the private sector industry.

Indicator 2: Number of site-specific Environmental Management Plans Value (quantitative or qualitative)

1 3 - 1

Date Achieved 06/30/2011 06/30/2012 - 06/30/2012


Target partly achieved (33%): For Limpopo province, an EMP was prepared by the disposal contractor, and approved by the World Bank in 2010. For Western Cape and Free State provinces, EMPs were not prepared as the disposal contractor was not recruited prior project closing.

Indicator 3: Number of sites in which identified stocks are collected Value (quantitative or qualitative)

1 3 - 1

Date Achieved 06/30/2011 06/30/2012 - 6/30/2012


Target partly achieved (33%): In Limpopo province, all stocks were collected in 2007. In Western Cape and Free State provinces, stocks were not collected, as the disposal contractor was not recruited prior project closing.

Indicator 4: Training/study tours implemented


1 2 (Study tour to Australia; auditing and monitoring courses)

- 1

Date Achieved 06/30/2011 06/30/2012 - 06/30/2012


Target partly achieved (50%): Government staff participated in a study tour to Australia in 2008 to examine the country’s collection and disposal system. No information was available on whether auditing and monitoring courses were conducted.

G. Ratings of Project Performance in ISRs - Some DO and IP ratings and actual disbursement data are not available because ISRs reported on ASP-P1 as a whole, and only occasionally on country-specific project performance, including disbursement.

No. Date ISR Archived DO IP

Actual Disbursements (US$ millions)

1 10/31/2005 - - 0 2 06/30/2006 Satisfactory Satisfactory 0 3 12/27/2006 Satisfactory Satisfactory 0 4 06/28/2007 Moderately Satisfactory Moderately Satisfactory 0 5 12/18/2007 Moderately Satisfactory Satisfactory 0

150

6 05/30/2008 - - 0.08 7 12/18/2008 Moderately Satisfactory Satisfactory - 8 03/25/2009 Moderately Satisfactory Satisfactory 0.29 9 12/29/2009 Unsatisfactory Unsatisfactory 0.37 10 06/29/2010 Unsatisfactory Unsatisfactory 0.37 11 03/27/2011 - Unsatisfactory 0.55 12 12/25/2011 Unsatisfactory Unsatisfactory 0.71 13 06/29/2012 Unsatisfactory Unsatisfactory 0.71 06/30/2012 Final Disbursement 0.71 H. Restructuring


Board Approved

GEO Change



in US$ millions


GEO IP

11/25/2009 No Not

Available Not

Available 0.37

• Extension of closing date by 17 months (11/30/2009 to 04/30/2011)

• Reallocation of proceeds to accommodate costs associated with a storage facility for the stocks at Limpopo, disposal of stocks at two additional provinces, and project management

• Increase of the percentage of expenditure to be financed to 100% for all but Consultant Services categories due to budgetary constraints within the Government

05/03/2011 No Not

Available Not

Available 0.55 • Extension of closing date by 14 months

(04/30/2011 to 06/30/2012)

06/30/2011 Yes Not

Available Not

Available 0.55

• Revision of the PDO and indicators to reflect a shift from complete disposal to the development and piloting of a sustainable system to identify, collect, and dispose of stocks in the private and public sectors

• Reallocation of proceeds due to changes associated with a) costs incurred by the storage of stocks collected at Limpopo; b) fewer

151


Board Approved

GEO Change



in US$ millions


GEO IP

workshops needed and training carried out through parallel financing (CLI); c) disposal of and/or safeguard additional stocks; and d) increased operating costs because of previous extension.

• Change in the financing plan: addition of US$0.51 million from CLI

• Introduction of a revised safeguards instrument




Against Original PDO targets Moderately Unsatisfactory Against formally revised PDO targets Moderately Unsatisfactory Overall (weighted) rating Moderately Unsatisfactory I. Disbursement Profile – A disbursement profile is not available (see explanation under G).

152



1. Country background: South Africa’s agricultural share of the economy has been outstripped by those of the mining and secondary industries. In the 1930s, agriculture contributed about 20% of the GDP; at project approval in 2005, the sector’s share of the GDP had dropped to 2.6%.174 Despite the farming industry's declining share, it has remained vital to the economy, development, and stability of South Africa, with the industry employing approximately 30% of the workforce.

2. Sector background: To boost production, chemical pesticides had been widely used in the agricultural sector. These included large quantities of Dichlorodiphenyltrichloroethane (DDT) between the 1950s and early 1980, before a 1983 ban on all uses of DDT as an active ingredient for agricultural purposes was introduced. The environment in which pesticides were applied, however, had been largely uncontrolled, as pesticides had primarily been used by unskilled farm workers with a low level of literacy and by households in domestic applications. The same was relevant for chemicals sold to the public or used in products. Chemicals sold to the industry, on the other hand, had been sold within a controlled environment where workers were skilled or semi-skilled and the use could be managed. Another use of pesticides concerned the health sector to prevent and control malaria. Since 1946, DDT had been used for malaria vector control in South Africa for indoor residual spraying.175 In 1996, the government stopped the usage of DDT and relied on pyrethroid insecticides. However, DDT was re-introduced for indoor residual spraying in 2000, at the height of a malaria epidemic.

3. Obsolete pesticide accumulation: The accumulation of obsolete pesticides in South Africa had resulted from pesticide bans, insufficient control, and a lack of awareness of their efficient use. In general, government involvement in pesticide purchasing had been limited and the import and distribution was conducted mainly by the private sector, and large amounts of obsolete pesticides had been accumulated by private farmers. In addition, the industry had accumulated large amounts of pesticide stocks due to the unavailability of local destruction technologies, and the high scale of commercial farming throughout the country also resulted in significant amounts of obsolete stocks.

4. Institutional background: Between 1947 and 2010, management of pesticides was governed by the National Department of Agriculture (NDA, currently titled Department of Agriculture, Forestry and Fisheries - DAFF).176 In 2010, DAFF approved and published the national Pesticides Management Policy,177 which stated that DAFF would receive a statutory mandate to regulate pesticides in collaboration with other relevant departments through Memoranda of Understanding; and that

174 The Economist Intelligence Unit, Country Profile South Africa, 2005 175 In 1996, DDT was replaced with pyrethroids before a subsequent increase in infection rates had forced the reintroduction of DDT in 2000. 176 Under the Fertilizers, Farm Feeds, Agricultural Remedies and Stock Remedies Act of 1947. 177 Fertilizers, Farm Feeds, Agricultural Remedies and Stock Remedies Act, Act No 36 of 1947. Pesticide Management Policy for South Africa. GG No 33899, GN 1120 of 2010. DAFF 2010.

153

Table 1.1: ASP-P1 South Africa Original PDO

collection programs for obsolete pesticides would be instituted by DAFF. However, since 2008, pesticide waste has been managed by the Department of Water and Environmental Affairs (DEA).178

5. Actions taken: In 1997, DAFF launched a National Retrieval Project (NRP) aimed at disposing unwanted pesticides nation-wide. With national funds, pesticides were centralized in 45 collection depots, and approximately 1,040 tonnes of obsolete pesticides, including 70 tonnes of DDT and other Persistent Organic Pollutants (POPs), were disposed of overseas and locally.179 Different surveys undertaken in the Stellenbosch and neighboring areas in 1995, 2000, and 2003, however, concluded that since the NRP, new stocks had accumulated. In addition to government owned stocks that had been used to combat migratory pests, there were significant quantities scattered amidst the estimated 55,000-60,000 commercial farms throughout the country. In 2004, the Association of Veterinary and Crops Associations of South Africa (AVCASA) and the Endangered Wildlife Trust (a local Non-Governmental Organization (NGO)) initiated a project, which collected pesticides directly from households. Obsolete pesticides were dropped off at participating nurseries countrywide, to then be collected and repackaged for disposal. The initiative collected approximately 12 tonnes of pesticides, including 2% of POPs, which were stored in a landfill facility. These stocks were subsequently integrated into ASP-P1 South Africa.

6. Rationale for World Bank assistance: Please refer to Section 1.1 in the program-level ICR. 7. Higher-level objectives: The project intended to contribute to South Africa’s national development

strategies in the areas of public health, poverty alleviation, environmental protection, and to strengthen the agricultural and rural development sector. In particular, the project aimed to (i) improve the quality of life in poor communities by reducing environmental health risks, (ii) improve environmental protection, and (iii) enhance the capacity of the agricultural sector to better manage crop pests. Contributing to international efforts to eliminate POPs, improving management of toxic chemicals and promoting alternatives to pesticide usage that include Integrated Pest Management (IPM) and Integrated Vector Management (IVM) represented further objectives.


8. The Project Development Objective (PDO), indicators, and components that are stated in the Project Appraisal Document (PAD)180 slightly differ from those stated in the GEF Trust Fund Grant Agreement. This ICR bases its evaluation on the PDO, indicators, and components presented in the GEF Trust Fund Grant Agreement, which served as the principal guiding document for the project’s implementation activities.

Original PDO as presented in the PAD

Original PDO as presented in the GEF Trust Fund Grant Agreement

178 Under the National Environmental Management: Waste Act of 2008 DEA, which is responsible for regulation of hazardous and non-hazardous waste. 179 The most toxic portion of the stocks was incinerated in the UK while the less toxic stocks were disposed of at a commercial hazardous waste site in South Africa. 180 South Africa and Tunisia: Africa Stockpile Program – Project 1: PAD (Report No: 32232-MNA), August 19, 2005

154

Table 1.2: ASP-P1 South Africa Original PDO Indicators

To eliminate 250 tonnes of inventoried publicly held obsolete pesticides stockpiles and associated wastes,181 and implement measures to reduce and prevent future related risks.

To assist the recipient in: (a) eliminating inventoried publicly held obsolete pesticide stocks and associated waste; and (b) implementing measures to reduce and prevent future related risks.

9. At appraisal, only an overall ASP-P1 results framework had been prepared (see program-level ICR); results frameworks for the individual ASP-P1 countries, including South Africa, were intended to be finalized during the individual ASP-P1 country project launches. However, country-specific PDO indicators were developed at appraisal. For ASP-P1 South Africa, these were as follows:

Original PDO Indicators as presented in the PAD and the GEF Trust Fund Grant Agreement a. Inventory database of publicly held obsolete pesticide stocks in place and being used by the Project Management Unit

(PMU) and the government's staff. b. Completion of Country Environmental and Social Assessment (CESA) and implementation of the measures contained

therein. c. Inventoried publicly held obsolete pesticides stocks disposed of, as a result of the completion of the disposal services

contract(s) in accordance with national and international laws and regulations. d. Legal and regulatory framework for pesticide management improved, including measures to strengthen compliance with

the Basel Convention and the Rotterdam Convention. e. The training program under Part D of the project is implemented and the knowledge so acquired is being used by the

PMU and the recipient's government staff.182 f. The PMU is functional and its operation is satisfactory to the World Bank.


10. The PDO and associated indicators for ASP-P1 South Africa were revised through a Level 1 restructuring on June 30, 2011.183

11. ASP-P1 South Africa’s revised PDO: To assist the recipient in the development and piloting of a sustainable system to identify, collect and dispose of obsolete pesticides and associated waste.

12. ASP-P1 South Africa’s revised PDO Indicators: (i) A system for identifying, collecting and disposing of private and public obsolete pesticide stocks and associated waste developed through collaboration with the private sector association; and (ii) three pilot safeguarding and/or disposal operations implemented.


13. The original project design as described in the PAD targeted: (i) rural communities and individuals living in proximity to publicly held stocks of obsolete pesticides across the country, and (ii) government agencies, principally DEA and DAFF, which were expected to benefit from sustainable

181 The GEF Trust Fund Grant Agreement defined associated waste as materials that are contaminated by obsolete pesticides, including inter alia containers, equipment, and heavily contaminated soils where the World Bank agrees that those soils are within the project scope. 182 See Section 1.5 for the project’s Component D, here referred to as “Part D”. 183 Restructuring Paper on a Proposed Restructuring of the Africa Stockpiles Program to the Republic of South Africa, (Report No: 62795-ZA), June 20, 2011.

155

Table 1.3: ASP-P1 South Africa Original and Revised PDO Indicators

mechanisms aimed at managing accumulations of publicly held obsolete pesticides, and targeted training. Following the expansion of the project's scope to include private stocks and reduction of activities to focus on the Limpopo, Western Cape and Free State provinces, the main beneficiaries of the project became: (i) small scale farmers in the three provinces, and (ii) the pesticide industry, which became an active player in both the cleanup of the three provinces and the development of long-term mechanisms for collecting and disposing stocks.


14. The PDO of ASP-P1 South Africa was to be achieved through the implementation of the following components. The table also shows revisions that were made by the June 2011 Level 1 restructuring.184

Original Components as presented in the PAD185

Original Components as presented in the GEF Trust Fund Grant

Agreement

Revised Components (Level I Restructuring June 30, 2011)

Component 1.A – Disposal of obsolete pesticides (Total: US$0.73 million; GEF: US$0.97 million): a. To undertake a detailed pesticide

inventory; b. To produce a CESA including an

Environmental Management Plan (EMP);

c. To safeguard obsolete stockpiles and associate waste;

d. To undertake a technological assessment of disposal options for the inventory and disposal activities; and

e. Disposal.

Part A – Obsolete pesticide inventory: a. Collection of data and compilation of

a data base; b. A risk-based prioritization of the

stocks identified in the data base; and c. Preparation of the CESA.

Part A – Identification and collection of obsolete pesticides and associated waste: a. Establishing a booking line where farmers

in selected provinces voluntarily declare obsolete pesticides and associated waste in their possession;

b. Preparing EMPs; and c. Collecting and transporting identified

obsolete pesticides and associated waste to collection centers.

Part B – Disposal of obsolete pesticides: a. Carrying out a technological


b. Implementation of the treatment and/or disposal technology selected; and

c. Improving the operations of obsolete pesticide stocks collection centers, including improving safety measures at those centers.

Part B - Safeguarding and/or disposal of obsolete pesticides and associated waste: a. Carrying out of a technological

assessment of available safeguarding and disposal options; and

b. Implementing the safeguarding and/or disposal technology selected in three pilot provinces.

Component 1.B - Prevention of accumulation (Total: US$0.45 million; GEF: US$0.30 million): a. Raise awareness about activities

to be undertaken; b. Review regulations, laws and

enforcement and prepare

Part C – Prevention of obsolete pesticide accumulation: a. Awareness raising activities on the

health and environmental hazards of pesticides;

b. A review and update of the legal and regulatory framework to control

184 Restructuring Paper on a Proposed Restructuring of the Africa Stockpiles Program to the Republic of South Africa, (Report No: 62795-ZA), June 20, 2011 185 Indicated GEF $amounts are not in line with GEF $amounts presented in Annex 8.1 as a result of inconsistencies in the PAD.

156

revisions; c. Prepare a draft IPM strategy and

initiate four pilot projects; and d. Review pesticide management

practices and prepare a revised training plan.

Obsolete Pesticide accumulation and strengthen enforcement mechanisms;


d. A review of pesticide management practices and preparation and implementation of a training plan.

Component 1.C - Capacity building (Total: US$0.34 million; GEF: US$0.23 million): a. Strengthen the project

management skills of the PMU; and

b. Training program and workshops.

Part D – Capacity building: Strengthening the capacity of the PMU to implement the project through the provision of training.

Part C – Awareness raising and capacity building: Carrying out of activities aimed at raising awareness on the health and environmental hazards of pesticides and strengthening the capacity of the PMU to implement the project through the provision of training.

Component 1.D - Project management and Monitoring and Evaluation(M&E) (Total: US $0.28 million; GEF: US$0.30 million): a. Establish a PMU at the National

Department of Environmental Affairs and Tourism (DEAT); and

b. Support the further design and implementation of a project Monitoring and Evaluation system.

Part E – Project management: Supporting the PMU in carrying out Project coordination, implementation, monitoring and evaluation, through the provision of technical advisory services, equipment and operational support.

Part D – Project management: Supporting the PMU in carrying out project coordination, implementation, monitoring and evaluation through the provision of technical advisory services, equipment and operational support.

1.6. OTHER SIGINIFICANT CHANGES


Extension of closing date Extension of closing date by 17 months from 11/30/2009 to 04/30/2011.

11/25/2009 To compensate for delays, and to ensure implementation of the sustainability option.

Regional VP

Extension of closing date by 14 months from 04/30/2011 to 06/30/2012.

05/03/2011 To allow for the completion of the Level 1 restructuring and remaining disposal and prevention activities.

Regional VP

Reallocation of proceeds

Reallocation of proceeds from the Unallocated category (-US$165,000) to the Goods (+US$80,000), Works (+US$35,000) and Operating Costs (+US$50,000) categories. Increase of the percentage of refunding of expenditures to 100% for all categories but

11/25/2009 To accommodate rent costs of the storage facility and continuation of project management during the extended period, and budgetary constraints within the Government of South Africa, which had kept certain activities from moving forward.

Regional VP

Table 1.4: ASP-P1 South Africa Other Significant Changes

157

Consultant Services.

Reallocation of proceeds from the categories Consultant Services (US$215,000), Workshops and Training (US$75,000) and Unallocated (US$245,000) to the categories Goods (+US$50,000), Works (+US$250,000), Disposal Services (+US$165,000) and Operating Costs (+US$70,000).

06/30/2011 To accommodate a) costs incurred by the storage of Limpopo stocks; b) fewer workshops needed and necessary training carried out using parallel financing; c) funding needs for additional stocks; and d) increased operating costs because of the previous extension.

Board

Redefinition of activities Adjustment of safeguards instruments for the safeguarding and disposal work

06/30/2011 Following implementation of the EMP and independent review during the Limpopo pilot.

Board

Redefinition of the term “Disposal Services” from services related to the treatment or disposal of obsolete pesticides and associated waste, to services related to the safeguarding and/or disposal of the same.

06/30/2011 Addition of in country safeguarding as a possible cheaper end result for lower risk stocks; this was due to larger than budgeted stocks of obsolete pesticides and associated waste.

Board

Change of financing plan Change in the financing plan: addition of US$0.51 million from CLI.

06/30/2011 To reflect CLI’s parallel financing of the privately held hazardous waste disposal from the Limpopo Province, an activity that was not funded by the GEF Grant. CLI’s contribution was provided through a direct agreement between CLI and DEA.

Board

Addition of risks Addition of a Moderate-L risk: Inventory is inaccurate.

06/30/2011 To compensate for potential delays in the implementation of the collection and disposal contract given past performance.

Board



15. ASP-P1 South Africa was prepared between 2001 and 2005, as part of preparatory activities at the program-level. It took more than four years from GEF eligibility in July 2001, and more than three years from concept note in July 2002 to project approval by the Board in September 2005.186

16. Soundness of background analysis: While a considerable amount of background analysis was conducted during project preparation, key data were missing, which could have qualified and quantified needed interventions. The government's laboratory, financial and procurement capacity was assessed, existing IPM practices and relevant national legislation were analyzed, and previous clean up experience was taken into account. Although the World Bank team recommended carrying out a rapid inventory and an assessment of disposal technological options during preparation,187 these and a number of other preparatory activities were postponed to the implementation phase. Key

186 The long preparation time resulted from a lengthy partnership building/negotiation process at the program-level resulting from the complex implementation arrangement and project design (see Section 2.1 in the program-level ICR). 187 The appraisal Aide-Memoire also recommended carrying out a pesticides management baseline and a brief audit of the project management capability of PMU staff.

158

preparatory outputs such as the PAD and the Operational Manual were therefore lacking: (i) a precise estimate of the nature and amount of stocks to determine the focus (public vs. private stocks) and budget of the project; (ii) an assessment of disposal technological options (also to determine costs); and (iii) an analysis of pesticide use in the country, identifying the sources of stocks. They were further lacking a country-specific results framework and a disbursement estimate, and contained only preliminary work plans and cost tables. World Bank preparation mission reports demonstrate good collaboration between DEA, DAFF, the World Bank, FAO and CLI. Missions were carried out jointly and there was general consensus on the project design by the key stakeholders.188 Representatives from relevant central and provincial departments, NGOs, the pesticide industry and research institutions provided recommendations on the design of the inventory. There is no indication, however, that communities were consulted during preparation. These consultations were planned to take place during implementation. DEA and DAFF agreed on their respective responsibilities for implementation in line with their departmental focus: DEA intended to focus on the inventory, the CESA, and disposal activities (Component A), while DAFF intended to lead prevention activities (Component B).

17. Assessment of project design: The project’s design (as for all ASP-P1 participating countries) was preconfigured at the program-level, and did not reflect country-specific needs and modifications. Using a broad generic program-level design affected project implementation performance of ASP-P1 South Africa: (i) the project’s focus on publicly held obsolete pesticide stocks was not relevant to South Africa’s environment, as the majority of stocks was privately held, and a substantial amount of publicly held obsolete pesticides had already been disposed of in prior years; and (ii) the envisaged inventory and subsequent data maintenance through application of FAO’s Pesticide Stock Management System (PSMS) were not considered cost-effective in light of large quantities of obsolete pesticides being held privately and an already existing national register that was used by the government to maintain pesticide data.

18. No country-specific lessons learned were spelled out in the PAD, while some were highlighted in the Operational Manual. The key lesson learned for South Africa emerged from previous disposal initiatives that had been undertaken in the country, which demonstrated that prevention strategies were as important as the actual disposal of stocks. It should be noted that previous cleanup operations showed that in South Africa, unlike other ASP-P1 projects, government involvement in pesticide purchasing had been extremely limited and the import and distribution was conducted mainly by the private sector.189

19. Government commitment: Government showed high level of commitment towards the issue of pesticide management throughout preparation. During project pre-appraisal, high-level government officials expressed their interest in launching the project;190 at appraisal, the project was considered a priority for the government.191 It should be noted that at the time South Africa requested World Bank/GEF assistance through ASP-P1, the country was in the early process of preparing a new legislation to strengthen the regulation of chemical and hazardous waste; the project was intended to be a key instrument to implement this policy.

188 Appraisal Aide-Memoire, June 2004. 189 See Operations Manual, page vi 190 See pre-appraisal Aide Memoire, March 2003 191 See appraisal Aide Memoire, June 2004

159

20. Assessment of risks: While country-specific risks were not discussed in the PAD, the Operational Manual included a comprehensive risk assessment. Risks within the control of the project were correctly associated with poor project planning, a possible lack of funds and public opposition against to project activities resulting from insufficient awareness. Risks outside the control of the project were also adequately identified and included changes in national priorities or government implementation arrangements, delays related to local bureaucracy/issuance of licenses and permits, seasonal weather patterns, and a lack of support from farmers and the pesticide industry. Correlating risk mitigation measures for controllable risks were identified in the Operational Manual.

2.2 IMPLEMENTATION

21. The project was subject to two extensions for a total of 31 months, and a Level 1 restructuring in 2011.

22. Key factors affecting implementation and outcomes included: • Recognition of project approach not being appropriate for South Africa affected implementation

of originally envisaged disposal activities. During the first year of project implementation, the envisaged project design was judged being inadequate by all partners involved, as it was not applicable to the South African context. The existing design that was applied to all ASP-P1 participating countries, including South Africa, was based on the assumption that pesticide stocks were held by public agencies and that the private sector would be relatively underdeveloped; however, in South Africa the majority of stocks was held by private farmers and not by the government - public stocks were limited to controlling migratory pests and a substantial amount of publicly held obsolete pesticides had already been disposed of in prior years.192 To allow for a better assessment of the issue of obsolete pesticide stocks in the South African context that could inform the development of an improved project implementation approach, the World Bank and partners quickly reacted by agreeing to pilot the identification and collection of both public and private stocks in one province, Limpopo.193

• The first phase of the pilot was completed in 2007, with CLI having played a key role in the planning and implementation. The pilot revealed higher than estimated stocks of obsolete pesticides, and the already presumed high proportion of privately held stocks. The pilot confirmed the need to adopt a different collection and disposal model for the project, limiting the project’s geographic coverage and encompassing both private and public stocks because: (i) a country-wide obsolete pesticide cleanup was not considered suitable as it was estimated to cost about US$3 million more than budgeted at appraisal;194 and (ii) the cleanup of predominantly privately-held stocks using public funds would not have been supported by DEA, as South

192 During the late 1990s, pesticides were centralized in 45 collection depots, and approximately 1,040 tonnes of obsolete pesticides, including 70 tonnes of DDT and other POPs, were disposed of overseas and locally with national funds. See Section 1.1 for more information. 193 Limpopo, a province in the north of South Africa, was considered well prepared for participation in the pilot, and was characterized by a strong agricultural sector. 194 Following the collection of approximately 102.5 tonnes of obsolete pesticides at Limpopo alone, 700-900 tonnes were estimated to be scattered across South Africa, versus 250 tonnes estimated at appraisal.

160

African law required the producer or generator of pollution to pay for the costs of avoiding pollution, cleaning up, or remedying its effects.195

• Development of a different collection and disposal model, including the creation of a public-private partnership was a lengthy process. The government and the World Bank agreed to halt originally envisaged project activities until a better adapted project model for the disposal of obsolete pesticides in South Africa was identified. Instead, the focus was placed on facilitating a dialogue between the government and the private sector, to establish a partnership for collection and disposal of obsolete stocks, which the new approach could be based on; however, given the stakes involved, the process was lengthy and slow. The mid-term review, conducted in January 2009, consequently resulted in a sharp downgrade of ratings and confirmed the need to restructure the project for better alignment with the country's setting. Eventually in March 2010, and informed by the Limpopo pilot and an Australian pesticide collection system that had been examined during a study tour, DEA and the private industry association, AVCASA were able to reach an agreement. It was decided to implement two additional pilot cleanup operations (with utilization of remaining GEF funds), intended to help inform the development of an industry-run scheme for dealing with existing and future accumulations of both public and private stocks in the country.

• Challenges associated with institutional arrangements delayed and eventually excluded implementation of prevention activities from the project. The implementation of prevention activities was planned to be implemented by DAFF.196 However, the necessary co-operation between DEA and DAFF suffered from difficulties in advancing budgetary resources between the departments and a missing special account, and subsequently resulted in prevention activities still not having started in 2010. Upon review of activities included under this component, it finally became apparent that most prevention activities would be covered by DAFF under the newly gazetted National Pesticide Policy (2010), and by AVCASA under its stewardship program. The two entities subsequently informed the World Bank that GEF funding would no longer be needed as prevention activities would be integrated in DAFF's work program, and disconnected from the project.

• Restructuring to reflect the project’s modified disposal model was implemented late in the process. Only in June 2011, ASP-P1 South Africa underwent a Level 1 restructuring to correct the identified design flaws. In addition to the above-mentioned reasons, the delay also resulted from the World Bank’s preference to restructure ASP-P1 South Africa together with three other ASP-P1 participant countries. The restructuring included: (i) a revised PDO and associated indicators to reflect the changed focus and geographic coverage; (ii) a revised project description to reflect the modified PDO and the fact that prevention activities were carried out with DAFF’s own funds and no longer part of the project; (iii) endorsement of a country-specific results framework which met current World Bank/GEF requirements; and (iv) revised safeguard requirements including a shift from a CESA to independently reviewed Environmental Management Plans (EMPs).

195 Including 12.5 tonnes of pesticides, which were collected in 2002 by the NGO Endangered Wildlife and safeguarded by CLI. It was agreed to include these stocks for disposal. 196 DEA and DAFF signed a Memorandum of Understanding in 2006, to formalize the departments’ work relationship and ensure joint responsibility for project outputs.

161

• Procurement issues associated with the disposal contract prevented disposal initiation at remaining two pilot areas. Following a failed bidding process due to inadequate procurement according to World Bank procedures for the disposal services in April 2011, disposal bidding documents were rejected again by the World Bank in April 2012 due to an improper technical assessment and non-adherence of procurement requirements as spelled out in the bidding document. Based on the experience gained in other ASP countries and previous procurement failures within the framework of the project in South Africa, the World Bank considered the remaining time frame too short to successfully contract a service provider, and asked DEA to consider requesting an extension of the project’s closing date. However, such a request was sent to the World Bank only three days before actual project closing, preventing the extension process to be completed in time. The project was subsequently closed without the service provider having been contracted.


23. Overall M&E - Rating Moderately Unsatisfactory. 24. M&E design – Rating Moderately Unsatisfactory: Despite the lengthy preparation time, an M&E

system specific for South Africa was not developed at appraisal.197 A results framework, which was intended to be finalized by the TSU during project launch and included in the project’s Operational Manual, also did not materialize. Only a set of PDO indicators linked to the different project components was developed in the PAD and GEF Trust Fund Grant Agreement, however, these did not allow for adequate monitoring of project implementation progress. Their design focused on measuring outputs as opposed to progress towards project outcomes. Expected results under the prevention component, in particular, were not adequately considered by the indicators.

25. M&E implementation – Rating Moderately Unsatisfactory: M&E staffing was adequate, and progress reports were submitted timely throughout project implementation. However, they only focused on describing implementation activities and lacked an evaluation or assessment of performance. Following revision of the disposal approach that had informally been initiated in 2007, several of the indicators agreed on at appraisal had become irrelevant; they were not modified accordingly. Only in 2011, the PDO and associated indicators were officially revised as part of the Level 1 restructuring; a results framework including intermediate outcome indicators was finally introduced. However, the amendments were introduced too late as the project closed a year later.

26. M&E utilization – Rating Moderately Satisfactory: The identification and collection of obsolete pesticides during the Limpopo pilot in 2007, was conducted under application of the FAO Environmental Management Toolkit (EMTK).198 The data was, however, only captured in an Excel spreadsheet, which did not represent an adequate M&E instrument. In 2011, a booking line, where farmers voluntarily declare stocks in their possession, was tested and established in the two pilot

197 Lengthy discussions between the partners during project preparation on whether to have an M&E system at the program-level alone, or also at the country-level, may have contributed to the absence of an M&E system at the country-level at project launch. However, no information was available on whether it was a deliberate decision not to have an M&E system in place at the country-level. 198 The tool kit provides a practical set of methodologies to assist countries in the risk-based management of obsolete pesticides, including guidance on the management of collection/storage locations, based on international best practice.

162

provinces, Free State and Western Cape. The generated data was successfully used to determine waste collection points in these two provinces, which were subsequently assessed for suitability using the environmental assessment criteria outlined in the EMTK. However, a collection schedule was never developed as the disposal contractor was not recruited prior to project closing. The booking line and the government’s existing national register replaced the PSMS database, which was considered an unnecessary tool.



28. During project implementation, the World Bank continuously assessed the requirement of a CESA in view of the planned modifications to the disposal approach. In 2010, it was agreed that in the absence of a nation-wide inventory, the assessment of environmental and social risks would be based on the obsolete pesticides data from the Limpopo pilot. An EMP for the disposal of the Limpopo stocks (conducted in 2011), was prepared and reviewed by two independent reviewers. The approach was cleared by the World Bank, and recommended for disposal activities in the two remaining provinces.

29. OP 4.09 on Pest Management: Since project activities themselves met requirements of a pest management plan (i.e., through pesticide management, Integrated Pest Management (IPM), and capacity building), individual country projects, including ASP-P1 South Africa, were not required to prepare self-standing documents. See Section 2.4 in the program-level for more details.

30. Compliance with OP 4.01 on Environmental Assessment is summarized in the following: • The collection and inventory of obsolete pesticides in Limpopo province was not in full

compliance with OP 4.01 on Environmental Assessment. The pilot operation in Limpopo did not make use of the inventory taking method proposed by FAO and the Environmental and Social Management Framework because it was not considered suitable for this specific case by the partners (i.e., obsolete pesticide stocks mainly held by private farmers). The pilot operation required a different approach consisting of close cooperation between pesticide distributors and farmer cooperatives to identify, package and deliver obsolete pesticide stocks to 24 existing pesticide distribution centers. Environmental assessments were carried out for each of the distribution centers. Farmers were instructed on safely transporting the stocks and a comprehensive communications and awareness campaign was implemented.199 When the collected stocks were inventoried, it became apparent that some of the stocks were posing higher than estimated risks and that the proposed inventory procedure would not be adequate. By applying due diligence, an experienced national waste management company was subsequently recruited by DEA, to conduct the inventory and repackage the stocks for safe interim storage. Two no-objection letters were submitted to the World Bank requesting approval for the service provider; however, in an attempt to minimize exposure risks, DEA began conducting the inventory and repackaging activities before the World Bank responded to the requests.

• Disposal of obsolete pesticides from Limpopo province was conducted in compliance with OP 4.01 on Environmental Assessment. In 2010, DEA initiated the disposal of the Limpopo stocks with CLI funding. A risk assessment report and an EMP were developed, peer reviewed by DEA,

199 Awareness was raised through radio programs, press releases, publications and SMS alerts to all commercial farmers.

163

an independent consultant and the ASP-South Africa NGO, and cleared by the World Bank on November 15, 2010. Disposal activities were subsequently implemented in compliance with safeguard policies, and in accordance with all necessary protocols, including the Basel Convention guidelines. Final incineration was achieved in August 2011.200

• Disposal of obsolete pesticides in Western Cape and Free State was not conducted – therefore OP 4.01 did not apply. Following the successful independent review and implementation of the EMP during disposal activities at Limpopo, the World Bank and DEA agreed that the same approach would be used for disposal activities at the other two provinces. It was agreed that the selected disposal contractor would prepare the EMPs for the operations. However, eventually the EMPs were not prepared because the contractor had not been hired prior to project closure.

31. Financial Management compliance – Rating Moderately Satisfactory: The project submitted acceptable Interim unaudited Financial Reports (IFRs) and audit reports. However, several delays were noted, resulting from: (i) a small value of transactions at the beginning of the project, leading the World Bank to waive the first two audits (for FY07 and FY08); (ii) a long authorization process within DEA for audits; and (iii) the inability of DEA’s financial system to produce the agreed IFRs. The project did not fully comply with Financial Management (FM) arrangements; although it was supported by adequately qualified departmental staff, their availability to the project was lower than agreed.

32. Procurement – Rating Moderately Unsatisfactory: The key challenge for procurement above the shopping threshold was the reconciliation of South African procurement practices with the World Bank’s procurement guidelines. Changes in procurement staff at the beginning of the project, the lack of clearly identified procedures in the Operational Manual and the decision to start the Limpopo pilot before receiving procurement training, led the PMU to initiate procurement of services and consultants using government, rather than World Bank procurement procedures. The procurement of a service contractor for the disposal at Western Cape and Free State provinces failed twice.201 Although the World Bank had recognized the weak procurement capacity (for World Bank procedures) at appraisal, additional procurement training was not provided during project implementation.


33. Post-completion: No information could be obtained on post-completion activities. 34. Next phase: No information could be obtained on future activities.



35. Relevance – Rating Substantial: The revised PDO is relevant to South Africa's current development and global priorities. Objectives are in line with the 2010 National Pesticide Policy and the National

200 As reported by CLI, disposal certificates indicate destruction of stocks on August 22, 2011. 201 In April 2011, the bidding process failed due to inadequate procurement according to World Bank procedures for the disposal services; in April 2012, disposal bidding documents were rejected again by the World Bank due to an improper technical assessment and non-adherence with bidding document requirements.

164

Environmental Management: Waste Act of 2008; they also support the Government of South Africa in complying with the Stockholm Convention on POPs, and the Basel Convention on the control of trans-boundary movements of hazardous wastes and their disposal. The objectives are also fully in line with the World Bank's Country Partnership Strategy,202 which emphasizes the need to continue and support pollution management as part of a continent-wide program to dispose of obsolete pesticides. The engagement of the pesticide industry in discussions with government about long-term management of stocks is in line with the World Bank’s priority of supporting private-public partnerships in South Africa, and helping to remove constraints to the industry's engagement.203


36. Taking account of the project's Level 1 restructuring in June 2011, the project is evaluated against its original and revised PDO by measuring project achievements at the time of restructuring (June 2011) and project closing (June 2012), respectively. Associated project achievements, which were not adequately reflected in the results framework are also factored into the evaluation of PDO achievement.

37. Original PDO achievement – Rating Moderately Unsatisfactory. The assessment took the project’s six PDO indicators into account, of which one was fully (100%), three partly (7-80%), and two not (0%) achieved.

38. The three main project outcomes measured by indicators include: (i) sound disposal and management of obsolete pesticides; (ii) prevention of accumulation of new stockpiles of obsolete pesticides; and (iii) enhanced capacity and institutional strengthening of pesticide management.

39. Sound disposal and management of obsolete pesticides. The level of risks emanating from publicly held obsolete pesticides and associated waste to adjacent and wider communities or critical natural resources was not determined and reduced to the extent originally planned. The originally planned countrywide inventory of publicly held obsolete stocks (PDO Indicator #1 - 0%) was not carried out.204 In the absence of the originally planned inventory, preparation of a CESA was neither completed (PDO Indicator #2 – 0%). Efforts were instead focused on developing an improved approach for sound disposal and pesticide management, encompassing both, privately and publicly held obsolete pesticides. To inform the development of this approach, a pilot operation was conducted in Limpopo province in 2007, where privately held obsolete pesticides were packaged and delivered to existing pesticide distribution centers in the province. All stocks were centralized to a main storage location in the province, and repackaged for interim storage. Approximately 17 tonnes of publicly

202 Country Partnership Strategy for the Republic of South Africa for the Period 2008-2012, (Report No. 38156-ZA), December 12, 2007. 203 As explained, the original project design was not entirely relevant to the obsolete pesticides situation in South Africa, where most stocks are held by private farmers rather than the public sector; nor was it relevant to the storage infrastructure and government's existing capacity to manage obsolete pesticides. On the other hand, the revised design responded adequately to the main origin of pesticide stocks, existing infrastructure and capacity of government, as well as to the industry's capability to manage the stocks. Implementation closely followed the government's policies regarding the 'polluter pays' principle, and the 2010 Pesticides Management Policy's requirement from the industry to prepare an Integrated Waste Management Plan (IWMP). 204 It was considered cost-ineffective in an environment where most stocks were held by private farmers, and not by the public sector; in addition, application of the FAO PSMS for maintaining the inventory was not considered value adding as the government was using an existing national register to maintain pesticide data.

165

held obsolete pesticides were classified as non-hazardous waste and disposed of locally in 2009 (PDO Indicator #3 – 6.8 %), thus having reduced risks emanating from these stocks.205

40. Prevention of accumulation of new stockpiles of obsolete pesticides. Prevention activities were not implemented as envisaged at appraisal. Instead, beginning in 2010, efforts to prevent future accumulation of obsolete pesticides were implemented under the respective work programs of DAAF and AVCASA. Activities were funded directly by these two entities and were therefore implemented independently from the project. The project strengthened pesticide management by improving pesticide registration, licensing, and enforcement of import controls. The legal and regulatory framework for pesticide management was strengthened in 2010, when DAFF adopted and published a national Pesticides Management Policy, which supported the objectives of the Basel and Rotterdam conventions. Direct project support was not needed due to the government's strong engagement (PDO Indicator #4 - 100%).

41. Enhanced capacity and institutional strengthening of pesticide management. The PMU had received fiduciary training as well as training in project management. Training was also provided in preparing a CESA and using the PSMS. In addition, relevant government staff participated in a study tour to Australia, to examine the country’s pesticide collection system (PDO Indicator #5 – 80%). However, the procurement training did not strengthen the capacity of the PMU because procurement staff changed following the training, and no additional training was provided. The PSMS and CESA trainings were not useful, as these tools were not used during project implementation.

42. Revised PDO achievement – Rating Moderately Unsatisfactory. The assessment took the project’s two revised PDO indicators into account, which were both only partly achieved (55% and 33%). In addition, four Intermediate Outcome Indicators framed the evaluation, of which one was fully (100%), and three partly (33-50%) achieved. Outcomes were modified following the restructuring and included: (i) improved ability to identify, collect and dispose of obsolete pesticides and associated waste, and (ii) gained experience in removing obsolete pesticides (public and private) and associated waste.

43. Improved ability to identify, and collect obsolete pesticides and associated waste. The project improved the government’s ability to identify, collect and dispose of obsolete pesticides and associated waste through the testing and establishment of a booking line, where farmers voluntarily declare stocks in their possession (Intermediate Outcome Indicator #1 - 100%). The booking line was operated by the private sector, supervised by the government, and was used to determine the scale of the collection and treatment operations. Following its successful piloting, the system was introduced to the remaining provinces in South Africa. The system was adopted from a model used in Australia, which the government was able to examine during a study tour that was financed by CLI (Intermediate Outcome Indicator #4 - 50%). As outlined further below, private and public obsolete pesticides and associated waste were identified in three provinces, however, only collected, and disposed of in one province (Revised PDO Indicator #1 – 55%). Moreover, the booking line and its application in the two provinces informed the development of an Integrated Waste Management Plan (IWMP), which intends to dictate a private sector-run scheme for collection and disposal as stipulated by national law. The IWMP is in line with the country’s National Environmental Management Waste

205 For more details see Aide-Memoire: South Africa – Africa Stockpiles Program (ASP) – Mid-Term review, January 23 – February 6, 2009; African Stockpiles Program (ASP) South Africa – Progress Report Period March 2008 – December 2008.

166

Act of 2008, and was submitted to the DEA for adoption in June 2011. If approved, the plan is considered representing a sustainable prevention mechanism against re-accumulation of obsolete pesticide stocks; however, at ICR drafting, the plan had not yet been approved as reported by CLI. A privately run recycling scheme already existed for empty plastic pesticide containers.206

44. Through the Limpopo pilot disposal operation, experience was gained in removing obsolete pesticides (public and private) and associated waste. However, the project failed to remove obsolete pesticides in the remaining provinces, due to procurement issues and a subsequent missed opportunity to extend the project. The project reduced the risks emanating from obsolete pesticides and associated waste to communities, natural resources, and the global environment in one of the three pilot provinces, Limpopo Province. In addition to the 17 tonnes of publicly held stocks that had already been disposed of prior to restructuring, 92.5 tonnes 207 of privately held obsolete pesticides were disposed of with funds provided by CLI. A risk assessment report and an EMP had been developed for this province, peer reviewed by DEA, an independent consultant and the ASP-South Africa NGO representative, and cleared by the World Bank on November 15, 2010 (Intermediate Outcome Indicator #2 – 33%). Stocks were shipped to the UK for disposal by following all necessary protocols, including the Basel Convention guidelines, and eliminated in August 2011.208 Approximately 120 tonnes of obsolete pesticides and associated waste were identified in the two additional provinces, consisting of about 70% privately held, and 30% publicly held obsolete pesticides. However, the project did not complete the disposal and/or safeguarding of these stocks as the disposal contractor was not hired prior to project closing due to the above-mentioned procurement issues and a missed opportunity to extend the project (Revised PDO Indicator #2 – 33%; Intermediate Outcome Indicator #3 – 33%, respectively).

45. Weighted average PDO achievement – Rating Moderately Unsatisfactory:


1. Rating Moderately Unsatisfactory

Moderately Unsatisfactory -




5. Final rating - - Moderately Unsatisfactory


46. The project aimed to directly contribute to the GEO of reducing the effects of POPs on the global environment through: (i) improving the quality of life in poor communities, by reducing

206 For more information on the empty plastic pesticide container recycling scheme, please visit http://www.avcasa.co.za/. 207 These included 84 tonnes of obsolete pesticides collected at Limpopo province, 2.5 tonnes collected at Gauteng Municipality, and associated packaging materials. 208 As reported by CLI, disposal certificates indicate destruction of stocks on August 22, 2011.


167

environmental risks; (ii) improving environmental protection; and (iii) enhancing the capacity of the agricultural sector to better manage crop pests. GEO achievement is measured by PDO achievement, and, thus, also rated moderately unsatisfactory (see Section 3.2).

3.4 EFFICIENCY

47. Efficiency – Rating Low. This rating represents a broad estimate, as financial and economic data to support this rating are scarce.

48. At appraisal, an economic or financial analysis was neither undertaken for ASP-P1 at the program nor at the country-level due to non-measurable benefits of reducing the risk of contaminating the environment. Based on approximately US$15 million spent on the disposal of 3,240 tonnes of obsolete pesticides between 1990 and 2004, it was estimated that 300-900 tonnes could be disposed of per annum at the cost of US$3,400 per ton. (see Section 3.4 in the program-level ICR for details).

49. Despite the modification of the project’s design following the recognition that most obsolete pesticide stocks in South Africa were being held privately, only a limited amount of pesticides was disposed of under GEF funds by the end of the project. Actual unit cost per ton disposed of does therefore not allow for an accurate cost efficiency assessment. The project’s low efficiency rating is therefore based on the limited use of resources for disposal of obsolete pesticides in relation to lengthy project implementation. Delays were caused by a lack of implementation progress resulting from the lengthy process that was associated with adapting the project’s design to the South Africa’s country-context. About 35% of the funds were spent on the storage of the Limpopo stocks (5 years at a monthly rate of $7,660) following the government's decision not to pay for the disposal of private stocks, and the time it took to reach a financing agreement with the private sector to dispose of the collected stocks. Prevention activities eventually showed significant results with development of the 2010 Pesticide Management Policy, and capacity building efforts demonstrated an important impact in view of the dialogue with the private sector; however, these were financed by the government directly, or mostly by the private sector, respectively.


50. Overall Outcome – Rating Moderately Unsatisfactory: On the basis of the project’s efficiency (low), relevance (high), and PDO achievement (moderately unsatisfactory), the overall outcome is rated moderately unsatisfactory. However, it should be noted that the effective collaboration between the government and industry resulted in the development and submission of an IWMP. If the plan is approved by DEA as envisaged, the cleanup of obsolete pesticide stocks could be systematically guided in future; this would be a sustainable and strong outcome, which, however, goes beyond the revised objectives of the project.


51. Poverty impacts, gender aspects, and social development: The project had a direct positive impact on farming communities in Limpopo province, where obsolete pesticide stocks were collected and disposed of, thereby eliminating a prominent local development risk. Indirectly, the country’s population as a whole benefitted from a revised legal framework for pesticide management, which is expected to decrease the social burden pesticides pose on communities, females and males alike. Industry’s engagement in the dialogue ensured a more integrated approach toward future cleanup

168

operations. While the impact of improved pesticide management legislation was foreseeable, the impact of the project on the private sector (farmers and industry) was not initially foreseen given that the original project design focused on the public sector.

52. Institutional change/strengthening: While the project did not have a major impact on government’s already adequate capacity to implement clean up and prevention activities as planned, it did have a catalyzing effect on government’s regulatory role concerning pesticide management through improved policy formulation and entering into constructive dialogue with industry.

53. Other unintended outcomes and impacts: Despite its design inadequacies and implementation shortcomings, the project had an unintended impact on the way obsolete pesticides are expected to be managed in South Africa in the future, and the roles government and industry will play. The development and utilization of a booking line, a system that was adapted from Australia, demonstrated the feasibility to identify private stocks countrywide with limited government involvement and funding.209


54. No beneficiary survey was conducted and no stakeholder workshop was held. An ICR mission was planned but could not be carried out within the implementation period. The government’s completion report contained limited information and World Bank’s request for additional information did not reveal any particular issues.


55. Risk to the development outcome - Rating Moderate: DAFF and DEA, the governmental departments mandated with managing pesticides and pesticide waste, respectively, demonstrated existing good capacity to implement their mandates and continuous strong commitment to the issue. The project further increased the government's capacity by sensitizing key stakeholders to the Australian pesticide collection system, and by supporting the dialogue with the private sector industry. This experience led the pesticide industry association to develop the IWMP and submit it for approval to DEA. If approved, the IWMP will regulate responsibilities and lay out plans for future collection operations with the private sector industry playing a key implementing role.

56. However, despite the commitment and capacity of both industry and government, and the appropriate regulatory framework in place, the development outcome could be at risk if: (i) the IWMP is not approved by the minister or approved with considerable delay; (ii) industry and/or other responsible entities do not implement the IWMP or lack capacity to perform satisfactorily; and (iii) a financial mechanism is not set up or does not raise sufficient funds for a meaningful impact.

209 Government provides indemnity to stock holders who register.

169

5. ASSESSMENT OF BANK AND BORROWER PERFORMANCE


57. World Bank performance in ensuring quality at entry – Rating Moderately Unsatisfactory: Project preparation of ASP-P1 South Africa was conducted in conjunction with preparatory activities at the program-level, which were lengthy (see Section 2.1 in the program-level ICR). While considerable time was invested in overall preparation, project documents for ASP-P1 South Africa only broadly laid out implementation plans, and lacked country-specific information (see Section 2.1). Most importantly, despite the lengthy preparation process, ASP-P1 South Africa was launched with a project design that turned out to be not applicable to the South African context, as recognized in the beginning of implementation. This resulted in considerable delays, and the need to redesign the project during implementation.

58. Quality of World Bank supervision – Rating Moderately Unsatisfactory: Project supervision was carried out via missions, email and telephone exchanges. One launching mission and five supervision missions took place, averaging one mission per year. Until mid-2008 the project was intermittently supervised by the ASP Task Team Leaders (TTLs) in Washington DC and co-TTLs based in the Pretoria World Bank Office. Beginning mid-2008 until project closing, the project was not assigned a co-TTL (unlike all other ASP projects) and was supervised by the ASP TTL from Washington DC. This may explain the infrequency of World Bank missions to the project. Procurement and FM were supervised by World Bank staff based in Pretoria: FM supervision was carried out in all missions, while procurement supervision was conducted only in the first two missions (in June 2006 for training and March 2007). One official post procurement review was carried out in May 2011. Safeguards were supervised by staff from the Pretoria and Washington DC offices during all missions except the mid-term review in February 2009 (due to unavailability of a safeguard specialist to join the mission). During the lifetime of the project, the project was supervised by six different TTLs/co-TTLs.

59. Despite early recognition of the need to modify the design, the project was only restructured in 2011. During the first five years of the project, project activities were not implemented according to the provisions of the GEF Trust Fund Grant Agreement; amended activities such as the piloted collection, storage and eventual disposal of obsolete stocks in Limpopo province were conducted without the necessary M&E framework in place to monitor the activities’ performance or progress.

60. Overall World Bank performance – Rating Moderately Unsatisfactory: Based on the above presented evaluation, the overall borrower performance is rated moderately unsatisfactory.


61. Government performance – Rating Satisfactory: Government performance was overall satisfactory throughout implementation as reflected by its commitment to the issue. Government commitment at entry continued to be strong as was evident from its strong engagement during the Limpopo pilot. Both DEA and DAFF engaged in a close dialogue with AVCASA and CropLife South Africa, and an effective public-private partnership was formed. This relationship was later maintained and the government continued to engage South Africa’s private sector industry in agreed follow up activities. Government approval of the National Pesticide Policy in 2010 demonstrated continued commitment to the issue of pesticide management. DAFF’s decision to implement prevention activities

170

independently from the project affected project performance; however, this seems to have been the result of administrative difficulties between the project and DAFF and not due to a lack of commitment.

62. Implementing Agency performance – Rating Moderately Unsatisfactory: During project preparation DEA collaborated well with the World Bank and other ASP partners. DEA prepared a comprehensive Operations Manual, which provided considerable information considering the lack of key baseline data. However, a staff turnover at the beginning and at the end of the project delayed project activities; in addition, the misplacement of legal documents in one occasion, lengthy authorization procedures within and outside the department affected timely submission of audits, and countersigning of amendments. DEA executed its fiduciary duties, however, not in full compliance with World Bank legal covenants. As previously described, the bidding process for disposal failed twice due to improper procurement. In addition, audits and IFRs were often submitted delayed. However, DEA provided co-financing and in-kind support to the project as planned.

63. Overall borrower performance – Rating Moderately Unsatisfactory: Taking into account that the project’s overall outcome is rated moderately unsatisfactory, and the implementation agency’s performance often delays implementation progress, overall borrower performance is rated moderately unsatisfactory.

6. LESSONS LEARNED

64. The most significant lessons learned are outlined in Section 6 of the program-level ICR, as the experience they are based on was often reflected across all country-specific projects. Lessons learned that were particularly relevant to ASP-P1 South Africa include:

• Future projects should place particular emphasis on tailoring individual country projects to the country-specific environments. The uniform approach ASP-P1 was applying to all participating countries was not appropriate for the South African context, and caused significant delays and transaction costs.

• Private-public partnerships are important for the collection and disposal of privately-held obsolete pesticides. In countries which hold significant amounts of privately-held stocks should seek to form private-public partnerships between the relevant governmental agency and the private sector, and include private stocks in the disposal and prevention components.



171

8. ANNEXES


a. Project Cost by Component – All sources (in US$ million equivalent) – as of 05/15/2013 Component

Appraisal Estimate (in US$ million equivalent)




5. Disposal of obsoletes pesticides 0.97 N/A210 N/A




Total 1.80 0.71 39%


Co-financing


equivalent)

Restructuring Estimate

June 2011 (in US$ million

equivalent)



Borrower Counterpart Funding 0.07 0.70 N/A N/A

Global Environment Facility (GEF) Grant 1.73 1.70 N/A N/A

CropLife International (CLI) Parallel Financing 0 0.51 N/A N/A

Total 1.80 2.91 0.71 39%


172





Part A – Obsolete pesticides inventory: Achievements under this component are rated Unsatisfactory Carrying out of a detailed inventory of the recipient’s publicly held obsolete pesticides stocks through: a. Collection of data and compilation of a



Most of the originally envisaged project activities were not applicable to the South African context, and, therefore, could not be implemented as planned. a. Carrying out an inventory was considered cost-ineffective in an

environment where most stocks were held privately; in addition, the government was using an existing national register to maintain pesticide data (Original PDO Indicator #1 – 0%).

b. A risk-based prioritization was subsequently not possible. c. In 2010, it was agreed that in absence of a nation-wide inventory, a

CESA would not be the appropriate safeguard tool, as it required inventory data for detailed analysis and prioritization (Original PDO Indicator #2 – 0%).



disposal technology selected; and c. Improving the operations of obsolete pesticide


a. The technological assessment study was launched. It was, however, only finalized after the restructuring; therefore, this activity was not completed.

b. Obsolete pesticides classified as non-hazardous waste under national and WHO regulations (17 tonnes) were disposed of locally in 2009 (Original PDO Indicator #3 – 6.8%). The hazardous portion of the stocks was not disposed of by the time of the restructuring.

c. The need for improving the operations of obsolete pesticide stocks collection centers, including improving safety measures at those centers, was not reviewed at appraisal, as large pesticide stores in South Africa already had to meet minimum safety standards which met the guidelines established by FAO.

Part C – Prevention of obsolete pesticide accumulation: Achievements under this component are rated Moderately Satisfactory Carrying out of activities at preventing obsolete pesticide accumulation, including: a. Awareness raising activities on the health and

environmental hazards of pesticides; b. A review and update of the legal and

regulatory framework to control obsolete pesticide accumulation and enforcement mechanisms;

c. Preparation and implementation of an IPM

a. The South African NGO Network prepared and distributed literature on the appropriate handling of pesticides. Furthermore, NGOs conducted community monitoring and awareness-raising activities in Kwa-Zulu province where pesticides had been implicated wildlife-poisoning incidents. Awareness raising activities were also conducted in preparation of collection activities as part of the Limpopo pilot.

b. Regulatory framework gaps analysis was carried out by DAFF's legal unit. DAFF used the gap analysis to draft the required

Table II.1: Outputs by Original Components as presented in the GEF Trust Fund Grant Agreement

173

strategy; and d. A review of pesticide management practices

and preparation and implementation of a training plan.

amendment and revisions to an existing policy. In 2010 a National Pesticide Policy was approved by the government and published (Original PDO Indicator #4 - 100%).

c. Since IPM did not represent an official government policy, it was not integrated into DAFF's prevention work program. Moreover, the IPM budget allocation was deemed too small to have a meaningful impact.

d. No training plan for the improvement of pesticide management practices was prepared; however, during the course of implementation the PMU received PSMS and CESA training and participated in a study tour to Australia to learn about a local system for stocks collection via a public-private partnership. (Original PDO Indicator #5 - 80%)

Part D – Capacity building: Achievements under this component are rated Moderately Satisfactory. Strengthening the capacity of the PMU to implement the project through the provision of training.

The PMU was trained on procurement, financial management, CESA preparation, project management and PSMS (Original PDO Indicator #6 – 80%). The procurement training did not strengthen the capacity of the PMU because procurement staff changed after the training and no additional procurement training was provided. The PSMS and CESA training were not useful as these tools were not used during project implementation in South Africa. They were provided as part of the ASP-wide training by FAO and the World Bank, respectively.

Part E – Project Management: Achievements under this component are rated Moderately Satisfactory. Supporting the PMU in carrying out project coordination, implementation, monitoring and evaluation, through the provision of technical advisory services, equipment and operational support.

DEA received equipment and funding for operational support; a TAD was hired by CLI and was housed in DEA for 2 years (2006-2008); and the above-mentioned training was provided to the PMU.



Part A – Identification and collection of obsolete pesticides and associated waste: Achievements under this component are rated Moderately Unsatisfactory.

211 Restructuring Paper on a Proposed Restructuring of the Africa Stockpiles Program to the Republic of South Africa, (Report No: 62795-ZA), June 20, 2011

Table II.2: Outputs by Revised Components as presented in the Level 1 Restructuring June 30, 2011

174

Identification and collection of obsolete pesticides and associated waste by: a. Establishing a booking line where farmers in

selected provinces voluntarily declare obsolete pesticides and associated waste in their possession;

b. Preparing EMPs; and c. Collecting and transporting identified obsolete

pesticides and associated waste to collection centers.

a. A booking line was established and piloted for Western Cape and Free State provinces, before it was expanded to additional provinces in 2012. The booking line was successful in identifying 120 tonnes of obsolete pesticides and associated waste from the public sector (30%) and private sector (70%) (Intermediate Indicator #1 – 100%).

b. A risk assessment report and an EMP had been developed for Limpopo province, peer reviewed by DEA, an independent consultant and the ASP-South Africa NGO representative, and cleared by the World Bank on November 15, 2010. No EMPs were prepared for the two additional provinces because the disposal contractor was not recruited prior to project closing (Intermediate Outcome Indicator #2 – 33%).

c. Obsolete pesticides and associated waste were collected and transported to collection centers in Limpopo province, during a pilot operation in 2007. Collection of identified obsolete pesticides in Free State and Western Cape provinces was not conducted as the disposal contractor was not recruited prior to project closing (Intermediate Outcome indicator #3– 33%).

Part B – Safeguarding and/or disposal of obsolete pesticides and associated waste: Achievements under this component are rated Moderately Unsatisfactory. Safeguarding and/or disposal of obsolete pesticides stocks and associated waste by: a. Carrying out of a technological assessment of

available safeguarding and disposal options; and

b. Implementing the safeguarding and/or disposal technology selected in selected provinces.

a. A study on disposal technology options was finalized in early 2012. The study involved identifying South Africa’s waste profile, identification of environmentally sound waste management technologies available and benchmarking them with the best international technologies.

b. The hazardous portion of the Limpopo stocks (92.5 tonnes) was shipped for incineration to the UK, and disposed of on August 22, 2011. A certificate of incineration was received as reported by CLI. The disposal of obsolete pesticides at the two remaining pilot provinces, Western Cape and Free State, was not completed as the disposal contractor had not been recruited prior to project closing (Revised PDO Indicator #2 – 33%).

Part C – Capacity building: Achievements under this component are rated Moderately Satisfactory Carrying out of activities aimed at raising awareness on the health and environmental hazards of pesticides and strengthening the capacity of the PMU to implement the project through the provision of training.

See Outputs by Original Components C, D and E.

Part D – Project management: Achievements under this component are rated Moderately satisfactory Supporting the PMU in carrying out project coordination, implementation, monitoring and evaluation through the provision of technical advisory services, equipment and operational support.

See Outputs by Original Component E. A new TAD was not recruited to support the PMU; however, the TSU supported the preparation of the disposal bidding documents.



175

8.4 WORLD BANK LENDINGA AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS



Names Specialization Unit Responsibility/Specialty Abushakra, Hadi Chief Legal Counsel LEGMS Adebowale, Modupe Senior Finance Officer LOAG2 Allan, Christine Operations Analyst MNSRE Bakayoko, Siaka Senior Financial Management

Specialist MNACS

Brackmann, Stefanie Environmental Specialist MNSRE Chummun, Zakia Language Program Assistant MNSRE Di Leva, Charles Chief Counsel LEGEN Ebro, Kristyn Communications Officer ESDVP Gorman, Steve Lead Environmental Specialist ENVGC Kaniaru, Muthoni Counsel LEGAF Kranz, Frederick Senior Procurement Specialist MNACS Krishnakumar, V.S. Manager ASPTC Lagnaoui, Abdelaziz Senior Environmental Specialist ESDQC Lotayef, Dahlia Senior GEF Operations

Coordinator MNSRE


Mani, Angeline Language Program Assistant MNSRE Melkonian, Hovsep Senior Finance Officer LOAG2 Newton, Murray Consultant ENVMP Olowo-Okere, Edward Lead Financial Management

Specialist AFTFM

Opsal, Knut Senior Social Development Specialist

MNSRE

Parish, Matthew T. Counsel LEGMS Tynan, Ellen Environmental Specialist ENVMP Warner, Christopher Senior Environmental Specialist AFTS1

Supervision/ICR Names Specialization Unit Responsibility/Specialty

Aka, Henri Procurement Specialist


Baimu, Evarist Senior Counsel

Bouzaher, Aziz Lead Environmental Specialist Co-TTL


Butler, Yesmeana Program Assistant

176

Chitalu, Fenwick Financial Management Specialist

da Silva, George Disbursement Specialist

Gxaba, Thandiwe Senior Environmental Specialist

Harlley, Jemima Program Assistant



Maber, Steve Senior Operations Officer TTL

McCue, Theresa Disbursement Analyst

Morita, Sachiko Counsel

Mphande, Donald Financial Management Specialist

Mpundu, Marjorie Senior Counsel

Msiwa, Tandile Gugu Financial Management Specialist Nieuwoudt, Christiaan Johannes

Finance Analyst

Odendaal, Erika Program Assistant

Peled, Ayala Environmental Economist Consultant



Consultant

Schmidt, Veruschka Strategy Officer CFRPA

Tete, Patrick Umah Senior Financial Management Specialist


b. Staff time and cost – Staff time and costs associated with project preparation and supervision was not possible to determine as a result of poor recording in the system.






5. The recipient submitted a completion report; messages contained therein were integrated into this document.

6. The draft ICR was shared with the borrower; however, no comments were received.

8.8 COMMENTS OF COFINANCIERS AND OTHER PARTNERS/STAKEHOLDERS

7. No comments were received from co-financiers and other partners/stakeholders on the ASP-P1 South Africa draft ICR.

177


178

TECHNICAL ANNEX V: ASP-P1 TANZANIA

179


AfDB African Development Bank






DDT Dichlorodiphenyltrichloroethane

DNOC Dinitro-o-Cresol

DoE Division of Environment


EMTK Environmental Management Tool Kit





GIZ Deutsche Gesellschaft für Internationale Zusammenarbeit

IMDG International Maritime Dangerous Goods

IPCS International Program on Chemical Safety




MAFSC Ministry of Agriculture, Food Security, and Cooperative

MoA Ministry of Agriculture

NEMC National Environment Management Council


NSAF National Stakeholders Advisory Forum



PCT Project Coordination Team


180



SIDA Swedish International Development Cooperation Agency


TORs Terms of Reference




UNITAR United Nations Institute of Training and Research


181

TABLE OF CONTENTS

1. PROJECT CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES AND DESIGN ...... 190 1.1 CONTEXT AT APPRAISAL ............................................................................................................................................ 190 1.2 ORIGINAL PROJECT DEVELOPMENT OBJECTIVE AND INDICATORS ................................................................ 191 1.3 REVISED PROJECT DEVELOPMENT OBJECTIVE AND INDICATORS .................................................................. 192 1.4 MAIN BENEFICIARIES................................................................................................................................................... 193 1.5 ORIGINAL AND REVISED COMPONENTS ................................................................................................................. 193 1.6 OTHER SIGNIFICANT CHANGES ................................................................................................................................. 194

2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES ..................................................................... 195 2.1 PROJECT PREPARATION, DESIGN, AND QUALITY AT ENTRY ............................................................................ 195 2.2 IMPLEMENTATION ........................................................................................................................................................ 196 2.3 MONITORING AND EVALUATION (M&E) ................................................................................................................. 198 2.4 SAFEGUARD AND FIDUCIARY COMPLIANCE ......................................................................................................... 198 2.5 POST-COMPLETION OPERATION/NEXT PHASE ...................................................................................................... 200

3. ASSESSMENT OF OUTCOMES ......................................................................................................................................... 200 3.1 RELEVANCE OF OBJECTIVES, DESIGN, AND IMPLEMENTATION ...................................................................... 200 3.2 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVE ................................................................................... 201 3.3 ACHIEVEMENT OF GLOBAL DEVELOPMENT OBJECTIVE ................................................................................... 204 3.4 EFFICIENCY .................................................................................................................................................................... 204 3.5 JUSTIFICATION OF OVERALL OUTCOME RATING ................................................................................................. 205 3.6 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS ............................................................................ 205 3.7 SUMMARY OF FINDING OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS ...................... 206




8.1 PROJECT COSTS AND FINANCING ............................................................................................................................. 208 8.2 OUTPUTS BY COMPONENT ......................................................................................................................................... 209 8.3 ECONOMIC AND FINANCIAL ANALYSIS .................................................................................................................. 213 8.4 WORLD BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS ................................. 213 8.5 BENEFICIARY SURVEY RESULTS .............................................................................................................................. 214 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS ............................................................................................ 214 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR ................................................................. 215 8.8 COMMENTS OF CO-FINANCIERS AND OTHER PARTNERS/STAKEHOLDERS ................................................... 216 8.9 VERBATIM COMMENTS ON DRAFT ICR ................................................................................................................... 217

182


Country: Tanzania Project Name: Africa Stockpiles Program - Project 1 Project ID: P103189 L/C/TF Number: TF56693 ICR Date: 06/28/2013 ICR Type: Core ICR Lending Instrument: APL Borrower: United Republic of Tanzania Original Total Commitment:

US$6.87 million


Revised Amount: - Environmental Category: A Global Focal Area: CHEM Implementing Agency: National Environment Management Council (NEMC) Co-financiers and Other External Partners: Government of the United Republic of Tanzania, Food and Agriculture Organization (FAO), CropLife International (CLI), World Wildlife Fund (WWF), and Pesticide Action Network (PAN-UK and PAN-Africa).

B. Key Dates

Process Date Process Original Date Revised / Actual Date(s) Concept Review: 07/17/2002 Effectiveness: 02/12/2007 12/19/2006 Appraisal: 04/19/2004 Restructuring: 06/30/2011 Approval: 12/05/2006 Mid-term Review: 08/31/2008 10/16/2009

Closing: 06/30/2010 12/31/2011; 12/31/2012; 05/31/2013.

C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Global Environment Outcome Moderate World Bank Performance: Moderately Unsatisfactory Borrower Performance: Satisfactory C.2 Detailed Ratings of World Bank and Borrower Performance World Bank Ratings Borrower Ratings Quality at Entry: Moderately Unsatisfactory Government: Satisfactory

Quality of Supervision: Moderately Satisfactory Implementing Agency/Agencies:

Satisfactory

Overall World Bank Moderately Unsatisfactory Overall Borrower Satisfactory


183

Performance: Performance:


E. World Bank Staff

Positions At ICR At Approval Vice President: Makhtar Diop Gobind T. Nankani Country Director: Philippe Dongier Mark D. Tomlinson Regional Integration Director Colin Bruce -

Sector Director - Michel Wormser Sector Manager: Magda Lovei Mary Barton-Dock Project Team Leader: Dinesh Aryal Peter Kristensen/Denis Jordy ICR Team Leader: Dinesh Aryal ICR Primary Author: Veruschka Schmidt F. Results Framework Analysis - At project approval, no country-specific results framework existed; only six PDO Indicators were defined. Following a Level 1 Restructuring in June 2011, a country-specific results framework was introduced. The original six PDO Indicators were revised and reduced to three PDO Indicators, and in addition, six Intermediate Outcome Indicators were introduced.



-






Satisfactory

184

Project Development Objective (PDO) – as presented in the GEF Trust Fund Grant Agreement:213 To assist the recipient in: (a) eliminating inventoried publicly held obsolete pesticide stocks and associated waste; and (b) implementing measures to reduce and prevent future related risks. Revised PDO – as presented in the Level 1 Restructuring of June 2011 (Report No: 62796-TZ): Assist the recipient in: (i) disposing and/or safeguarding inventoried publicly held obsolete pesticide stocks and associated waste; and (ii) developing a strategy for sustainable management of future accumulations. Global Environment Objectives (GEO) - as presented in the PAD (Report No: 36273-AFR): Reduction of the effects of Persistent Organic Pollutants (POPs) on the global environment. Revised GEO – Not Applicable (a) PDO Indicators – Original PDO Indicators as presented in the GEF Trust Fund Grant Agreement are assessed against achievements realized at the time of the project’s Level 1 restructuring in June 2011; revised PDO Indicators are assessed against achievements realized at project closing in May 2013.





Values


Years


Indicator 1: Inventory database of publicly held obsolete pesticides stocks in place and being used by the Project Coordination Team (PCT) and the recipient’s government staff.


Date Achieved 04/19/2004 06/30/2010 - 11/20/2009


Target fully achieved (100%): An inventory was completed and validated in 2009, and data was entered into the FAO web-based Pesticide Stock Management System (PSMS). The database was subsequently used for site prioritization, preparation of the CESA, and disposal activities.

Indicator 2: Completion of Country Environmental and Social Assessment (CESA), and implementation of the measures contained therein.



Target not achieved (0%): A CESA was still being finalized and not completed at restructuring.

Indicator 3: Inventoried publicly held obsolete pesticides stocks disposed of, as a result of the


185

completion of the disposal services contract(s) in accordance with national and international laws and regulations.

Value (quantitative or qualitative) 0% 100% - 0%

Date Achieved 04/19/2004 06/30/2010 - 06/30/2011


Target not achieved (0%): A draft disposal tender document had been prepared and submitted to the World Bank for review in May 2011. None of the inventoried publicly held obsolete pesticide stocks had been disposed of at restructuring.

Indicator 4: Legal and regulatory framework for pesticide management improved, including measures to strengthen compliance with the Conventions.



Target not achieved (0%): The legal and regulatory framework for pesticide management had not been improved by the time of restructuring.

Indicator 5: Integrated Pest Management (IPM) strategy is developed and implemented. Value (quantitative or qualitative)

No Yes - No

Date Achieved 04/19/2004 06/30/2010 - 06/30/2011


Target not achieved (0%): An IPM strategy was neither developed nor implemented, because in 2010, it was agreed by the PMU and the World Bank that instead of developing an IPM strategy, a national workshop on IPM would be held. However, the workshop had not been held by the time of restructuring.

Indicator 6: The training program under Part D214 of the project is implemented and the knowledge so acquired is being used by the recipient.


No Yes - Yes

Date Achieved 04/19/2004 06/30/2010 - 06/30/2011


Target fully achieved (100%): PMU members received training on project management, conducting an inventory and PSMS, preparing a CESA, safeguards, storekeeping, and procurement. The acquired knowledge was continuously applied.


Indicator 1: Quantity of inventoried publicly held obsolete pesticides and associated waste disposed of and/or safeguarded.


0 700 tonnes (gross) 665 tonnes (net)

- 575 tonnes (net)

Date Achieved 06/30/2011 05/31/2013 - 05/31/2013


Target fully achieved (100%): 575 tonnes (net) of the inventoried 665 tonnes (net) of publicly held obsolete pesticides and associated waste were disposed of. The original quantity of 665 tonnes of obsolete pesticides turned out to be an overestimate, and


186

the actual value was 575 tonnes. Thus, with disposal of 575 tonnes, the target was fully achieved.

Indicator 2: Quantity of inventoried heavily contaminated soil disposed of and/or safeguarded.

Value (quantitative or qualitative) 0 300 tonnes (net) - 392 tonnes (net)


Target exceeded (131%): 392 tonnes (net) of the inventoried 300 tonnes (net) of heavily contaminated soil were disposed of.

Indicator 3: Comprehensive strategy for dealing with future stocks of obsolete pesticides and associated wastes adopted by the steering committee.


No strategy in place Strategy finalized and adopted by the Steering Committee

- Strategy finalized and adopted by the Steering Committee

Date Achieved 06/30/2011 05/31/2013 - 05/31/2013


Target fully achieved (100%): The comprehensive strategy included action plans on pesticide management, IPM, and container management, finalized and adopted by the Steering Committee on April 22, 2013; and draft legislation for a Plant Protection Bill and a Pesticide Management Bill, submitted to the Ministry of Agriculture for review.





approval documents)




Indicator 1: Validated inventory data entered on Pesticide Stock Management System (PSMS)

Value (quantitative or qualitative) Yes - - Yes

Date Achieved 06/30/2011 - - 11/20/2009


Indicator corresponds to original PDO Indicator #1 and was completed prior to restructuring. Therefore, the indicator is not taken into account for post-restructuring assessment.

Indicator 2: Country Environmental and Social Assessment (CESA) completed Value (quantitative or qualitative)

No Yes - Yes

Date Achieved 06/30/2011 05/31/2013 - 06/27/2012 Comments (incl. % Indicator corresponds to original PDO indicator #2. Target fully achieved (100%): A

187

achievement) CESA was completed on June 15, 2012, and disclosed to the World Bank InfoShop on June 27, 2012.

Indicator 3: Four communications actions implemented Value (quantitative or qualitative) 0 4 - 4

Date Achieved 06/30/2011 05/31/2013 - 05/31/2013


Target fully achieved (100%): Awareness was raised through: (i) two TV spots aired in 2013; (ii) two radio spots aired in 2013; (iii) participation in five exhibitions; (iv) outreach visits to three zones in Tanzania.

Indicator 4: A final draft container management plan submitted for endorsement by the Steering Committee


No. A study conducted and

reviewed. Yes - Yes


Target fully achieved (100%): A container management plan was drafted, and endorsed by the Steering Committee on April 22, 2013.

Indicator 5: A final draft sustainability roadmap developed and submitted for endorsement by the Steering Committee


No Yes - Yes

Date Achieved 06/30/2011 05/31/2013 - 04/22/2013


Target fully achieved (100%): A draft sustainability roadmap was finalized and endorsed by the Steering Committee in June 2010. The roadmap identified key steps towards sustainable pesticide management in Tanzania.

Indicator 6: Three training and/or field visits implemented on improved pesticide management practices

Value (quantitative or qualitative) 0 3 - 3


Target achieved (100%): Field visits were conducted during disposal activities to three different zones to raise awareness on improved pesticide management practices.

G. Ratings of Project Performance in ISRs – Some DO and IP ratings and actual disbursement data are not available because ISRs reported on ASP-P1 as a whole, and only occasionally on country-specific project performance, including disbursement.



1 10/31/2005 Not applicable as ASP-P1 Tanzania not yet launched

2 06/30/2006

188

3 12/27/2006 4 06/28/2007 Moderately Satisfactory Moderately Satisfactory - 5 12/18/2007 - - - 6 05/30/2008 - - - 7 12/18/2008 Moderately Unsatisfactory Unsatisfactory - 8 03/25/2009 Moderately Satisfactory Moderately Satisfactory - 9 12/29/2009 Moderately Satisfactory Moderately Satisfactory 0.82 10 06/29/2010 Moderately Satisfactory Satisfactory - 11 03/27/2011 - Satisfactory - 12 12/25/2011 Moderately Satisfactory Satisfactory - 13 06/29/2012 Satisfactory Satisfactory 2.73 13-1 12/09/2012 Satisfactory Satisfactory - 13-2 05/29/2013 Satisfactory Satisfactory - 05/31/2013 Final disbursement 6.87 H. Restructuring


Board Approved

PDO Change



in USD millions


PDO IP


Not Available

Not Available

• Extension of closing date by 18 months to compensate for delays (06/30/2010 to 12/31/2011).

06/30/2011 Yes Moderately Satisfactory

Satisfactory 2.1 • Revision of the PDO and indicators to reflect a shift from complete disposal to disposal and safeguarding of stocks



• Reallocation of proceeds to accommodate field activities, training and workshops related to the prevention component.


Not Available

Not Available

• Extension of closing date by five months to allow time for the shipment of stocks to the incineration facilities thus avoiding contamination on transit; and

189


Board Approved

PDO Change



in USD millions


PDO IP

reception of certificates of disposal from the incineration facilities to enable payment and avoid reputational risks. (12/31/2012 to 05/31/2013).

• Reallocation of proceeds to maximize the amount of hazardous materials to be disposed of.

If PDO and/or Key Outcome Targets were formally revised (approved by the original approving body) enter ratings below:


Against Original PDO targets Unsatisfactory Against formally revised PDO targets Highly Satisfactory Overall (weighted) rating Satisfactory I. Disbursement Profile – A disbursement profile is not available (see explanation under G).

190



1. Country background: Tanzania’s agriculture sector had been highly labor-intensive, employing an estimated two-thirds of the country’s workforce. At project approval in December 2006, the sector accounted for just under 50% of GDP, with considerable under-utilized land, and the land that was in use, had been dominated by small-scale subsistence farmers.215

2. Sector background: Following their introduction in the 1940s and 1950s, pesticides in Tanzania had been primarily used in agriculture for crop protection, as pests accounted for more than 40% of crop losses annually. The launching of the economic recovery programs and liberalization of trade in Tanzania had resulted in a rise of the amount of pesticides imported into the country from 500 tonnes in 2000 to 2,500 tonnes in 2003. To a lesser extent, pesticides had been used in the public health sector and other areas including for protecting buildings from insect pests.

3. Obsolete pesticide accumulation: Tanzania had accumulated large stockpiles of obsolete pesticides as a result of incorrect needs assessment and procurement; uncoordinated donations of pesticides; poor storage conditions; and poor stock management.

4. Institutional background: Pesticide legislation was updated in 1997 to reflect international agreements.216 The Ministry of Agriculture, Food Security, and Cooperative (MAFSC) was responsible for approval and registration of pesticides, while the National Environmental Management Council (NEMC) represented the principal agency in responsible for regulating the environment, including chemical waste and disposal. The Division of Environment (DoE) – Vice Presidents’ Office (under which NEMC was situated) had the overall function of formulating policies on environment, coordination and monitoring environmental issues, environmental planning and carrying out policy oriented environmental research. At approval, the importation and distribution of pesticides was handled by the private sector, while the Government concentrated on regulating and controlling the chain. Although some of the pesticides used in agriculture were similar to those used in public health, pesticides for animal husbandry and public health were regulated under the Ministry of Livestock Development and Ministry of Health, respectively.217 The Office of the Vice-President, and the ministries had adopted legal and regulatory provisions to improve the management of pesticides and to prevent impacts on human health and the environment.218

5. Actions taken: Considerable efforts had been undertaken by the government to safeguard and dispose of obsolete pesticides,219 before in 1997-1998, the Government of the Netherlands supported

215 The Economist Intelligence Unit, Country Profile Tanzania, 2006 216 The Plant Protection Act was enacted in 1997, dealing specifically with plant protection and pesticide control. Regulations were put in place in 1999 but enforcement of the Act was weak. 217 Pesticide and Poverty – A Case Study on Trade and Utilization of Pesticides in Tanzania: Implication to Stockpiling, Agenda for Environment and Responsible Development in collaboration with Pesticide Action Network UK, August 2006. 218 These included the Environmental Management Act (2004), which included a provision for the development of a regulation for hazardous waste management, the Occupational Safety and Health Act (2003), which ensured prevention to workers exposure to chemical and pesticide substance, and the Industrial and Consumer Chemical Act (2003). 219 In example, in 1986, the Swedish International Development Cooperation Agency (SIDA) provided assistance to repack large quantities of pesticides at the coastal region; in 1995, the Government of the Netherlands funded the removal of hazardous waste

191

Table 1.1: ASP-P1 Tanzania Original PDO

Tanzania in carrying out a nationwide inventory of obsolete stockpiles, which identified about 1000 tonnes of obsolete pesticides. 70% of the pesticides were estimated to be publicly held and included large volumes of organ-chlorinated products, notably Dichlorodiphenyltrichloroethane (DDT).220 The majority of the stores did not meet the basic requirements for environmentally sound storage of hazardous chemicals as set by the Food and Agriculture Organization (FAO) and the United Nations Environment Program (UNEP). Several other activities were subsequently undertaken. These included an inventory that was organized in collaboration with UNEP to assess status of specifically POPs in Tanzania, a capacity-building project on risk management decision-making undertaken by the Ministry of Health221, and Integrated Pest Management (IPM) and Integrated Vector Management (IVM) programs organized by the Ministry of Agriculture.222 In recognition of the problem identified by this inventory, Tanzania made an official appeal for assistance to the international donor community during the FAO Third Consultation on Prevention and Disposal of Obsolete Pesticide Stocks, held in Rome in March 1998. Tanzania was subsequently considered for ASP participation, and a country workshop was held in August 2002 in Dar es Salaam to discuss and evaluate the scope of the pesticide problem.

6. Rationale for World Bank assistance: Please refer to Section 1.1 in the program-level ICR. 7. Higher-level objectives: The project intended to support Tanzania’s Country Assistance Strategy

(CAS)223 to help realizing an environmentally sustainable rural development strategy, and support commercial agriculture to become more dynamic and productive. The project further aimed to assist the government in implementing policies focusing on the conservation of natural resources and management of the environment, and raising awareness for the sustainable use of natural resources. Contributing to international efforts to eliminate POPs, improving management of toxic chemicals, and promoting alternatives to pesticide usage that include IPM and IVM represented further objectives.


8. The Project Development Objective (PDO), indicators, and components that are stated in the ASP-P1 Tanzania Project Appraisal Document (PAD)224 differ from those stated in the GEF Trust Fund Grant Agreement.225 This ICR bases its evaluation on the PDO, indicators, and components presented in the GEF Trust Fund Grant Agreement, which served as the guiding document for the project’s implementation activities.

Original PDO as presented in the PAD Original PDO

from the islands of Zanzibar and Pemba; also in 1995, the Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) provided technical assistance to incinerate 57 tonnes of Dinitro-o-Cresol (DNOC) in a cement kiln in Dar es Salaam. 220 Dichlorodiphenyltrichloroethane (DDT) 221 With support of the United Nations Institute of Training and Research (UNITAR) and the International Program on Chemical Safety (IPCS) 222 With support of the GIZ. 223 Country Assistance Strategy for the United Republic of Tanzania, (Report No. 16554-TA), May 6, 1997 224 Project Appraisal Document for the Republic of Mali, the Kingdom of Morocco, and the United Republic of Tanzania in support of the Africa Stockpile Program – Technical Annex 3, (Report No. 36273-AFR), November 7, 2006 225 GEF Trust Fund Grant Agreement between the United Republic of Tanzania and International Bank for Reconstruction and Development, (Number TF056693-TA), 2006

192

Table 1.2: ASP-P1 Tanzania Original PDO Indicators

as presented in the GEF Trust Fund Grant Agreement To assist the Government of Tanzania to dispose of publicly held obsolete pesticide stocks at priority-selected sites and to support priority actions to help prevent the future accumulation of new stocks of obsolete pesticides.

To assist the recipient in: (a) eliminating inventoried publicly held obsolete pesticide stocks and associated waste; and (b) implementing measures to reduce and prevent future related risks.

9. At appraisal, only an overall ASP-P1 results framework was prepared (see program-level ICR); results frameworks for the individual ASP-P1 countries, including Tanzania, were intended to be finalized during the individual ASP-P1 country project launches. However, country-specific PDO indicators were developed at appraisal. For ASP-P1 Tanzania, these were as follows:

Original PDO Indicators as presented in the PAD and the GEF Trust Fund Agreement a. Inventory database of publicly held obsolete pesticides stocks in place and being used by the Project Coordination

Team’s (PCT) and the recipient’s government staff. b. Completion of Country Environmental and Social Assessment (CESA) and implementation of measures contained

therein. c. Inventoried publicly held obsolete pesticides stocks disposed of, as a result of the completion of the disposal services

contract(s) in accordance with national and international laws and regulations. d. Legal and regulatory framework for pesticide management improved, including measures to strengthen compliance with

the Conventions. e. Integrated Pest Management (IPM) strategy is developed and implemented. f. The training program under Part D of the project is implemented and the knowledge so acquired is being used by the

recipient.226


10. The PDO and associated indicators for ASP-P1 Tanzania were revised through a Level I restructuring on June 30, 2011.227 The original objective was based on the incorrect assumption that project funds would be sufficient to dispose of all inventoried stocks, and that demand for pesticides could be substantially reduced through project-funded prevention activities despite evidence of increasing demand for pesticides as a result of agricultural intensification. After it became evident that there were higher than estimated volumes of associated waste, particularly contaminated soils, and that future stocks could not be fully prevented, the original PDO, indicators and related components were revised to reflect a shift from complete elimination of all inventoried publicly-held obsolete stocks and associated waste to elimination overseas and lower-cost in-country safeguarding of the stocks. This also reflected a transition from prevention of accumulation of stocks to the development of a strategy to deal with future obsolete pesticides and associated waste re-accumulation, endorsed by the government.

226 See Section 1.5 for the project’s Component D, here referred to as “Part D”. 227 Revisions were approved on June 30, 2011, on the basis of: Restructuring Paper on a Proposed Project Restructuring of Africa Stockpiles Program – Project 1 to the United Republic of Tanzania, (Report No: 62796-TZ), June 20, 2011

193

11. ASP-P1 Tanzania’s revised PDO: Assist the recipient in: (i) disposing and/or safeguarding inventoried publicly held obsolete pesticide stocks and associated waste; and (ii) developing a strategy for sustainable management of future accumulations.

12. ASP-P1 Tanzania PDO indicators were reduced in number and revised to focus on the updated project objective. In addition, a country-specific results framework was introduced for effective monitoring and evaluation of project progress.

13. ASP-P1 Tanzania’s revised PDO Indicators: (i) Quantity of inventoried publicly held obsolete pesticides and associated waste disposed of and/or safeguarded; (ii) quantity of inventoried heavily contaminated soil disposed of and/or safeguarded; and (iii) comprehensive strategy for dealing with future stocks of obsolete pesticides and associated wastes adopted by the steering committee.


14. Please refer to Section1.4 in the program-level ICR.


15. The PDO of ASP-P1 Tanzania was to be achieved through the implementation of four distinct components (outlined in Table 1.3). The table also shows revisions that were made through a Level I restructuring on June 30, 2011.



Revised Components (Level I Restructuring June 30, 2011)

Component 1.A – Cleanup and disposal (Total: US$5.83 million; GEF: US$5.48 million): a. Designing an obsolete pesticide

data base, inventory and risk assessment for free standing pesticides, contaminated sites, soils, buried pesticides and contaminated equipment;

b. Producing a CESA; c. Safeguarding obsolete pesticide

stocks; and d. Disposing of the identified,

publicly held, obsolete pesticides.

Part A – Obsolete pesticide inventory: a. Collection of data and compilation of a

data base; b. A risk-based prioritization of the stocks

identified in the data base; and c. Preparation of the CESA.

Part A – Obsolete pesticide inventory: No changes were introduced at restructuring.



b. Implementation of the treatment and/or disposal technology selected;

c. Improving the operations of obsolete pesticide stocks collection centers including store stabilization and safety measures at those centers; and


Part B – Disposal and/or safeguarding of obsolete pesticides and associated waste: a. Carrying out a technological


b. Implementing the safeguarding and/or disposal technology selected; and

c. Improving the operations of obsolete pesticide stocks collection centers, including store stabilization and safety measures at those centers.

The changes reflected the addition of

Table 1.3: ASP-P1 Tanzania Original and Revised Components

194

safeguarding as an outcome and clearly defined associated waste as an integral part of the stocks to be treated.

Component 1.B - Prevention of obsolete pesticide accumulation (Total: US$0.28 million; GEF: US$0.27 million): a. Raising awareness of ASP and

obsolete pesticide disposal options;

b. Reviewing and revising pesticide laws and regulations; and

c. Producing and piloting implementation of an IPM strategy.

Part C – Prevention of obsolete pesticide accumulation: a. Awareness raising activities on the

Program, and health and environmental hazards of pesticides;

b. A review and update of the legal and regulatory framework to control Obsolete Pesticide accumulation and strengthen enforcement mechanisms; and

c. Preparation and implementation of an IPM strategy.

Part C - Reduction of obsolete pesticide re-accumulation: a. Awareness raising activities on the

program, and health and environmental hazards of pesticides;



d. Preparation of a pesticide and container management strategies.

The change reflected the transition from prevention (zero new accumulation) to reduction of new accumulations and the addition of a pesticide and container management strategies as key outputs.

Component 1.C - Capacity building (Total: US$0.15 million; GEF: US$0.14 million): Approx. 14 training courses were planned to be provided to the government, farmers and the NGO sector. The training and capacity building intended to strengthen: a. Disposal activities including

inventory, risk assessment, CESA and disposal;

b. Prevention activities including pesticide legislation, management and IPM; and

c. Project management and monitoring and evaluation.

Part D – Capacity building: a. Strengthening the recipient’s capacity to

carry out environmental assessments; b. Reviewing the recipient’ s pesticide

management practices and providing training on improved methodologies; and

c. Reviewing the implementation capacity of the PCT and providing requisite training.

Component 1.D - Project management and monitoring (Total: US$1.22 million; GEF: US$0.99 million): Provide overall project management, monitoring and reporting; and Finance the project coordination team and the project for effective implementation.

Part E – Project management: Supporting the PCT in carrying out Project coordination, implementation, monitoring and evaluation, through the provision of technical advisory services, equipment and operational support.



Extension of closing date

Table 1.4: ASP-P1 Tanzania Other Significant Changes

195


07/01/2010 To compensate for start-up delays, and allow completion of the disposal contract, taking into account a potential lag time between obsolete pesticides being exported from the country and the date of actual incineration (up to 12 months), which was inadequately assessed during appraisal.

Country Director


06/30/2011 To compensate for delays experienced during the disposal tender process and allow completion of all core activities, including the treatment of large quantities of highly contaminated soils.

Board


12/13/2012 To allow time for the shipment of stocks to the incineration facilities thus avoiding contamination on transit; and reception of certificates of disposal from the incineration facilities to enable payment and avoid reputational risks.


Reallocation of proceeds Reallocation of proceeds from the categories Goods (-US$60,000), Works (-US$40,000), Consultant Services (-US$50,000) and Unallocated (-US$620,000) to the categories Training and Workshops (+US$370,000) and Operating Costs (+US$400,000).

06/30/2011 To accommodate field activities, training and workshops related to the prevention component to take place during the extended period.

Board

Reallocation of proceeds from the categories Consultant Services (-US$170,000), Training and Workshops (-US$285,000) and Operating Costs (-US$90,000) to the categories Goods (+US$85,000) and Disposal Services (+US$460,000).

12/13/2012 To maximize the amount of hazardous materials to be disposed of.



06/30/2011 Addition of in country safeguarding as a possible cheaper end result for lower risk stocks as a result of larger than budgeted stocks of obsolete pesticides and associated waste.

Board



16. ASP-P1 Tanzania was prepared between 2002 and 2006, as part of preparatory activities at the program-level. It took more than four years from GEF eligibility in July 2001, and more than three

196

years from concept note in July 2002 to project approval by the World Bank Board in December 2006.228

17. Soundness of background analysis: Project preparation was conducted jointly by FAO and the World Bank. Preparation activities were further characterized by close collaboration with CropLife International (CLI) and the African Development Bank (AfDB), as mission reports show.229

18. Background analysis was comprehensive. A preliminary procurement capacity assessment was conducted by the World Bank in July 2003. It was noted that no procurement specialist was envisaged to be part of PMU staffing; the World Bank correctly emphasized that procurement staff would be vital to avoid delays in project implementation. In May 2004, NEMC’s procurement capacity was assessed again, and considered inadequate. The World Bank recommended that NEMC hire a procurement specialist on a short-term basis for twelve months to support the day-to-day procurement. Regular procurement training to improve procurement skills was further recommended. Assessments of the project’s management arrangements and Financial Management (FM) capacity were equally undertaken. Key preparatory outputs such as the PAD and Operational Manual, however, were lacking detailed country-specific information regarding Monitoring and Evaluation (M&E), and risks and risk mitigation measures.

19. Assessment of project design: The project’s design (as for all ASP-P1 participating countries) was preconfigured at the program-level, and did not reflect any country-specific modifications or lessons learned from pervious projects executed in Tanzania. Design weaknesses at the program level that translated into implementation delays at the country level, including ASP-P1 Tanzania, are outlined in Section 2.1 of the program-level ICR.

20. Government commitment: The Government of Tanzania had demonstrated strong commitment to increasing pesticide management efforts, which was translated into the above-mentioned projects and operations (see Section 1.1). In addition, before project approval, the government had ratified the Basel (April 1993), Rotterdam (April 2002), and Stockholm (April 2004) Conventions.

21. Assessment of risks: Program-level risks were adequately identified and outlined in the PAD. They were presented in the form of a matrix and represented a summary of risk assessments undertaken during identification, pre-appraisal and appraisal missions to ASP-P1 participating countries. Even though the assessment of risks was comprehensive and included a broad range of mitigation measures in generic terms, an assessment of risks and mitigation measures specific to Tanzania was lacking. At the same time, it should be noted that it was difficult to assess country-specific risks in the absence of inventories of obsolete pesticides and CESAs, which were only planned to be undertaken during project implementation.

2.2 IMPLEMENTATION

22. Throughout implementation, the project experienced long delays, eventually resulting in three extensions for a total of 35 months, and a Level 1 restructuring in 2011.

23. Key factors affecting implementation and outcomes included:

228 The long preparation time resulted from a lengthy partnership building/negotiation process at the program-level resulting from the complex implementation arrangement and project design (see Section 2.1 in the program-level ICR). 229 Pre-Appraisal mission March 9-14, 2003 (NEMC, World Bank, and FAO); World Bank Appraisal mission April 25-30, 2004 (NEMC, World Bank, FAO, and CLI); and AfDB Appraisal mission July 15-20, 2005 (NEMC, AfDB, and World Bank).

197

• Low project management capacity affected project implementation in the beginning. The project was approved by the Board in December 2006 and became effective two weeks later. However, implementation started slowly due to a shortage of key personnel in the PMU,230 substantial procurement delays due to the absence of a procurement officer during the first ten months of the project and a resulting lengthy procurement of personal protective equipment that was required for the inventory (see Section 2.4 for more detail). Ineffective communication within the PMU further contributed to initial project implementation delays.

• Complex and loosely defined implementation arrangement at the program-level affected project implementation. The loosely defined implementation arrangement at the program level had caused collaboration issues between FAO and the World Bank, which subsequently affected the effective and timely technical and operational support the agencies were foreseen to provide (see Section 2.2 in the program-level ICR). Equally affected was the involvement of the local Non-Governmental Organization (NGO) network, whose Terms of Reference (TORs) and methods of procurement were unclear. Efforts were subsequently undertaken by the PMU, the Pesticide Action Network (PAN-UK) and World Bank to better integrate these activities into the project’s plans, and to provide additional guidance on procurement. For technical support during disposal activities, a Technical Advisor for Disposal (TAD) was eventually recruited and financed by CLI, which (although considerably delayed) positively affected project implementation.

• Strong collaboration with the local NGO network positively supported project activities. A network of local NGOs with experience in health and environmental issues, formed in December 2002. Once integrated into the project plans, it substantially supported the project’s efforts in raising awareness at the local level through sensitizing selected communities, and training community members in independently identifying and monitoring chemical effects at the local level.

• Discovery of large amounts of contaminated soils resulted in a funding gap. While the amount of obsolete pesticides inventoried was much lower than estimated at appraisal, large amounts of heavily contaminated soils had been discovered, which had not been budgeted for. In 2010, the project therefore faced an estimated funding gap of $2.4 million, for all inventoried stocks were disposed of adequately. The World Bank and the government subsequently agreed to dispose of obsolete pesticides and high-risk contaminated soils and safeguard low risk soil in country (as captured at restructuring).

• Restructuring. At mid-term review in October 2009, the project was considered delayed and its PDO unlikely to be achieved on the basis of the original indicators (see Sections 1.2 and 1.3). The World Bank recommended revising the PDO, introducing a country-level results framework, and extending the project’s closing date. The project was eventually restructured in June 2011, and subsequently facilitated PDO achievement.231 Further, in response to issues discussed during the mid-term review concerning disbursement, the ceiling for the designated account was increased from US$100,000 to US$200,000 based on projected expenditures.

230 The PMU coordinator position had only been appointed in January 2008. Until then, NEMC’s director had been coordinating the project in addition to his role as director, which led to a heavy workload and subsequent project implementation delays. 231 It took over two years to restructure the project, as ASP-P1 countries were planned to be restructured together and some of the other countries were delays due to political reluctance to restructure the project.

198

2.3 MONITORING AND EVALUATION (M&E)

24. Overall M&E – Rating Moderately Unsatisfactory based on the following evaluation. 25. M&E design – Rating Unsatisfactory: Despite the lengthy preparation time, at appraisal, an M&E

system specific for Tanzania was not developed.232 A results framework, which was intended to be finalized during project launch and included in the project’s Operational Manual, did not materialize. Only a set of PDO indicators linked to the different project components was developed and outlined in the PAD and Operational Manual, however, these did not allow for adequate monitoring of project implementation progress. The indicators focused on measuring outputs as opposed to progress towards project outcomes. 233 Expected results under the prevention component, in particular, were not adequately considered by the indicators. Other key M&E elements, such as strategies, responsibilities, timelines, and budget were properly detailed in the Operational Manual.

26. M&E implementation – Rating Moderately Unsatisfactory: Until the mid-term review, the PMU compensated the lack of an M&E system with using Excel sheets to monitor project implementation. Eventually in 2009, the TSU provided the PMU with a simplified M&E system, which was subsequently used by the PMU. Monthly and bi-annual progress reports were timely prepared and consisted of quantitative and qualitative information on progress. In 2011, following the restructuring of the project, a results framework specific to Tanzania was finally introduced, and the PDO and associated indicators were revised to focus on the two main project achievements (disposal and/or safeguarding and reduction of future accumulations), which strengthened the M&E of the project, albeit late in the process.

27. M&E utilization – Rating Moderately Satisfactory: All identified sites in the country together with their respective stocks of obsolete pesticides, were subjected to risk assessments based on the Environmental Management Tool Kits (EMTKs) 234 developed by FAO. Data from each site was entered into the Pesticide Stock Management System (PSMS), a FAO web-based database, to calculate risk factors related to the product and environmental conditions, to prioritize each site, and to record pesticides. For the preparation of the CESA, the PSMS prioritization criteria were used to rank all sites into high, medium, and low risk. The PSMS together the EMTK proved to be very useful tools in managing obsolete pesticides stocks.



29. ASP-P1 Tanzania complied with OP 4.01 on Environmental Assessment: An environmental safeguard review that was conducted by the World Bank at mid-term review noted that during inventory activities, staff had not undergone health checks as required, and that contaminated soils had not been

232 Lengthy discussions between the partners during project preparation on whether to have an M&E system at the program-level alone, or also at the country-level, may have contributed to the absence of an M&E system at the country-level at project launch. However, no information was available on whether it was a deliberate decision not to have an M&E system in place at the country-level. 233 See Tanzania Africa Stockpiles Program – Project 1 (ASP-P1) – Joint Mid-Term Review Mission (October 5-16, 2009) Aide Memoire. 234 The tool kit provides a practical set of methodologies to assist countries in the risk-based management of obsolete pesticides, including guidance on the management of collection/storage locations, based on international best practice.

199

adequately safeguarded at two high-risk sites (Vikuge and Morogoro).235 However, shortly after, corrective measures were undertaken; awareness-raising activities were conducted, and contaminated soils were adequately demarcated at these two sites.236 Local NGOs significantly supported consultation activities and independent monitoring of health and safety compliance during inventory activities.

30. A CESA identifying mitigation measures to treat possible negative environmental and social impacts that were considered likely to occur during implementation of the project was completed in October 2011, with guidance of a World Bank safeguards specialist. The assessment had been found satisfactory by the World Bank, and associated Environmental Management Plans (EMPs) that were linked to a Health and Safety Management System were equally completed. The assessment’s delayed completion237 was caused by: a) the sequential design of the project (the inventory was only finalized in November 2009); b) procurement delays; and c) a lengthy World Bank review and clearance process, particularly of the TORs. Project activities involving storage, transportation, disposal, and site remediation were implemented in compliance with the recommendations and requirements of the CESA, the associated EMPs, and the Health and Safety Management System which ensured that all workers were adequately protected against chemical hazards and using safe working practices. A World Bank safeguard specialist monitored and ensured compliance with safeguard requirements through participation in supervision missions and specific safeguard missions.

31. OP 4.09 on Pest Management: Since project activities themselves met requirements of a pest management plan (i.e., through pesticide management, IPM, and capacity building), individual country projects, including ASP-P1 Tanzania, were not required to prepare self-standing documents. See Section 2.4 in the program-level for more details.

32. Financial Management compliance – Rating Moderately Satisfactory: Financial Monitoring Reports (FMRs) were prepared and submitted on a quarterly basis to the World Bank’s satisfaction, as were audit reports. Shortcomings included occasional variances that were found during financial management reviews but which were subsequently corrected; and a missing audit committee during the first year of project implementation, which was eventually established in March 2008 following World Bank recommendations.238

33. Disbursement: Disbursement levels were low during the first three years of project implementation with only 13% disbursement at mid-term review. A six months delay in opening a special account and a low account ceiling (US$100,000) contributed to low disbursement rates as did general delays in effecting disbursements, and procurement shortcomings (see below). Disbursement levels only increased substantially towards the end of the project, attributable to the increase in the ceiling (US$200,000) and the beginning of disposal activities (which were allocated the majority of project funds).

235 See Annex 3 on Safeguards in Tanzania Africa Stockpiles Program – Project 1 (ASP-P1) – Joint Mid-Term Review Mission (October 5-16, 2009) Aide Memoire. 236 See Annex 5 Safeguard Compliance Assessment in Tanzania Africa Stockpiles Program – Project 1 (ASP-P1) – Implementation Support Mission (November 8-12, 2010) Aide Memoire. 237 In relation to the project having been effective in December 2006. 238 A World Bank mission in 2007 had noted that an audit committee is required to be in place according to the 2001 Public Finance Act.

200

34. Procurement – Rating Moderately Satisfactory: All services were procured to the World Bank’s satisfaction; however, occasional delays and discrepancies in procurement plans delayed overall project activities, particularly during the first half of project implementation. Initial delays were caused by the absence of a procurement specialist during the first ten months of implementation.239 The subsequent arrangement, which included a procurement officer, supplies officer, and a part-time procurement consultant, did not prove to be effective; the consultant’s support was not sought by the PMU as intended, therefore poor procurement documentation and procurement delays continued. In particular, the lengthy procurement of the personal protective equipment (February - November 2008), hindered project implementation to progress as the pesticide inventory and subsequent activities could not be started without the equipment. In 2009, the PMU received procurement training,240 which substantially improved its procurement capacity.


35. Post completion: At project completion, two draft legislations had been formulated and were awaiting cabinet clearance; they included a Plant Protection Bill and a Pesticide Management Bill.241

36. Next phase: A follow-on project is currently being prepared to focus on the remaining contaminated soils that continue to be in the country; ASP-P1 Tanzania only managed to address a portion of these contaminated soils. The envisaged project intends to focus on adequate hazardous waste disposal facilities including the establishment of an engineered bio-reactor landfill, capacity building, and enhancing institutional collaboration. A concept note is being prepared by the government for GEF funding consideration.



37. Relevance – Rating High: Despite having been too ambitious, the project’s original objectives were relevant when the project was prepared. The project’s revised objective, design, and implementation are considered highly relevant to the current national development priorities. The Government of Tanzania places strong emphasis on preserving the environment and mainstreaming environmental concerns into all future policy measures, as evident from the country’s five year development plan for the period 2011/12-2015/16.242 The project’s objective is also consistent with the World Bank’s FY12-15 Country Assistance Strategy for Tanzania,243 which supports environmental capacity

239 The project’s procurement officer left the project in January 2007, shortly after project approval. The procurement unit was subsequently staffed with only one supplies officer, who had inadequate experience with World Bank procurement procedures. 240 Two staff attended procurement trainings that were conducted by the Eastern and Southern African Management Institute in cooperation with the World Bank. 241 The Plant Protection Act intends to prevent the introduction or spread of plant pests, provide for phytosanitary control measures, and facilitate trade in plants and plant products. The Pesticide Management Act intends to regulate the manufacture, formulation, importation into and exportation from the country, transport, storage, distribution, sale, use, and disposal of pesticides. 242 The Tanzania Five Year Development Plan 2011/12-2015/16 – Unleashing Tanzania’s Latent Growth Potentials, June 2011, United Republic of Tanzania – President’s Office, Planning Commission 243 Country Assistance Strategy for the United Republic of Tanzania for the Period FY12-15, (Report No. 60269-TZ), May 9, 2011

201

building in most sectors, and enhanced sustainability and improved management of natural resources. The project has also been supporting the Government of Tanzania in complying with the Stockholm Convention on POPs, and the Basel Convention on the control of trans-boundary movements of hazardous wastes and their disposal.


38. Taking account of the project’s Level 1 restructuring in June 2011, the project has been evaluated against its original and revised PDO by measuring achievements at the time of the project’s restructuring (June 2011) and project closing (May 2013), respectively.

39. Original PDO achievement - Rating Unsatisfactory: The assessment took the project’s six PDO indicators into account, of which two were fully (100%), and four not (0%) achieved. The three main project outcomes measured by the indicators include: (i) sound disposal and management of obsolete pesticides; (ii) prevention of accumulation of new stockpiles of obsolete pesticides; and (iii) enhanced capacity and institutional strengthening of pesticide management.244

40. Sound disposal and management of obsolete pesticides. The level of risks emanating from publicly held obsolete pesticides and associated waste to nearby communities or critical natural resources was determined and quantified through the completion of a national inventory of publicly held obsolete pesticides in November 2009 (PDO Indicator #1 – 100%). All identified sites in the country together with their respective stocks of obsolete pesticides were subject to risk assessments based on the EMTKs. Data from each site was entered into the PSMS, which calculates risk factors related to the product and environmental conditions, and assists in prioritizing the different sites. These efforts were substantially supported by the local ASP NGO network through independent monitoring of inventory activities to ensure safety and health standards. The data informed the preparation of the CESA and the disposal and safeguarding of stockpiles. However, at restructuring, the CESA was not yet completed (PDO Indicator #2 – 0%). Consequently, disposal activities had not begun at restructuring (PDO Indicator #3 – 0%).245 However, it should be noted that the project conducted an emergency safeguarding operation at a high-risk site in Morogoro246 in October 2010.247 The operation had been advised by the World Bank and was conducted under application of an approved Environmental Management Plan (EMP). The operation reduced the risk emerging from obsolete pesticides and particularly contaminated soil, and provided the project with experience in preparing a satisfactory EMP and carrying out safeguarding.

41. Prevention of accumulation of new stockpiles of obsolete pesticides. Tanzania’s legal and regulatory framework for pesticide management had not been strengthened at restructuring (PDO Indicator #4 – 0%); the envisaged IPM strategy was not developed (PDO Indicator #5 – 0%). It should be noted that in 2010, it was agreed by the World Bank and the PMU that such strategy would not be developed, but instead a national workshop on IPM would be held, however, the workshop had not taken place

244 Achievements were identified based on intended outcomes presented in Annex 1 in Project Appraisal Document for the Republic of Mali, the Kingdom of Morocco, and the United Republic of Tanzania in support of the Africa Stockpile Program – Technical Annex 3, (Report No. 36273-AFR), November 7, 2006 245 Only a draft disposal tender document had been prepared and submitted to the World Bank for review on May 26, 2011 246 Morogoro is a municipality in the central part of Tanzania, 190 km west of Dar es Salaam. 247 The emergency safeguarding operation included fencing off the contaminated site with a barbed wire fence, putting in place danger signs and hazard tape to prevent people from entering. During disposal activities, 392 tonnes of contaminated soil were excavated from this site.

202

by the time of restructuring. The project raised awareness on the health and environmental hazards of pesticides and encouraging safe pesticide handling. A communication strategy was developed in collaboration with the World Wildlife Fund (WWF), and its implementation was launched in 2009. Activities included awareness raising visits to all agricultural zones (particularly to accompany the inventory), development of briefing materials, and press conferences. A network of local NGOs with experience in health and environmental issues and formed in December 2002, had substantially supported these efforts through sensitizing selected communities, and training community members in independently identifying and monitoring chemical effects at the local level. A National Stakeholders Advisory Forum (NSAF)248 was established and held on July 31, 2009, to provide a forum of knowledge exchange and coordination of activities implemented by local NGOs.

42. Enhanced capacity and institutional strengthening of pesticide management. Between 2005 and 2010, PMU members and representatives from the Ministry of Agriculture (MoA) received training in project management, conducting a national inventory, entering data into the PSMS, preparing a CESA, safeguarding of hazardous waste, and procurement.. Capacity was strengthened as became evident through application of the acquired knowledge while conducting the inventory, and preparing safeguard instruments (PDO Indicator #6 – 100%).

43. Revised PDO achievement – Rating Highly Satisfactory: The assessment took the project’s three revised PDO indicators into account, of which two were fully (100%) achieved and one exceeded its target (131%). In addition, six Intermediate Outcome Indicators framed the evaluation. One of those was not taken into account as it corresponded with an Original PDO Indicator that had been achieved prior to restructuring; all others were fully (100%) achieved. Outcomes were slightly modified following the restructuring and included: (i) Sound disposal and management of obsolete pesticides and associated waste including safeguarding; (ii) reduction of re-accumulation of obsolete pesticides and associated waste through regulations, pest management, and stakeholder involvement; and (iii) enhanced capacity and institutional strengthening of pesticide management.

44. Sound disposal and management of obsolete pesticides and associated waste including safeguarding: The inventory was effectively used to prepare the CESA (Intermediate Outcome Indicator #2 – 100%).249 The CESA led to important improvements in project design, such as the proposal to consolidate low-risk stores into main collection centers using country capacity. Such a strategy led to important cost savings, as it significantly reduced the disposal contractors’ fieldwork. A total of 665 tonnes250 of obsolete pesticides and 2,515 tonnes of associated waste including large amounts of contaminated soils were inventoried. Since the amount of associated waste (particularly contaminated soils) had turned out to be much higher than estimated and the required funding for disposal subsequently beyond the allocated budget, it was agreed at restructuring that the disposal contract would include all inventoried obsolete pesticides (665 tonnes), and of the inventoried heavily contaminated soils only those that were posing a significant risk to human and animal health (300 tonnes). At project closure, all inventoried obsolete pesticides (575 tonnes) were disposed of overseas in an environmentally and technically sound manner (Revised PDO Indicator #1 – 100%). During disposal activities, it became clear that the total amount of obsolete pesticides was lower than

248 The NSAF consisted of representatives from local NGOs. PMU members, the private sector, media, the national registrar for pesticides. 249 The document was finalized on June 15, 2012, and disclosed to the World Bank InfoShop on June 27, 2012. 250 The 665 tonnes of obsolete pesticides represents net weight, and equals 700 tonnes gross weight.

203

anticipated; only representing 575 instead of 665 tonnes. As a result, even though the project disposed of fewer tonnes than originally targeted, the Revised Indicator #1 was fully achieved as all inventoried obsolete pesticides were eliminated. The generated cost savings were subsequently applied to the disposal of high risk contaminated soils to maximize the outcome. Eventually, the project disposed of 392 tonnes of the inventoried heavily contaminated soils (Revised PDO Indictor #2 – 131%), exceeding its target level of 300 tonnes.

45. Reduction of re-accumulation of obsolete pesticides and associated waste through regulations, pest management, and stakeholder involvement. The project implemented measures targeted at sustainably managing and reducing the re-accumulation of obsolete pesticides and associated wastes through the following activities: A comprehensive strategy to better regulate and control the registration, production, storage, and sale of pesticides in Tanzania was adopted by the Steering Committee on April 22, 2013 (Revised PDO Indicator #3 – 100%). The strategy intends to serve as basis and guidance for dealing with future stocks, and is based on a sustainability roadmap developed in 2010, which identified key steps towards sustainable pesticide management in Tanzania (Intermediate Outcome Indicator #5 – 100%). Overall, the strategy was composed of action plans for pesticide management and IPM including container management (Intermediate Outcome Indicator #4 – 100%), which were informed by various studies and reviews.251 At project completion, two draft legislations had been formulated and were awaiting cabinet clearance; they included a Plant Protection Bill and a Pesticide Management Bill.252 In addition to the awareness raising activities highlighted before, the project raised awareness on the health and environmental hazards of pesticides and encouraged safe pesticide handling by having implemented four different communication actions (Intermediate Outcome Indicator #3 – 100%). These included the development of two TV and two

251 These included an assessment of contaminated soil and buried pesticides, a study on pesticide management, and a review of pesticide legislation. 252 The Plant Protection Act intends to prevent the introduction or spread of plant pests, provide for phytosanitary control measures, and facilitate trade in plants and plant products. The Pesticide Management Act intends to regulate the manufacture, formulation, importation into and exportation from the country, transport, storage, distribution, sale, use, and disposal of pesticides.

575 392665

300665

2515

0

500

1000

1500

2000

2500

3000

Obsolete Pesticides Associated Waste(particularly

Contaminated Soils)

Tonn

es

Inventoried 2009

Targets set atRestructuring June2011

Eliminated May 2013

Figure 3.1: Publicly Held Obsolete Pesticides and Associated Waste

Inventoried and Eliminated

204

radio spots, broadcasted in early 2013, raising awareness through participation in five exhibitions,253 and outreach activities during disposal activities in three different zones of Tanzania (Intermediate Outcome Indicator #6 – 100%).

46. Enhanced capacity and institutional strengthening of pesticide management. In addition to the capacity building achieved prior to restructuring, the project further strengthened the government’s capacity through additional training. Training was provided in International Maritime Dangerous Goods (IMDG) Code, safety and health, and preparation and implementation of national prevention actions plans.

47. Weighted average PDO achievement – Rating Satisfactory: At restructuring in June 2011, the project had disbursed US$1.96 million, representing 28.5% of total disbursed funds of US$6.87 million. On this basis and as illustrated by the table below, the weighted average of the project’s PDO achievement is rated moderately satisfactory.


1. Rating Unsatisfactory Highly Satisfactory -




5. Final rating - - Satisfactory


48. The project aimed to directly contribute to the GEO of reducing the effect of POPs on the global environment through: (i) improving the quality of life in poor communities, by reducing environmental risks; (ii) improving environmental protection; and (iii) enhancing the capacity of the agricultural sector to better manage crop pests. GEO achievement is measured by PDO achievement, and, thus, is also rated satisfactory (see Section 3.2).

3.4 EFFICIENCY

49. Efficiency – Rating Substantial: This rating represents a broad estimate, as financial and economic data to support this rating are scarce. At appraisal, an economic or financial analysis was neither undertaken for ASP-P1 at the program nor at the country-level due to non-measurable benefits of reducing the risk of contaminating the environment.

50. Based on the fact that approximately 74% (US$5.07 million) of disbursed GEF funds (US$6.87 million) went towards disposal activities under ASP-P1 Tanzania, the project’s efficiency was primarily assessed by its investment in the disposal of obsolete pesticides and associated waste. The project spend all available funds, and succeeded in achieving its revised PDO by disposing all inventoried obsolete pesticides, and a larger amount of contaminated soils than targeted.

253 Exhibitions included Farmers Day (August 2012), Earth/World Environment Day (June 2012), Public Service Week (July 2012), 50 Years Independence Anniversary event (December 2012), and Maji (Water) Day (March 2012).


205

51. During appraisal, the unit cost for disposal was estimated at US$3,400 per tonne (based on approximately US$15 million spent on the disposal of 3,240 tonnes of obsolete pesticides between 1990 and 2004 across Africa). (See Section 3.4 in the program-level ICR for details). At project closing, 967 tonnes of obsolete pesticides and associated waste had been disposed of at the cost of approximately US$5,243 per tonne, which represents a slightly higher cost than originally anticipated.254 However, the US$ depreciation since 2004 and an increase in the price of fuel, which has increased the cost of disposal in US$ terms, is not taken into account by this calculation; neither are varying distances from collection centers to harbors, varying distances of shipping routes, and responses to call of bid by different companies.


52. Overall Outcome - Rating Satisfactory: On the basis of the project’s efficiency (substantial), relevance (high), and PDO achievement (satisfactory), the overall outcome is rated satisfactory. The overall outcome rating takes into account that despite substantial implementation challenges prior to mid-term review (see Section 2), the project successfully reduced risks to the environment and public health by disposing of all inventoried obsolete pesticides and a substantial amount of highly contaminated soils. In addition, the project succeeded in significantly improving measures to reduce future re-accumulations by establishing the prerequisites for a sustainable system of pest and pesticide management.


53. Poverty impacts, gender aspects, and social development: The disposal of obsolete pesticides and associated wastes had a direct positive impact on poor and farming communities, species and ecosystems, located in proximity to sites carrying obsolete pesticide stocks and contaminated soils. Baseline data on affected populations, species, and ecosystems was not conducted at appraisal; exact impacts are therefore difficult to measure.

54. Institutional change/strengthening: The project strengthened the government’s institutional capacity on several levels: (i) an improved regulatory and legal framework for pest and pesticide management was drafted and is awaiting final enactment; (ii) the logistics of hazardous waste transportation have been internalized and the knowledge gained can inform future disposal activities; (iii) the experience and skills gained in managing the project, particularly in the areas of procurement and FM, could be applied to other projects and subsequently contribute to success; and (iv) knowledge gained on sound pesticide management through the various training that were provided to governmental staff.

55. Other unintended outcomes and impacts: The mid-term review drew attention to 660 tonnes of unwanted sulfur stocks. The stocks could not be included in the project, but the World Bank offered specialist advice to the Government in finding a solution to the problem.255 The CESA TORs were subsequently modified to include the sulfur as an annex to the main report, and eventually the stocks were repacked and used as input by a fertilizer manufacturer.

254 Costs include costs associated with repackaging, transport, and incineration of obsolete pesticides and other related activities such as for example the inventory, safeguarding efforts, associated studies, and advisory services. 255 The sulfur stocks could not be included in the project as (i) the sulfur stocks were privately owned and not part of the publicly held stocks the ASP was looking at, and (ii) it presented a much lower risk to the environment and human populations than the soil contamination or obsolete pesticide stocks.

206

3.7 SUMMARY OF FINDING OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS

56. An ICR mission including a stakeholder workshop was conducted on April 29-30, 2013 in Morogoro, Tanzania. For a detailed summary of the findings, please see Section 8.6. A beneficiary survey was not conducted.


57. The risk to the development outcome is rated moderate: The risk is confined to the re-accumulation of obsolete pesticides, and a potential weakening of government commitment to sustaining the project’s outcomes, particularly with respect to prevention activities. However, the project set the basis for strengthened pesticide legislation and national capacity; if enacted and together with the various activities agreed upon in the comprehensive strategy for dealing with future stocks of obsolete pesticides and associated waste, the government is well positioned to sustainably manage and reduce the re-accumulation of obsolete pesticides. The government’s commitment that has been visible particularly following the project’s restructuring is likely to continue as visible by a follow-on project that is currently being prepared to focus on the remaining contaminated soils that continue to be in the country.



58. World Bank performance in ensuring quality at entry – Rating Moderately Unsatisfactory: Project preparation of ASP-P1 Tanzania was conducted in conjunction with preparatory activities at the program-level, which were lengthy (see Section 2.1 in the program-level ICR). While considerable time was invested in overall preparation, key project preparation documents for ASP-P1 Tanzania only broadly laid out project implementation plans, and lacked country-specific information. The project’s design was characterized by several weaknesses, which translated into substantial project implementation delays (see Section 2.1). FM and procurement assessments and subsequent recommendations were adequate.

59. Quality of World Bank supervision – Rating Moderately Satisfactory: The project was supervised by the Task Team Leader (TTL) responsible for ASP at the program-level, and a co-TTL specifically assigned to ASP-P1 Tanzania. Supervision missions took place regularly, albeit only once a year prior to 2009.

60. During the first three years, World Bank performance was affected by issues at the program-level that resulted in delays.256 Following the mid-term review, the World Bank team improved its support and addressed more proactively the project’s implementation delays. Efforts included: (i) effective discussions with high-level representatives from the government on how to improve project

256 World Bank performance had contributed to initial project implementation delays through: (i) The late opening of a special account for the project, which delayed disbursements of first installments by six months; (ii) collaboration issues with FAO, which had resulted in the PMU raising concern about differing guidance received from the two agencies; and (iii) lengthy review and clearance processes for non-objections.

207

implementation; (ii) more efficient project implementation support through local World Bank staff; (iii) an extension of the project’s closing date to compensate for start-up delays; and (iv) the preparation (with subsequent approval by the Board) of a Level 1 restructuring involving revision of the PDO to respond to sector realities and country needs, and to ensure PDO achievement. Successful disposal activities were ensured through: (i) the reallocation of existing GEF funds to maximize the amount of hazardous materials to be disposed of; and (ii) the timely preparation of an additional extension of the closing date to ensure the safe arrival of already shipped obsolete pesticide materials at their destination, PDO achievement, and project funding eligibility of all disposal-related activities.

61. FM reviews were regularly conducted by World Bank FM specialists. This included reviewing project FM arrangements (i.e., budgeting, accounting, reporting, funds flow, internal controls and external audit arrangements), and the implementation of internal and external audit queries.

62. Overall World Bank performance – Rating Moderately Unsatisfactory: The project suffered from shortcomings in project preparation and design at the program-level, which caused substantial delays during project implementation. However, following slow performance during the first half of the project, the World Bank team improved its performance through corrective actions, which ultimately contributed to successful PDO achievement.


63. Government performance – Rating Satisfactory: Government commitment at entry towards the issue of obsolete pesticides was evident through actions the government had already undertaken in view of the problem obsolete pesticides were posing. During project preparation and implementation, there was equally strong support for the project as demonstrated by active participation in preparatory and implementation activities.

64. Implementing Agency – Rating Satisfactory: Overall, NEMC’s performance was satisfactory, with gradual improvement throughout project implementation. Initially, the day-to-day management of project activities was hampered by NEMC’s project management capacity. The PMU coordinator position was only appointed in January 2008; a lack of specialized procurement staff further contributed to initial delays. However, following initial challenges, NEMC showed increasing capacity and commitment towards achieving the PDO. Project implementation was substantially improved, and a high level of effectiveness was maintained until project closure; performance was at a highly satisfactory level during the second half of project implementation. NEMC also complied with all World Bank loan covenants, and executed its fiduciary duties in a timely manner, including safeguards.

65. Overall borrower performance – Rating Satisfactory: Based on the above presented evaluation, the overall borrower performance is rated satisfactory.

6. LESSONS LEARNED

66. The most significant lessons learned are outlined in Section 6 of the program-level ICR, as the experience they are based on was often reflected across all country-specific projects. Lessons learned that were particularly relevant to ASP-P1 Tanzania include:

208

• The project’s effectiveness and development impact can significantly benefit from correct application of World Bank safeguard instruments. The CESA identified suitable collection centers, to which obsolete pesticide stocks were transferred to for safer storage and convenient transportation. The collection centers eventually represented a significant cost-saving benefit when final disposal activities were undertaken.

• Participation of local NGOs in project activities can significantly enhance outcomes at the local level. The NGO network was substantially involved in project activities related to conducting the national inventory, and prevention. Awareness raising, independent monitoring of health and safety standards, and promotion of participatory community monitoring significantly improved the project’s impact at the local level. In addition, knowledge sharing platforms, implemented in form of the NSAF, proved to be a valuable venue to raise awareness for the issue of obsolete pesticides.

• Local World Bank presence was important to getting implementation on track. The client struggled with getting guidance and approvals for many day-to-day issues. The presence of a locally based co-TTL was helpful in expediting these processes.

• World Bank support for communications can be very useful. The country-based World Bank team provided important support to the client in communicating the important results obtained. The communications officer and co-TTL worked to convene key media outlets so that they could learn of the program's achievements.



8. ANNEXES



Component


equivalent)








Total 7.48 8.23 110%

209




equivalent)



Borrower In-kind 0.39 0.65 167% Global Environment Facility (GEF) Grant 6.87 6.87 100%

FAO (Netherlands) Parallel Co-financing/Grant 0.22 0.22

100%

CropLife International (CLI) Parallel Co-financing/Grant 0 0.49 -

Total 7.48 8.23 110%


1. Taking account of the project’s Level 1 restructuring in June 2011, the following table provides a comprehensive account of qualitative and quantitative information of outputs realized against both original (achievements at the time of restructuring in June 2011) and revised components (achievements at the time of project closing in May 2013).



Part A – Obsolete pesticides inventory: Achievements under this component are rated Moderately Satisfactory Carrying out of a detailed inventoried of the recipient’s publicly held obsolete pesticides stocks and associated waste through: a. Collection of data and compilation of a



a. An inventory of publicly held obsolete pesticides was completed and validated in November 2009, including 100% data entry into an inventory database, which has been under the management and oversight of FAO, as part of their PSMS (Original PDO Indicator #1 – 100%).

b. All identified sites in the country together with their respective stocks of obsolete pesticides were subject to risk assessments based on the EMTKs. Data from each site was entered into the PSMS, which calculated risk factors related to the product and environmental conditions, and assisted in prioritizing the different sites.

c. At restructuring, a CESA was still being finalized and not yet completed (Original PDO Indicator #2 – 0%).



disposal technology selected; and

a. A technological assessment of available treatment and disposal options was carried out by WWF in 2008.

b. Due to substantial project implementation delays, the disposal of publicly held obsolete pesticide had not begun at restructuring (PDO Indicator #3 – 0%). Only a draft disposal tender document had been prepared and submitted to the World Bank for review on May 26, 2011. However, it should be noted that the project conducted an emergency safeguarding operation at a high-risk site in Morogoro in


210



October 2010. The operation had been advised by the World Bank and was conducted under application of an approved EMP.

c. Operations of obsolete pesticide stocks collection centers had not started by the time of restructuring.

d. A container management review was not completed by the time of restructuring.

Part C – Prevention of obsolete pesticide accumulation: Achievements under this component are rated Moderately Unsatisfactory Carrying out of activities aimed at preventing obsolete pesticide accumulation, including: a. Awareness raising activities on the program,

health, and environmental hazards of pesticides.

b. A review and update of the legal and regulatory framework to control obsolete pesticide accumulation and enforcement mechanisms; and


a. A communication strategy was developed in collaboration with WWF, and its implementation was launched in 2009. Activities included awareness raising visits to all agricultural zones (particularly to accompany the inventory), development of briefing materials, and press conferences. A network of local NGOs with experience in health and environmental issues and formed in December 2002, had substantially supported these efforts through sensitizing selected communities, and training community members in independently identifying and monitoring chemical effects at the local level. NSAF was established and held on July 31, 2009, to provide a forum of knowledge exchange and coordination of activities implemented by local NGOs.

b. Tanzania’s legal and regulatory framework for pesticide management had not been strengthened at restructuring (PDO Indicator #4 – 0%);

c. The envisaged Integrated Pest Management (IPM) strategy was neither developed, nor implemented (PDO Indicator #5 - 0%). It should be noted that in 2010, it was agreed by the World Bank and the PMU that such strategy would not be developed, but instead a national workshop on IPM would be held, however, the workshop had not taken place by the time of restructuring.

Part D – Capacity building: Achievements under this component are rated Moderately Satisfactory. Carrying out of capacity building activities, including: a. Strengthening the recipient’s capacity to carry

out environmental assessments. b. Reviewing the recipient’s pesticide


c. Reviewing implementation capacity of the PMU and providing requisite training.

a. In 2008, the PMU received training on conducting environmental and social assessments; the knowledge gained was subsequently applied when preparing the CESA.

b. Two consultative studies had been conducted by FAO to assess contaminated soils and buried pesticides, and review pesticide management in Tanzania. Representatives from the ministries and the PMU participated in a CLI-funded study tour to Kenya to learn about a private-public program to identify, collect, and dispose obsolete pesticides.

c. Between 2005 and 2010, PMU members and representatives from MoA also received training in project management, conducting a national inventory of obsolete pesticides, entering data into the PSMS, safeguarding of hazardous waste, and procurement.. Capacity was strengthened as became evident through application of the acquired knowledge while conducting the inventory, and preparing safeguard instruments (PDO Indicator #6 - 100%).

Part E – Project management: Achievements under this component are rated Moderately Unsatisfactory. Supporting the PMU in carrying out project coordination, implementation, monitoring, and

An unclear partnership arrangement at the program-level hampered timely technical support, particularly during the first three years.

211

evaluation, through the provision of technical advisory services, equipment, and operational support.

• The highly technical and specialized expertise required for implementation, supervision and monitoring of project activities the PMU was supposed to receive from FAO’s TSU was not delivered to the extent planned. Valuable support was given in terms of equipment through the provision of the PSMS, and associated training. However, limited technical advisory services were given on the pre-qualification, bidding, and execution of the specialized services associated with procuring the selection processes of the CESA and disposal service providers, and M&E.

• The urgently needed operational support from the World Bank was not sufficiently provided prior to mid-term review; however, increased supervision, operational and technical support thereafter, and the subsequent deployment of an in-country co-TTL, improved the level of support. For technical support during disposal activities, a TAD was eventually recruited and financed by CLI, which positively affected project implementation.




Part A – Obsolete pesticides inventory: Achievements under this component are rated Satisfactory




a. See outputs achieved under Original Part A. b. See outputs achieved under Original Part A. c. Information on obsolete pesticides and associated waste obtained

through the completion of the national inventory of publicly held obsolete pesticides which had been completed in November 2009, was effectively used to inform the successful preparation of a CESA (Intermediate Outcome Indicator #2 - 100%). The document was finalized on June 15, 2012, and disclosed to the World Bank InfoShop on June 27, 2012.

Part B – Disposal and/or safeguarding of obsolete pesticides and associated waste: Achievements under this component are rated Satisfactory



212

Safeguarding and/or disposal of publicly held obsolete pesticides stocks and associated waste by: a. Carrying out a technological assessment of


disposal technology selected; and c. Improving the operations of obsolete

pesticide stocks collection centers, including store stabilization and safety measures at those centers.

a. See outputs achieved under Original Part B. b. All inventoried obsolete pesticides (575 tonnes) were disposed of

overseas in an environmentally and technically sound manner (Revised PDO Indicator #1 – 100%). During disposal activities, it became clear that the total amount of obsolete pesticides was less than anticipated; only representing 575 instead of 665 tonnes. As a result, even though the project disposed of less tonnes than originally targeted, the Revised Indicator #1 was fully achieved as all inventoried obsolete pesticides had been eliminated. The generated cost savings were subsequently applied to the disposal of high risk contaminated soils; the project disposed of 392 tonnes of the inventoried heavily contaminated soils (Revised PDO Indictor #2 – 131%), more than targeted.

c. Six major collection centers were rehabilitated, and low risk obsolete pesticide stocks were centralized at these centers.

Part C – Reduction of obsolete pesticide re-accumulation: Achievements under this component are rated Satisfactory. Carrying out of activities aimed at reducing obsolete pesticide and associated waste re-accumulation, including: a. Awareness raising activities on the program,

health and environmental hazards of pesticides.



d. Preparation of pesticide and container management strategies.

a. The project raised awareness on the health and environmental hazards of pesticides and encouraged safe pesticide handling by having implemented four different communication actions (Intermediate Outcome Indicator #3 - 100%). These included the development of two TV and two radio spots, broadcasted in early 2013, raising awareness through participation in five exhibitions, and outreach activities during disposal activities in three different zones of Tanzania (Intermediate Outcome Indicator #6 - 100%).

b. At project completion, two draft legislations had been formulated and were awaiting cabinet clearance; they included a Plant Protection Bill and a Pesticide Management Bill.

c. A comprehensive strategy to better regulate and control the registration, production, storage, and sale of pesticides in Tanzania was adopted by the Steering Committee on April 22, 2013 (Revised PDO Indicator #3 – 100%). Overall, the strategy was composed of action plans for pesticide management and IPM including container management (Intermediate Outcome Indicator #4 – 100%),

d. See above under c.

Part D – Capacity building: Achievements under this component are rated Satisfactory Carrying out of capacity building activities, including: a. Strengthening the recipient’s capacity to

carry out environmental assessments. b. Reviewing the recipient’s pesticide


c. Reviewing the implementation capacity of the PMU and providing requisite training.

a. See outputs achieved under Original Part D. b. See outputs achieved under Original Part D. c. See outputs achieved under Original Part D. Additional training was

provided on International Maritime Dangerous Goods, Safety and Health, and on Pesticide Life Cycle Management for District Plant Protection Officers and Storekeepers.

Part E – Project management: Achievements under this component are rated Moderately Satisfactory Supporting the PMU in carrying out project coordination, implementation, monitoring, and evaluation, through the provision of technical advisory services, equipment, and operational support.

In addition to outputs achieved under Original Part D, the World Bank provided effective operational support during the final disposal activities. For technical support, a TAD was eventually recruited and financed by CLI, which positively affected project implementation.

213







Adebowale, Modupe Senior Disbursement Officer LOAG2


Fye, Serigne Omar Senior Environment Specialist AFTS2

Jaya, Mel Team Assistant AFMZA

Jordy, Denis Senior Agricultural Specialist AFTS4 Co-TTL


Kebede, Meseret Language Program Assistant AFTS1

Kibbassa, Jane Environmental Specialist

Konare, Amadou Senior Environmental Specialist AFTS1

Kristensen, Peter Senior Environmental Specialist AFTS4 TTL

Lagnaoui, Abdelaziz Senior Pest Management Specialist ESDQC

Maber, Steven Senior Operations Officer MNSRE

Mneney, Donald Procurement Specialist AFTPM

Ndiaye, Marie-Jeanne Language Program Assistant AFTS4

Rashid, Aza Team Assistant AFMTZ

Sabai, Mercy Senior Finance Officer AFTFM

Tegwa, Pascal Procurement Specialist AFTPM

Tran, Lucie Operations Officer AFTS4

Tynan, Ellen Senior Environmental Specialist ENVMP

Warner, Christopher Senior Environmental Specialist AFTS1

Waters, Warren Regional Environmental and Safeguards Advisor AFTSD

Supervision/ICR




214

Chande, Faustine Team Assistant

Diallo, Bella Financial Specialist

Doucouré, Djibril Safeguards Specialist

Fodor, Martin Senior Environment Specialist

Follea, Salimata Operations Analyst

Glauber, Ann Jeannette Senior Environmental Specialist Co-TTL

Guibert, Yves Consultant

Ishemo, Sarah Team Assistant

Kibbassa, Jane Environmental Specialist Co-TTL

Kristensen, Peter Program Manager TTL


Matumbo, Faith-Lucy Team Assistant

Mengi, Hubert Financial Management Consultant

Mneney, Donald Senior Procurement Specialist

Mukuta, Leah Team Assistant

Okuny, Michael Financial Management Specialist

Peled Ben Ari, Ayala Environmental Economist



Sabai, Mercy Senior Finance Officer

Sakaya, Elizabeth Disbursement Assistant


Shoo, Geoffrey Financial Management Specialist

Tegwa, Pascal Senior Procurement Specialist


Von Platen-Hallermund, Tobias Junior Professional Officer





4. An ICR mission including a stakeholder workshop was conducted from April 29-30, 2013 in Morogoro, Tanzania. Stakeholders highlighted the following key points:

215

• The most important prevention achievements included improved laws, strengthened capacity building, establishment of a pesticides management system, development of a communication strategy, development of IPM, pesticide and container management strategies, and establishment of a national NGO network.

• The most important disposal achievements included disposal of obsolete pesticides and associated waste, improved collaboration between NGOs, research institutions, and the private sector, identification of obsolete pesticides locations, completion of a nation-wide inventory, and establishment of main collection centers for obsolete pesticides.

• Key challenges included data verification, information gaps regarding obsolete pesticide stocks, high turnover of staff, short project timeframe, communities’ low awareness of obsolete pesticide issue, delay in disbursement of funds, and lengthy procurement processes.

• Sustainability of project outcomes can be ensured through enforcement of existing legal instruments, additional funding for well-equipped laboratories to ensure quality control, awareness raising for the proper use or management of pesticides, an effective system for empty container management including the creation of collection centers, triple-rinsing, recycling, partnership between government and recycling industries, the creation of a multi-purpose disposing unit to cut across pesticides and other chemicals and industrial waste (even through a multi-regional approach), and careful management of subsidies including safety nets.

• Unclear institutional set up at the program-level negatively affected project implementation in form of disbursement delays, a resulting bureaucratic arrangement that hindered timely decision making, and unclear procurement due to different procurement procedures conducted between World Bank and FAO.

• Key lessons learned included: (i) involving target group from the beginning on is important; (ii) obsolete pesticide disposal was costly; (iii) strong collaboration was needed among stakeholders; (iv) in-country incineration should be explored; (v) Tanzania needed regulatory tools for prevention of accumulation of pesticides; and (vi) prevention was a cheaper alternative to overseas obsolete pesticide disposal.



6. Equally incorporated were comments received on the draft ICR. Additional comments included: a. It was suggested to upgrading the unsatisfactory rating for PDO achievement prior to the

restructuring, as according to the borrower ASP-P1 Tanzania progressed well prior to the restructuring as was evident from PDO and IP ratings between February 2009 and November 2010.

• World Bank response: The assessment took the project’s six PDO indicators into account, of which two were fully (100%), and four not (0%) achieved. Those not achieved included Original PDO Indicator #3 (Inventoried publicly held obsolete pesticides stocks disposed of, as a result of the completion of the disposal services contract(s) in accordance with national and international laws and regulations), which was closest linked to the PDO.

216


7. No comments were received from co-financiers and other partners/stakeholders on the ASP-P1 Tanzania draft ICR.


217


220

TECHNICAL ANNEX VI: ASP-P1 TUNISIA

221


ANGeD Agence Nationale de Gestion des Dechets

ANPE Agence Nationale de Protection de l’Environment






EMTK Environmental Management Toolkit























222



223

TABLE OF CONTENTS

1. PROJECT CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES AND DESIGN ...... 232 1.1 CONTEXT AT APPRAISAL ............................................................................................................................................ 232 1.2 ORIGINAL PDO AND INDICATORS ............................................................................................................................. 233 1.3 REVISED PDO AND INDICATORS ............................................................................................................................... 234 1.4 MAIN BENEFICIARIES................................................................................................................................................... 235 1.5 ORIGINAL AND REVISED COMPONENTS ................................................................................................................. 235 1.6. OTHER SIGINIFICANT CHANGES .............................................................................................................................. 237

2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES ..................................................................... 238 2.1 PROJECT PREPARATION, DESIGN, AND QUALITY AT ENTRY ............................................................................ 238 2.2 IMPLEMENTATION ........................................................................................................................................................ 239 2.3 MONITORING AND EVALUATION (M&E) DESIGN, IMPLEMENTATION, AND UTILIZATION ........................ 240 2.4 SAFEGUARD AND FIDUCIARY COMPLIANCE ......................................................................................................... 241 2.5 POST-COMPLETION OPERATION/NEXT PHASE ...................................................................................................... 243





8.1 PROJECT COSTS AND FINANCING ............................................................................................................................. 252 8.2 OUTPUTS BY COMPONENT ......................................................................................................................................... 254 8.3 ECONOMIC AND FINANCIAL ANALYSIS .................................................................................................................. 257 8.4 WORLD BANK LENDINGA AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS .............................. 257 8.5 BENEFICIARY SURVEY RESULTS .............................................................................................................................. 259 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS ............................................................................................ 259 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR ................................................................. 259 8.8 COMMENTS OF CO-FINANCIERS AND OTHER PARTNERS/STAKEHOLDERS ................................................... 259 8.9 VERBATIM COMMENTS ON DRAFT ICR ................................................................................................................... 260

224


Country: Tunisia Project Name: Africa Stockpiles Program - Project 1 Project ID: P075776 L/C/TF Number: TF055745 ICR Date: 06/28/2013 ICR Type: Core ICR Lending Instrument: APL Borrower: Republic of Tunisia Original Total Commitment: US$4.0 million Disbursed Amount: US$3.78 million (03/19/2013)

Revised Amount: - Environmental Category: A Global Focal Area: CHEM Implementing Agency: Agence Nationale de Gestion des Déchets (ANGed) Co-financiers and Other External Partners: Government of the Republic of Tunisia, French Global Environment Facility (FFEM), Food and Agriculture Organization (FAO), CropLife International (CLI), Pesticide Action Network (PAN-UK and PAN-Africa), and World Wildlife Fund (WWF).

B. Key Dates

Process Date Process Original Date Revised / Actual Date(s) Concept Review: 07/17/2002 Effectiveness: 09/30/2005 11/21/2005 Appraisal: 04/19/2004 Restructuring: 11/25/2009, 06/30/2011 Approval: 09/08/2005 Mid-term Review: 03/2008 10/31/2008 Closing: 09/30/2009 06/30/2012 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Global Environment Outcome: Moderate to Low World Bank Performance: Moderately Unsatisfactory Borrower Performance: Satisfactory C.2 Detailed Ratings of World Bank and Borrower Performance World Bank Ratings Borrower Ratings Quality at Entry: Moderately Unsatisfactory Government: Satisfactory

Quality of Supervision: Moderately Satisfactory Implementing Agency/Agencies:

Satisfactory



Satisfactory


225


E. World Bank Staff

Positions At ICR At Approval Vice President: Inger Andersen Christiaan J. Poortman /Gobind T. Nankani Country Director: Neil Simon M. Gray Theodore O. Ahlers Director (Regional Integration) Colin Bruce Mark Tomlinson

Sector Manager: Charles Cormier Inger Andersen Project Team Leader: Dinesh Aryal Steven P. Maber ICR Team Leader: Dinesh Aryal ICR Primary Author: Veruschka Schmidt F. Results Framework Analysis - At project approval, no country-specific results framework existed; only six PDO Indicators had been defined. Following a Level 1 Restructuring in June 2011, a country-specific results framework was introduced. The original six PDO Indicators were revised and reduced to two PDO Indicators, and in addition, seven Intermediate Outcome Indicators were introduced.



-






Satisfactory

226

Project Development Objective (PDO) – as presented in the GEF Trust Fund Grant Agreement:259 To assist the recipient in: (a) eliminating inventoried publicly held obsolete pesticide stocks and associated waste; and (b) implementing measures to reduce and prevent future related risks. Revised PDO – as presented in the Level 1 Restructuring of June 2011 (Report No: 62797-TN): To assist the Government of Tunisia in: (i) eliminating and/or safeguarding inventoried publicly held obsolete pesticide stocks and associated waste; and (ii) implementing measures to reduce future related risks. Global Environment Objectives (GEO) - as presented in the PAD (Report No: 32232-MNA): Reduction of the effects of Persistent Organic Pollutants (POPs) on the global environment. Revised GEO – Not Applicable (a) PDO Indicators – Original PDO Indicators as presented in the GEF Trust Fund Grant Agreement are assessed against achievements realized at the time of the project’s Level 1 restructuring in June 2011, before revised PDO Indicators are assessed against achievements realized at project closing in June 2012.





Values


Years


Indicator 1: Inventory database of publicly held obsolete pesticide stocks in place and being used by the PMU and government’s staff


Date Achieved 04/19/2004 09/30/2009 - 04/30/2007


Target fully achieved (100%): An inventory was completed and validated in 2007, including transfer of data into the FAO obsolete pesticide inventory database. The database was subsequently used for preparation of the CESA, and disposal activities.

Indicator 2: Completion of CESA and implementation of the measures contained therein. Value (quantitative or qualitative)

No Yes - Yes

Date Achieved 04/19/2004 09/30/2009 - 06/30/2011


Target fully achieved (100%): Despite delays, a CESA was completed in February 2010, and found satisfactory by the World Bank. Recommendations contained therein were used for disposal bidding documents and safeguarding activities.

Indicator 3: Inventoried publicly held obsolete pesticide stocks disposed of, as a result of the completion of the disposal services contract in accordance with national and international laws and regulations

259 The PDO, indicators, and components that are stated in the Project Appraisal Document (PAD) slightly differ from those stated in the GEF Trust Fund Grant Agreement. This ICR bases its evaluation on the PDO, indicators, and components presented in the GEF Trust Fund Grant Agreement, which served as the principal guiding document for the project’s implementation activities; in addition, changes under a Level 1 restructuring were introduced to the GEF Trust Fund Grant Agreement.

227


0% 100% - 0%

Date Achieved 04/19/2004 09/30/2009 - 06/30/2011


Target not achieved (0%): Disposal contract was in place. 560 tonnes of the inventoried publicly held obsolete pesticide stocks (2036 tonnes) had been repackaged but not yet shipped and disposed of overseas.




Date Achieved 04/19/2004 09/30/2009 - 06/30/2011


Target partly achieved (70%): Enactment of two legal regulations by the Ministry of Agriculture (100%). Only limited measures were introduced to strengthen compliance with Basel Convention; Rotterdam Convention was not ratified (40%).

Indicator 5: The training program under Part D260 of the project is implemented and the knowledge so acquired is being used by PMU and recipient’s government staff


Date Achieved 04/19/2004 09/30/2009 - 06/30/2011


Target fully achieved (100%): PMU members and representatives from the ministries of environment, agriculture, health, participated in six pesticide management trainings course. The acquired knowledge was applied during the inventory, and safeguard requirements.

Indicator 6: PMU is functional and operating satisfactorily Value (quantitative or qualitative)

No Yes - Yes


Target fully achieved (100%): PMU was functional and operated satisfactorily.


Indicator 1: Percent of publicly held inventoried obsolete pesticides and associated waste disposed of or safeguarded by end of project

Value (quantitative or qualitative) 0 100 - 85

Date Achieved 06/30/2011 06/30/2012 - 06/30/2012


Target partly achieved (85%): 1730 tonnes (85%) of the inventoried 2036 tonnes of publicly held obsolete pesticides and associated waste were disposed of or safeguarded. These include 1,595 tonnes of obsolete pesticides (against 1,280 tonnes inventoried), and 135 tonnes of associated waste (against 756 tonnes inventoried).


228

Indicator 2: A strategy and/or action plan to sustainably improve management and reduce accumulation of obsolete pesticides (including Integrated Pest Management) developed and endorsed by the Steering Committee


None Adoption - Adoption


Target fully achieved (100%): A national action plan was approved by the Steering Committee and subsequently adopted by the Ministry of Agriculture in June 2012.

(b) GEO Indicator(s) – The GEO was measured by the PDO and PDO indicators.




approval documents)




Indicator 1: PSMS training completed Value (quantitative or qualitative)

Training in 2007

Yes - Yes


Target fully achieved (100%): Second PSMS trainings conducted by FAO in 2012.

Indicator 2: Inventory completed Value (quantitative or qualitative) Yes - - Yes

Date Achieved 06/30/2011 - - 04/30/2007



Indicator 3: CESA completed Value (quantitative or qualitative)

Yes - - Yes

Date Achieved 06/30/2011 - - 02/28/2010



Indicator 4: Drafting and piloting of a container management plan in two governorates


None

Consolidation of the piloting

results in the two governorates.

-

Piloting of empty pesticide container collection in two governorates.

229

Date Achieved 06/30/2011 06/30/2012 - 06/30/2012


Target partly achieved (100%): A container management plan was drafted, piloted in two governorates since 2009, and evaluated in December 2010. The consolidation of results was not completed.

Indicator 5: Communication strategies drafted Value (quantitative or qualitative) 1 2 - 2

Date Achieved 06/30/2011 06/30/2012 - 06/30/2012


Target fully achieved (100%): Two awareness-raising strategies were drafted, of which one was drafted and subsequently implemented between April and December 2008; the other was finalized in June 2012 and implemented thereafter.

Indicator 6: Number of revised pesticide management regulation(s) drafted and submitted for promulgation


2 - - 2

Date Achieved 06/30/2011 - - 06/10/2011



Indicator 7: Number of training courses provided on improved pesticide management practices


32 28 - 36

Date Achieved 06/30/2011 06/30/2012 - 06/30/2012


Newly introduced indicator but building on original PDO Indicator #5. Target exceeded (114%) by eight training courses provided on improved pesticide management practices. Indicator did not specify beneficiaries of training courses.

G. Ratings of Project Performance in ISRs – Most DO and IP ratings and actual disbursement data are not available because ISRs reported on ASP-P1 as a whole, and only occasionally on country-specific project performance, including disbursement.

No. Date ISR Archived DO IP

Actual Disbursements (USD millions)

1 10/31/2005 - - 0.00 2 06/30/2006 - - - 3 12/27/2006 - - - 4 06/28/2007 Satisfactory Satisfactory - 5 12/18/2007 Satisfactory Satisfactory - 6 05/30/2008 - - - 7 12/18/2008 Moderately Satisfactory Satisfactory -

230

8 03/25/2009 Satisfactory Satisfactory - 9 12/29/2009 Moderately Satisfactory Satisfactory 0.68 10 06/29/2010 Satisfactory Satisfactory - 11 03/27/2011 - Moderately Satisfactory 0.99 12 12/25/2011 Moderately Satisfactory Satisfactory 0.99 13 06/29/2012 Satisfactory Satisfactory 2.69 03/19/2013 Final disbursement 3.78 H. Restructuring


Board Approved

PDO Change



in USD millions


PDO IP


Not Available

Not Available


• Reallocation of proceeds to allow for the recruitment of a Technical Advisor for Disposal (TAD), preparation of specific risks mitigation plans, and training

06/30/2011 Yes Moderately Satisfactory


1.0 • Revision of the PDO and indicators to reflect a shift from complete disposal to disposal and safeguarding of stocks


• Extension of closing date by eleven months to compensate for delays (07/31/2011 to 06/30/2012)

• Reallocation of proceeds to dispose additional pesticide stocks

• Addition of OP/BP 4.12 on Involuntary Resettlement



Against Original PDO targets Moderately Satisfactory Against formally revised PDO targets Satisfactory

231


Overall (weighted) rating Satisfactory I. Disbursement Profile – A disbursement profile is not available (see explanation under G).

232



1. Country background: Despite the change and diversification observed in the Tunisian economy (industrialization, growth of service sector and expansion of tourism), the agricultural sector had remained economically and socially important for achieving national objectives such as food security, employment, regional equilibrium and social cohesion. At project approval in September 2005, agriculture employed more than one-quarter of the country’s workforce, and accounted for around 13% of GDP.261

2. Sector background: Around 3000 tonnes of pesticides had been imported annually to boost agricultural production. The bulk was used on cereal crops to control insect pests, and diseases on food crops. The Ministry of Agriculture had been a major pesticide user during national pest control campaigns (e.g., locust and rodent control); in expectation of pest outbreaks, the Ministry had stored large quantities of pesticides.262

3. Obsolete pesticide accumulation: Studies conducted in 1997 had revealed the existence of a number of stocks of obsolete pesticides, including Dichlorodiphenyltrichloroethane (DDT). These stocks had mostly accumulated through massive overprovision in relation to actual needs for public health vector control and combating migratory pests such as locusts. Some of these stocks had been stored for over 45 years. At appraisal, Tunisia had hundreds of sites containing obsolete pesticide stockpiles, which potentially posed serious health hazards to nearby populations. Product bans, uncoordinated donations, poor stock management, and inadequate storage had been the additional reasons for the stockpiling of pesticides in the country.

4. Institutional background: Prior to Tunisia becoming a party to the Stockholm Convention in June 2004, a wide process of dialogue and information exchange had been initiated by the Ministry of Environment with other ministries and agencies, which were concerned about the management of dangerous chemicals and wastes; a number of high level meetings among all stakeholders were held to effectively collaborate on developing a National Implementation Plan (NIP) under the Stockholm Convention to ensure ecologically sound management of Persistent Organic Pollutants (POPs). At appraisal, all pesticides imported or formulated in the country had to be registered with the Ministry of Agriculture. Pesticides used for public health purposes were subject to prior authorization by the Ministry of Health. However, the implementation of these regulations was not always rigorous, particularly the regulations that governed the labeling, conditioning, handling, transport and storage of these products. 263

5. Actions taken: At the time of project design, the Government of Tunisia had initiated preparatory activities under its NIP,264 with the support of the United National Environment Program (UNEP) and

261 The Economist Intelligence Unit, Country Profile Tunisia 2005 262 Africa Stockpiles Program – IPM Research Project: Phase I Country Background Papers, Pesticide Action Network UK, February 2007 263 Tunisia Country Situation Report - Association for Environmental Protection in Kerkennah (APEK), October 2005, The International POPs Elimination Project 264 Plan d’Action National de la Tunisie pour la Mise en Œuvre de la Convention de Stockholm sur les Polluants Organiques Persistants, January 2007, République Tunisienne.

233

Table 1.2: ASP-P1 Tunisia Original PDO Indicators

the United Nations Industrial Development Organization (UNIDO). These activities included completion of a preliminary inventory of obsolete pesticides across the country, which had identified 128 sites with estimated 1,200 tonnes of obsolete stocks. Subsequently, the inventoried obsolete stocks were pooled into 17 secured sites. In addition, the government officially established a Project Management Unit (PMU) within the Department of Solid Wastes at the National Environment Protection Agency.

6. Rationale for World Bank assistance: Please refer to Section 1.1 in the program-level ICR. 7. Higher-level objectives: The project intended to contribute to Tunisia’s national development

strategies in the areas of public health, poverty alleviation, environmental protection, and strengthening the agricultural and rural development sector as outlined in Tunisia’s Country Assistance Strategy (CAS).265 In particular, the project aimed to (i) improve the quality of life in poor communities, by reducing environmental health risks, (ii) improve environmental protection, and (iii) enhance the capacity of the agricultural sector to better manage crop pests. Contributing to international efforts to eliminate POPs, improving management of toxic chemicals, and promoting alternatives to pesticide usage that include Integrated Pest Management (IPM) and Integrated Vector Management (IVM) represented further objectives.


8. The Project Development Objective (PDO), indicators, and components that are stated in the Project Appraisal Document (PAD)266 slightly differ from those stated in the GEF Trust Fund Grant Agreement.267 This ICR bases its evaluation on the PDO, indicators, and components presented in the GEF Trust Fund Grant Agreement, which served as the principal guiding document for the project’s implementation activities.

9. ASP-P1 Tunisia’s original PDO as presented in the GEF Trust Fund Grant Agreement:268 To assist the recipient in: (i) eliminating inventoried publicly held obsolete pesticide stocks and associated waste; and (ii) implementing measures to reduce and prevent future related risks.

10. At appraisal, only an overall ASP-P1 results framework had been prepared (see program-level ASP ICR); results frameworks for the individual ASP-P1 countries, including Tunisia, were intended to be finalized during the individual ASP-P1 country project launches. However, country-specific PDO indicators were developed at appraisal. For ASP-P1 Tunisia, these were as follows:

Original PDO Indicators as presented in the PAD

Original PDO Indicators as presented in the GEF Trust Fund Grant Agreement

a. Inventory database of publicly held obsolete pesticide stocks in place and being used by the PMU and the Government of Tunisia’s staff.

b. Completion of Country Environmental and Social Assessment (CESA) and implementation of each CESA’s

a. Inventory database of publicly held obsolete pesticides stocks in place and being used by the PMU and the recipient's government staff.

b. Completion of CESA and implementation of the

265 Country Assistance Strategy for the Republic of Tunisia, (Report No. 20161-TN), March 28, 2000. 266 South Africa and Tunisia: Africa Stockpile Program – Project 1: PAD (Report No: 32232-MNA), August 19, 2005 267 GEF Trust Fund Grant Agreement between the Republic of Tunisia and the International Bank for Reconstruction and Development, (Number TF055745-TN), November 21, 2005 268 ASP-P1 Tunisia’s PDO as presented in the PAD: To eliminate inventoried publicly held obsolete pesticide stockpiles and associated waste, and implement measures to reduce and prevent future related risks.

234

measures. c. Inventoried publicly held obsolete pesticides stocks

disposed of, as a result of the completion of the disposal services contract(s) in accordance with national and international laws and regulations.


e. IPM Strategy developed. f. Training program to be implemented and the knowledge so

acquired being used by PMU and government staff. g. The PMU to be functional and operating satisfactorily.

measures contained therein. c. Inventoried publicly held obsolete pesticides stocks

disposed of, as a result of the completion of the Disposal Services contract(s) in accordance with national and international laws and regulations.


e. The training program under Part D of the Project is implemented and the knowledge so acquired is being used by PMU and recipient's government staff. 269

f. The PMU is functional and its operation is satisfactory to the World Bank.


11. The PDO and associated indicators for ASP-P1 Tunisia were revised through a Level I restructuring on June 30, 2011.270 The original PDO had implied that all inventoried obsolete pesticides would be eliminated in high temperature incineration facilities overseas. With the discovery of higher than estimated stocks, including associated waste, it was preferable to adopt a blended approach of disposal overseas and lower cost safeguarding in-country based on risk profiling of the inventoried pesticides and associated waste, and availability of satisfactory storage facilities (i.e. obsolete pesticides and higher risk stocks of heavily contaminated soils being disposed of overseas while lower risk stocks of associated waste being safeguarded in-country). In addition, the original PDO had assumed no new accumulations of obsolete pesticides during the life of the project, which had been proven difficult to achieve in the context of agricultural intensification and food security. A focus on developing a strategy for sustainable management of future accumulations was pursued instead.

12. ASP-P1 Tunisia’s revised PDO: To assist the Government of Tunisia in: (i) eliminating and/or safeguarding inventoried publicly held obsolete pesticide stocks and associated waste; and (ii) implementing measures to reduce future related risks.

13. ASP-P1 Tunisia’s PDO indicators were reduced in number and revised to focus on the updated project objectives, i.e., disposal and/or safeguarding and reduction of future accumulations (see Table 1.4). In addition, a country-specific result framework was introduced for effective monitoring and evaluation of project progress (see Data Sheet).

14. ASP-P1 Tunisia’s revised PDO Indicators: (i) 100% of publicly held inventoried obsolete pesticides and associated waste disposed of and/or safeguarded by the end of the project; and (ii) a strategy and/or action plan to sustainably improve management and reduce accumulation of obsolete pesticides (including IPM) developed and endorsed by the Steering Committee.

269 See Section 1.5 for the project’s Component D, here referred to as “Part D”. 270 Revisions were approved on June 30, 2011, on the basis of: Restructuring Paper on a Proposed Project Restructuring of Africa Stockpiles Program – Project 1 to the Republic of Tunisia, (Report No: 62797-TN), June 20, 2011

235


15. ASP-P1 Tunisia was designed to eliminate a major source of risk to human health and environmental resources. Primary beneficiaries of the project were as follows:

16. Local beneficiaries: Primary beneficiaries were intended to be poor and farming communities. The project aimed to improve their quality of life by reducing environmental health risks: (i) Poor communities: Since obsolete pesticide stockpiles were often located in poorer communities where people had low awareness regarding the consequences of exposure to contaminated sites and storage equipment, the intended primary beneficiaries were the urban and rural poor; particularly those living in proximity to the 205 identified obsolete pesticide stockpile sites, spread over 24 governorates. (ii) Farming communities: A wide range of pesticides had been used for pest management and vector control in agricultural areas, with pesticide distributors and small-scale farmers often having low levels of awareness for exposure risks, little knowledge of safe pesticide handling and alternative pest control, inadequate storage facilities and limited alternatives.

17. National beneficiaries: The government, in particular the ministries of health, environment, and agriculture, were intended to be institutional beneficiaries. Benefits were planned to include cost-effective pesticide management in terms of reduced loss in capital, or improved effectiveness in appropriate planning, management, and monitoring regarding obsolete pesticides.

18. Global beneficiaries: The global community of people, species, and ecosystems was considered as global beneficiaries. Taking into account that obsolete pesticides are often characterized by high persistence in the environment,271 the environmental impact in the form of damaged ecosystems had been considered serious. Extensive cleanup and contamination-prevention activities were intended to allow for substantial global environmental health benefits.

19. Beneficiaries were not modified during project implementation.


20. The PDO of ASP-P1 Tunisia was to be achieved through the implementation of four distinct components (outlined in Table 1.3). The table also shows revisions that were made through a Level I restructuring on June 30, 2011.


Original Components as presented in the GEF Trust

Fund Grant Agreement

Revised Components (Level I Restructuring June 30, 2011)272

Component 1.A – Disposal of obsolete pesticides (Total:US$3.90 million; GEF:US$3.67 million): a. To complete detailed stock

inventories; b. To produce a CESA including an

Part A – Obsolete pesticides inventory: a. Collection of data and

compilation of a database; b. A risk-based prioritization of

the stocks identified in the

Part A – Disposal and/or safeguarding of obsolete pesticides and associated waste: a. Carrying out a technological assessment

of available treatment and disposal options;

b. Implementation of the safeguarding

271 For example, DDT persistence in soil ranges from 22 to 30 years, Toxaphene up to 14 years, and Dieldrin up to 7 years. 272 Changes outlined in more detail in the Restructuring Paper on a Proposed Project Restructuring of Africa Stockpiles Program – Project 1 to the Republic of Tunisia, (Report No: 62797-TN), June 20, 2011, were introduced to the original components as presented in the GEF Grant Agreement.

Table 1.3: ASP-P1 Tunisia Original and Revised Components

236

Environmental Management Plan (EMP); and

c. To dispose of obsolete pesticides including contaminated containers.

database; and c. Preparation of the CESA.

and/or disposal technology selected; and c. Improving the operations of obsolete

pesticide stocks collection centers, including improving safety measures at those centers.

The changes reflected the addition of safeguarding as an outcome and clearly defined associated waste as an integral part of the stocks to be treated.



b. Implementation of the treatment and/or disposal technology selected; and


Part B – Disposal of obsolete pesticides: No changes were introduced at restructuring

Component 1.B: Prevention of accumulation (Total: US$0.40; GEF: US$0 million): a. Strengthen existing regulatory

system for pesticide control; b. Promote ongoing IPM efforts,

particularly with small scale farmers; c. Promote certified organic

agricultural production; d. Develop a communication campaign

to raise awareness about pesticide impact and opportunities created by IPM; and

e. Upgrade pesticide storage facilities.

Part C – Prevention of obsolete pesticide accumulation: a. Awareness raising activities on

the health and environmental hazards of pesticides;

b. A review and update of the legal and regulatory framework to control obsolete pesticide accumulation and enforcement mechanisms;



Part C: Reduction of re-accumulation: a. Awareness raising activities on the health

and environmental hazards of pesticides; b. A review and update of the legal and


c. Preparation and implementation of an IPM strategy;

d. A review of pesticide management practices and preparation and implementation of a training plan; and

e. Preparation and endorsement of a strategy and/or action plan to sustainably manage and reduce the re-accumulation of obsolete pesticides.

The change reflected the transition from prevention (zero new accumulation) to reduction of new accumulations.

Component 1.C: Capacity building (Total: US$0.23 million; GEF: US$0 million): a. Risk assessment of soil

contamination; b. Assessment of laboratory needs; and c. Training program in pesticide

Part D – Capacity building: Strengthening the capacity of the PMU to implement the project through the provision of training.

Part D – Capacity building: No changes were introduced at restructuring

237

storage management and Integrated Pest Management (IPM).

Component 1.D: Project management and Monitoring and Evaluation (Total: US$0.60 million; GEF: US$0.32 million): The component was intended to establish: a. A PMU at ANPE; b. A National Steering Committee; and c. A Monitoring and Evaluation (M&E)

system.

Part E – Project management: Supporting the PMU in carrying out project coordination, implementation, monitoring and evaluation, through the provision of technical advisory services, equipment and operational support.

Part E – Project management: No changes were introduced at restructuring

1.6. OTHER SIGINIFICANT CHANGES

Change Date Justification Approval Extension of closing date Extension of closing date by 22 months from 09/30/2009 to 07/31/2011.

11/25/ 2009 To compensate for initial delays. Country Director


06/30/2011 To compensate for delays experienced during the disposal tender process and delays that had been caused by the socio-political unrest in Tunisia.

Board

Reallocation of proceeds Reallocation of proceeds from the Unallocated category (-US$400,000) to the Consultant Services, Audit, Training and Workshops category.

11/25/ 2009 To accommodate (i) the recruitment of a Technical Advisor for Disposal (TAD); (ii) the preparation of specific risks mitigation plans for high risk sites to guide disposal activities; and (iii) training, workshops and M&E activities related to the project’s EMP.

Country Director

Reallocation of proceeds from the categories Consulting Services (-US$250,000) and Operating Costs (-US$100,000) to the Disposal Services category (+US$350,000).

06/30/2011 To accommodate the 100% disposal and/or safeguarding of the additional stocks and associated waste inventoried. Contributions received from FFEM and the government were planned to cover the budgeted consulting services and operating costs, respectively.

Board


06/30/2011 Addition of in country safeguarding as a possible cheaper end result for lower risk stocks; this was due to larger than budgeted stocks of obsolete pesticides and associated waste.

Board

Addition of safeguard policies Addition of OP/BP 4.12 on Involuntary Resettlement to the list of triggered safeguard policies.

06/30/2011 As part of the Environmental Assessment carried out by the project, OP/BP 4.12 was triggered for a storage site, requiring a succinct resettlement plan.

Board

Table 1.4: ASP-P1 Tunisia Other Significant Changes

238



21. ASP-P1 Tunisia was prepared between 2001 and 2005, as part of preparatory activities at the program-level. It took more than four years from GEF eligibility in July 2001, and more than three years from concept note in July 2002 to project approval by the World Bank Board in September 2005.273

22. Soundness of background analysis: Project preparation was conducted in close collaboration between the government and the World Bank, the Food and Agriculture Organization (FAO), CropLife International (CLI), and FFEM, as was evident through joint missions, and pledged financial contributions from all partners. Key preparatory outputs such as the PAD and Operations Manual, however only broadly laid out the project’s plans; detailed country-specific information regarding M&E, risks and risk mitigation measures, Financial Management (FM) including disbursement plans, and procurement were missing. In addition, the political situation in Tunisia was not adequately reflected in the project’s design regarding Non-Governmental Organization (NGO) involvement, as became evident from the limited independence of local NGOs during project implementation.

23. During project preparation, Département des Déchets Solides (DDS) was identified as the implementing agency. DDS was anchored in the ANPE, whose overall capacity to implement the project was assessed adequate by the World Bank, particularly based on its experience in executing projects of similar magnitude.274 However, it was noted that specialized staff would need to be identified before appraisal and that training and technical supervision in project management, FM, and procurement would be crucial. Shortly before negotiations, the DDS was separated from ANPE and reestablished as Agence Nationale de Gestion des Dechets (ANGeD) under the Ministry of Environment, resulting in a different legal status. A renewed assessment of FM and procurement capacity was subsequently necessary, but only conducted together with a procurement assessment a couple of months after project approval. The late assessment drew attention to some key FM elements that were missing.275

24. Assessment of project design: The project’s design (as for all ASP-P1 participating countries) was preconfigured at the program level, and did not reflect any country-specific modifications. Design weaknesses at the program level that translated into implementation delays at the country-level, including ASP-P1 Ethiopia, are outlined in Section 2.1 of the program-level ICR.

25. Government commitment: Government commitment at entry towards the issue of obsolete pesticides was evident through preliminary steps the government had undertaken in view of the problem obsolete pesticides were posing. These included activities to reduce short-term risks through controlled storage, and activities targeted at reducing risks in the long-term through import regulation, legislation on registration of imported products, monitoring of wholesalers and retailers, and research

273 The long preparation time resulted from a lengthy partnership building/negotiation process at the program-level resulting from the complex implementation arrangement and project design (see Section 2.1 in the program-level ICR). 274 ANPE had been managing several environmental projects in collaboration with donors such as the KfW (German Development Bank), GEF, or the World Bank. 275 Missing elements included the development of an organizational manual, clearly assigned FM and accounting responsibilities, recruitment of an internal auditor, and presentation of a detailed budget plan.

239

and implementation of alternatives such as IPM, and organic farming. Further steps included the completion of a preliminary inventory of obsolete pesticides across the country with subsequent pooling of inventoried stocks into secured sites.

26. Assessment of risks: Program-level risks were adequately identified and outlined in the PAD. They were presented in the form of a matrix and represented a summary of risk assessments undertaken during identification, pre-appraisal and appraisal missions to ASP-P1 participating countries. Even though the assessment of risks was comprehensive and included a broad range of mitigation measures in generic terms, an assessment of risks and mitigation measures specific to Tunisia was lacking. At the same time, it should be noted that it was difficult to assess country-specific risks in the absence of inventories of obsolete pesticides and CESAs, which were only planned to be undertaken during project implementation.

2.2 IMPLEMENTATION

27. Throughout implementation, the project experienced long delays, eventually resulting in two extensions for a total of 33 months, and a Level 1 restructuring in 2011.

28. Key factors affecting implementation and outcomes included: • Delayed and insufficient technical support affected implementation performance. The PMU was

supposed to receive highly technical and specialized expertise (required for implementation, supervision and monitoring of project activities) from FAO’s Technical Support Unit (TSU); however, this support was not delivered to the extent originally planned. Reasons were mainly collaboration issues between the World Bank and FAO, resulting from a complex and loosely defined stakeholder implementation arrangement at the program-level (see Section 2.2 in the program-level ICR). The limited support of procurement and M&E significantly affected the PMU’s project implementation capacity. The delayed recruitment of a Technical Advisor for Disposal (TAD), originally intended to be financed by CLI, further affected implementation.276 To fill the gap, the World Bank temporarily recruited an experienced obsolete pesticide and hazardous waste expert in July 2009, and together with the TAD’s eventual recruitment in September 2010, good implementation progress was consequently achieved.

• Institutional challenges: The delayed institutional set-up had an effect on the project’s start-up activities. It took more than seven months before project activities began, because ANGed and the Ministry of Agriculture (DGPCQPA) had difficulties to agree on exact responsibilities. Their collaboration continued to be challenging thereafter and particularly affected implementation of prevention activities. Eventually in 2010, responsibilities associated with the management of the prevention component were handed over from ANGed to the Ministry of Agriculture.277 Collaboration improved gradually in the following. Further initial delays were caused by the PMU’s lack of specialized staff; until the mid-term review, the project coordinator was handling most of the project’s administrative tasks, including procurement and M&E related tasks. In

276 The recruitment of a TAD financed by CLI had been planned for February 2007. However, long negotiations between CLI, the World Bank, and the PMU on the financing modalities, had not led to an agreement. The eventual recruitment in 2010 was financed through reallocation of existing GEF funds. 277 Prevention activities were co-financed in parallel by the FFEM. As a result of the above mentioned initial delays, FFEM extended its financing after project closing by 12 months.

240

2010, the transition associated with the recruitment of a new project coordinator put project implementation temporarily on hold.

• Higher than estimated obsolete pesticide stocks: At mid-term review in 2008, the project was facing a funding gap of around US$2.5 million as a result of higher than estimated obsolete pesticides, and an increase in unit cost for disposal.278 A subsequent estimate in 2009, determined the funding gap to be US$1.735 million. The government requested additional GEF financing in the amount of US$770,000, and possible co-financing by CLI was considered. However, in 2010, it was assumed that no additional funding would be needed as existing funds under the original GEF grant would be sufficient following ANGed’s decision to reallocate some of the funds. In addition, the competitive bidding process for the elimination contract had resulted in a lower disposal cost per unit in comparison to the one originally estimated. During disposal activities, nevertheless, further obsolete pesticide stockpiles were identified.

• Weak NGO integration: The participation of local NGOs in prevention and information activities (intended to be strengthened through crosscutting support provided by PAN-UK and WWF) was weak throughout the project. Main reasons for the low involvement were (i) Tunisia’s political environment in which local NGOs were not allowed except for those mandated by the government, and (ii) a lack of agreements determining the extent of responsibilities, roles and extent of support between the PMU, and the NGOs. By recognizing the deficit, and to improve collaboration with the local NGO community, in 2009, the World Bank recommended carrying out a national workshop in order to jointly develop a framework for an improved and more effective involvement in project activities. Eventually in October 2012, such workshop took place; it was delayed as it was embedded in the second communication strategy that had only been completed in June 2012.

• Restructuring: At mid-term review, the World Bank recommended revising the PDO, raising additional funding, and extending the project’s closing date. In addition and in view of the lack of technical support described above, the World Bank urged to strengthen the PMU with technical expertise during disposal activities, and administrative and Monitoring and Evaluation (M&E) support. Even though it took over two years as a result of political reluctance in Tunisia to restructure the PDO, 279 eventually the project was restructured in June 2011.280


29. Overall M&E - Rating Moderately Unsatisfactory.

278 The estimated unit cost for disposal had increased to US$3,500 as opposed to US$2,625 at appraisal, as a result of higher than estimated stocks, fluctuations in the exchange rate of the US dollar, and higher energy costs. 279 See Implementation Status and Results Report (ISR) – Sequence #9 280 Additional delays following the restructuring of the project resulted from a necessary three-months extension of the disposal contract caused by: (ii) the Spanish authorities that took five months to issue the necessary authorizations which were required for exporting the obsolete pesticides in accordance with the recommendations of the Basel Convention; (iii) changes to the maritime itinerary imposed by the shipping company; and (iv) the events associated with the Tunisian revolution that began in December 2010, and which were accompanied by various governmental strikes and blocked roads affecting accessibility of several sites.

241

30. M&E design – Rating Unsatisfactory: Despite the lengthy preparation time, an M&E system specific for Tunisia was not developed before or during appraisal.281 A result framework, which was intended to be finalized by the TSU during project launch and included in the project’s Operational Manual, did not materialize as well. Only a set of PDO indicators linked to the different project components was developed in the PAD and GEF Trust Fund Grant Agreement, however, these did not allow for adequate monitoring of project implementation progress. The indicators focused on measuring outputs as opposed to progress towards project outcomes. Expected results under the prevention component, in particular, were not adequately considered by the indicators.

31. M&E implementation – Rating Unsatisfactory: Until the mid-term review in October 2008, there was neither an M&E specialist nor an M&E system in place, and several delays in submitting progress reports were noted. Eventually, in November 2009, the TSU provided the PMU with and trained it in using a Disposal-Project Tracking Workbook to monitor and evaluate project-related activities. However, the tool was complex, and introduced too late in the process to be used by the PMU as most of the activities the system should have monitored (e.g., the inventory) had already been completed. The PMU compensated for the lack of an M&E system by using self-created tables for progress monitoring. In 2011, following the restructuring of the project, a results framework specific to Tunisia was finally introduced and the PDO and associated indicators were revised to focus on the two main project achievements (disposal and/or safeguarding and reduction of future accumulations) which strengthened the M&E of the project, albeit late in the process.

32. M&E utilization – Rating Moderately Satisfactory: The inventory of obsolete pesticide stocks was conducted by using inventory forms and a manual provided by the FAO Environmental Management Toolkit (EMTK).282 The inventory data was entered into a database of the FAO web-based Pesticide Stock Management System (PSMS) to record and monitor the pesticide stocks, and subsequently validated. The data was used by the PMU for risk profiling of the stocks283 and preparation of the CESA. FAO had provided training to the PMU in how to best use the database, which took place in 2008 and 2012. The PSMS together with the EMTK proved to be very useful tools in managing obsolete pesticides stocks.


33. Safeguard compliance – Rating Satisfactory: ASP-P1 triggered two safeguard policies: OP 4.01 on Environmental Assessment and OP 4.09 on Pest Management; ASP-P1 Tunisia triggered OP 4.12 on Involuntary Resettlement.

34. ASP-P1 Tunisia complied with OP 4.01 on Environmental Assessment: A CESA identifying risks and devising mitigation measures to address potential negative environmental and social impacts was completed to the World Bank’s satisfaction in February 2010, albeit with significant delays. The assessment included extensive public consultations, conducted between June and December 2009.

281 Lengthy discussions between the partners during project preparation on whether to have an M&E system at the program-level alone, or also at the country-level, may have contributed to the absence of an M&E system at the country-level at project launch. However, no information was available on whether it was a deliberate decision not to have an M&E system in place at the country-level. 282 The tool kit provides a practical set of methodologies to assist countries in the risk-based management of obsolete pesticides, including guidance on the management of collection/storage locations, based on international best practice. 283 Sites were evaluated by risk category (high, medium, and low).

242

EMPs and a Health and Safety Plan were equally completed. The assessment’s late completion resulted from: a) the project’s sequential design (the inventory was only finalized in 2007); b) a complicated and time-consuming consultant procurement process; and c) lengthy World Bank review and clearance processes.284 Subsequently, project activities involving storage, transportation, disposal, and site remediation were implemented in full compliance with the recommendations and requirements of the CESA, the associated EMP, and the Health and Safety Plan.285 With exception of the first two years (which were intended to focus on training local staff in conducting the inventory and risk assessment), World Bank safeguard specialists monitored and ensured compliance with safeguard requirements through participation in supervision missions and specific safeguard review missions.286 Disposal operations were completed without reported incidents or accidents to personnel, or damages and/or pollution to the environment.287

35. OP 4.09 on Pest Management: Since project activities themselves met requirements of a pest management plan (i.e., through pesticide management, IPM, and capacity building), individual country projects, including ASP-P1 Tunisia, were not required to prepare self-standing documents. See Section 2.4 in the program-level for more details.

36. ASP-P1 Tunisia complied with OP 4.12 on Involuntary Resettlement: Based on the CESA’s recommendations, OP/BP 4.12 on Involuntary Resettlement was officially added to the list of triggered safeguard policies in June 2011 as part of the Level 1 restructuring. The policy was triggered for a storage site in Fathallah, a southern suburb of Tunis that was being used for business purposes by occupants. The high-risk site required a succinct Resettlement Action Plan (RAP) for the occupants of the site during disposal activities, as they were directly affected by the project. Despite smaller incidents and a difficult political environment resulting from the aftermath of the Tunisian revolution, the comprehensive resettlement action plan was implemented and completed to a highly satisfactory extent in June 2012.288

37. Financial Management compliance – Rating Moderately Satisfactory: The responsible finance and accounting department was adequately staffed. Financial Monitoring Reports together with annual audits (conducted by an external auditor) were regularly prepared and submitted to the World Bank’s satisfaction, however often delayed. In addition, recommendations made during World Bank missions were often implemented with delays, the PMU and the financial and accounting department poorly collaborated with each other on several occasions, and an ineligible expenditure was noted.289

284 The World Bank provided three sets of comments on the CESA over a period of 9 months (10/2008-06/2009), before it was submitted for final World Bank clearance and disclosure. 285 See Programme Africain relative aux Stocks de Pesticides Périmés (Phase I) – PASP Tunisie: Aide-mémoire – Mission de Supervision 5-8 juin 2012 286 In particular see Rapport de mission - Intégration des commentaires en vue de la finalisation de l’EIES PASP Tunisie 24–28 Juin 2009; and Mission de supervision du 05 au 08 juin 2012 – Aide-Mémoire. 287 Mission de supervision du 5-8 juin 2012 – Aide Memoire 288 A consultant was hired in October 2011, who prepared and oversaw necessary safeguarding activities until their completion in June 2012. Activities included a consultation campaign, financial compensation for the site’s occupiers, safeguarding activities, decontamination of the site, and the final return of the occupiers. During safeguarding activities, a scale (used by the occupiers to weigh the materials) and a metal gate owned by the occupiers broke but were subsequently compensated for by the World Bank. Also, the discovery of higher than estimated obsolete pesticides at the site delayed the estimated time frame of two months by four. 289 An unexplained discrepancy in the amount of US$1,128 between the balance of the special account listed on the statement of the Central Bank of Tunisia and the available balance shown in the counting records of ANGed in 2006 was considered as

243

38. Disbursement was low throughout implementation, with only 26% in June 2011, when the project was restructured. However, this was mainly attributable to the project’s design, which included the most costly activity (disposal) to take place at the end of the project.

39. Procurement – Rating Moderately Unsatisfactory: Procurement undertaken by ANGed suffered from a number of delays, particularly in selecting consulting firms to conduct the CESA290 and disposal of obsolete pesticides. The delay was attributable to a lengthy negotiation process between FAO and the World Bank regarding the CESA consulting firm’s Terms of Reference (TORs), the PMU’s weak procurement capacity, and insufficient technical support that the FAO-TSU had originally intended to provide on the pre-qualification, bidding, and execution of the specialized services associated with procuring the selection processes. However, it should be noted that despite these challenges and delays, all services were eventually procured to the World Bank’s satisfaction.


40. Post-completion: Following the project’s closure in June 2012, the Government of Tunisia financed the elimination and safeguarding of 218 additional tonnes of obsolete pesticides and 44 additional tonnes of associated waste that had not been treated under ASP-P1 due to insufficient funds. The originally projected total of 1,280 tonnes had been exceeded by about 533 tonnes following the discovery of additional stocks during disposal activities; of those additional tonnes, only 315 tonnes had been eliminated under the project. The government’s commitment indicated the high priority given by the national authorities to address and resolve the issue of obsolete pesticides.

41. Next phase: FFEM co-funding of the prevention component was extended until March 2014 following the project’s closure. Achievements since project closure and expected outputs planned to be achieved prior to March 2014 include the following:

• To raise awareness for sound pesticide management, two communication strategies had been drafted prior to project closing. The second draft was finalized in June 2012 and includes plans to raise awareness among approximately 4000 farmers located in five different governorates. Implementation is expected to be finalized by the end of 2013. Activities have been using communication tools such as posters, informational agricultural and scientific guide books, a video, three TV spots broadcasted on national television and three radio spots broadcasted on the local radio. To improve the collaboration with NGOs, ANGed called for proposals and, in May 2013, selected nine NGOs to support awareness raising activities. Specifically, they target raising awareness among primary school pupils to transmit information to their farming parents. Activities are expected to be completed by March 31, 2014.

• Following the piloting of empty pesticide container collection in two governorates under the project, the government plans to establish a nation-wide network for the collection and processing of empty pesticide containers by 2015. A study on the network’s technical and economic feasibility is expected to be completed in the first half of 2014. Also planned by 2015 and in collaboration with the National Agency of Sanitary and Environmental Control Products

ineligible expenditure at the end of 2011. The amount was subsequently refunded in March 2013, to the satisfaction of the World Bank. 290 An independent review of ANGed’s procurement procedures undertaken in June 2008 evaluated the processing time of 9.5 months for the selection of a consulting firm to conduct the CESA as too long.

244

is the establishment of a national vigilance system to control the risks associated with pesticides; a study regarding the system’s implementation is expected to be finalized in the first half of 2014.

• In addition to the two pesticide management regulations that were promulgated in 2010 and 2011, the Ministry of Health is currently preparing an executive order for the establishment of a registration committee for the use of biocides and pesticides for public health purposes.

• Lastly, following the acquisition of a mass spectrometer, which allows the government's laboratories to better analyze pesticides, a second mass spectrometer for Inductively Coupled Plasma analysis has been ordered and its delivery is expected at the end of 2013.291 The additional machine will allow for fast identification and detection of trace metals content in samples, and to eventually analyze all types of imported pesticides.



42. Relevance – Rating High: Despite having been too ambitious, the project’s original objectives were relevant when the project was prepared. The project’s revised objective, design, and implementation are considered highly relevant to the current national development priorities. The Government of Tunisia places strong emphasis on preserving the environment, reducing pollution, and supporting efforts to improve the quality of life in cities and rural areas, as evident from the country’s national development plan for the period 2010-14.292 The project’s objective is also consistent with the World Bank’s Country Partnership Strategy for Tunisia FY10-13,293 which supports efforts to reduce negative environmental and health impacts and to improve living conditions in urban areas; in particular, this includes consolidating improvements in solid waste collection, disposal, treatment, and management services with focus on cost-effectiveness and environmental sustainability.

43. The project has been supporting the Government of Tunisia in complying with the Stockholm Convention on POPs, and the Basel Convention on the control of trans-boundary movements of hazardous wastes and their disposal. In addition, the government has shown increasing commitment to eliminating obsolete pesticides and introducing measures to reduce future related risks, as demonstrated by its post-project efforts (see Section 2.5).


44. Taking account of the project’s Level 1 restructuring in June 2011, the project has been evaluated against its original and revised PDO by measuring achievements at the time of the project’s restructuring (June 2011) and project closing (June 2012), respectively.

45. Original PDO achievement – Rating Moderately Satisfactory: The assessment took the project’s six PDO indicators into account, of which four were fully (100%), one partly (70%), and one not

291 The distributor was contracted on October 4, 2013, with a delivery time of 90 days. 292 Economic and Social Development in Tunisia 2010-2014 – Towards an Innovation and Creation Based Growth, Republic of Tunisia, September 2010. 293 Country Partnership Strategy for the Republic of Tunisia for the Period FY10-13, (Report No. 50223-TUN), November 23, 2009

245

achieved (0%). The three main project outcomes measured by indicators include: 294 (i) sound disposal and management of obsolete pesticides; (ii) prevention of accumulation of new stockpiles of obsolete pesticides; and (iii) enhanced capacity and institutional strengthening of pesticide management.

46. Sound disposal and management of obsolete pesticides. The level of risks emanating from publicly held obsolete pesticides and associated waste to adjacent and wider communities or critical natural resources was determined and quantified through the completion of a national inventory of publicly held obsolete pesticides in 2007 (PDO Indicator #1 – 100%). The hereby-obtained information (geo-location of the storage sites, whether the storage sites were protected or not, identity of the contaminant, its quantity, and general condition of the stockpiles) provided the government with the knowledge necessary to inform the successful preparation of a CESA (PDO Indicator #2 – 100%), and eventual disposal and safeguarding of stockpiles. Due to substantial project implementation delays, the disposal of publicly held obsolete pesticide stocks under the disposal contract had only started preparatory activities by the time of the restructuring (PDO Indicator #3 – 0%). The project had therefore not yet reduced the risks emanating from the inventoried obsolete pesticides and associated waste.

47. Prevention of accumulation of new stockpiles of obsolete pesticides. Better regulation and control of the registration, production, storage, and sale of pesticides was achieved by substantially strengthening the legal and regulatory framework for pesticide management. The project fully achieved the enactment of two legal regulations by the Ministry of Agriculture, which reduced the number of products approved by the ministry from 1300 in 2010, to only 400 in 2012.295 The regulations’ preparation was based on an analysis of the regulatory framework governing the sector of agricultural pesticides and its alignment with international conventions. However, measures to strengthen compliance with the Basel Convention were only initiated; the Rotterdam Convention had not been ratified (PDO Indicator #4 – 70%).296 The project also achieved raising awareness on the health and environmental hazards of pesticides and encouraging safe pesticide handling by drafting and implementing one out of two comprehensive communication strategies.

48. Enhanced capacity and institutional strengthening of pesticide management. Between 2006 and 2011, PMU members and representatives from the ministries of environment, agriculture, and health, participated in several pesticide management training courses. Capacity was strengthened as became evident through application of the acquired knowledge during the inventory, and when preparing safeguard instruments (Original PDO Indicator #5 – 100%).

49. Revised PDO achievement – Rating Satisfactory: The assessment took the project’s two revised PDO indicators into account, of which one was fully (100%) and one partly (85%) achieved. In addition, seven Intermediate Outcome Indicators framed the evaluation. Three of those were not taken

294 Achievements were identified based on intended outcomes presented in Section B.1 and Annex I of the PAD (Report No: 32232-MNA) 295 These include (i) Decree No. 2973-2010 which was enacted on November 15, 2010, and which established modalities and conditions for registration, production and storage of pesticides; and (ii) the Executive Order No. 42/2011 which was issued on June 10, 2011, and which set and revised modalities for tax collection related to phytosanitary control analysis, homologation and provisional permit for sale of pesticides. 296 70% achievement of PDO Indicator #4 consists of the following: Enactment of two legal regulations by the Ministry of Agriculture achieved (100%); and only limited measures introduced to strengthen compliance with Basel Convention and non-ratification of the Rotterdam Convention (40%).

246

into account as they corresponded with Original PDO Indicators that had been achieved prior to restructuring; of those taken into account, one exceeded its target, and three were fully achieved. Outcomes were slightly modified following the restructuring and included: (i) sound disposal and management of obsolete pesticides and associated waste including safeguarding; (ii) reduction of re-accumulation of obsolete pesticides and associated waste through regulations, pest management, and stakeholder involvement; and (iii) enhanced capacity and institutional strengthening of pesticide management.

50. Sound disposal and management of obsolete pesticides and associated waste including safeguarding. The project reduced risks emanating from publicly held obsolete pesticides and associated waste to communities, natural resources, and the global environment through the sound disposal and safeguarding of 85% of inventoried publicly held obsolete pesticides and associated waste. In 2007, a total of 2,036 tonnes of publicly held obsolete pesticides and associated waste was inventoried, consisting of 1,280 tonnes of obsolete pesticides and 756 tonnes of associated waste. At project closure, 1,730 tonnes (i.e., 1595 tonnes of obsolete pesticides and 135 tonnes of associated waste) of the inventoried stocks had been disposed of overseas or safeguarded in country in an environmentally and technically sound manner (Revised PDO Indicator #1 – 85%). See Figure 3.1. The project had focused available funding towards eliminating obsolete pesticides and associated waste located at high-risk sites, while lower risk stocks of associated waste were safeguarded at a lower cost in-country. Specifically this included the disposal of (i) 1,595 tonnes of obsolete pesticides (including 18 tonnes that were safeguarded in country and 315 out of 533 tonnes that had been identified in addition to the inventoried 1,280 tonnes during disposal activities);297 and (ii) 135 tonnes of associated waste. The remaining 621 tonnes of associated waste were safeguarded;298 however, their final status could not be verified due to a lack of data verification and difficulties in measuring treatments. It should further be noted that the remaining 218 out of the 533 tonnes of obsolete pesticides that had been identified in addition to the inventoried 1,280 tonnes and which were not disposed of under the project due to time constraints, were disposed of under the government’s own funding after project closure, together with 44 tonnes of associated waste.

297 The additional 297 tonnes identified had not been inventoried in 2007 as a result of difficult site access. 298 According to the PMU empty containers were rinsed, crushed and recycled, contaminated soils located in storage facilities underwent superficial roughening or were delineated with restricted access, and contaminated building, equipment, and materials were cleaned with high-pressure air/vacuum.

Figure 3.1: Publicly Held Obsolete Pesticides and Associated Waste Inventoried and Eliminated/Safeguarded

1280

756

1595

135

218

44

621

0

200

400

600

800

1000

1200

1400

1600

1800

2000

ObsoletePesticides

AssociatedWaste

Safguarded(reported/not verfied)June 2012

Eliminated (orsafeguarded) afterproject closureDecember 2012

Eliminated (orsafeguarded) at projectclosure June 2012

Inventoried 2007

247

51. Reduction of re-accumulation of obsolete pesticides and associated waste through regulations, pest management, and stakeholder involvement. The project implemented measures targeted at sustainably managing and reducing the re-accumulation of obsolete pesticides and associated wastes through the following activities: A national action plan including an IPM was adopted by the Ministry of Agriculture (Revised PDO Indicator #2 – 100%).299 A diagnostic study on the management and use of pesticides in Tunisia was conducted beforehand and served as basis for the action plan.300 The project continued raising awareness on the health and environmental hazards of pesticides and encouraging safe pesticide handling by having drafted the second out of two comprehensive communication strategies (Intermediate Outcome Indicator #5 – 100%). Implementation of the second strategy is currently underway and focuses on training pesticide users and farmers in five different governorates (see Section 2.5 for details) in sound pesticide management. The project also improved the operations of collection centers through a piloting exercise of empty pesticide container collection in two governorates (Intermediate Outcome Indicator #4 – 100%). Plans to mainstream this pilot exercise are described in Section 2.5.

52. Enhanced capacity and institutional strengthening of pesticide management. In addition to the capacity building achieved prior to restructuring, the project further strengthened the government’s capacity in managing pesticides soundly. The government’s laboratory capacity was improved through the acquisition of a mass spectrometer which has allowed the government's laboratories to better analyze pesticides, and to improve the government's procurement capacity. Training courses on improved pesticide management practices were provided to government staff and national stakeholders (Intermediate Outcome Indicator #7 – 100%). Most notably, staff of the Ministry of Agriculture was trained in using and maintaining a container rinsing and crushing unit that was put in place and used for rinsing and crushing contaminated containers under the piloting exercise.301

53. Weighted average PDO – Rating Satisfactory: At restructuring in June 2011, the project had disbursed US$1.0 million, representing 26.3% of total disbursed funds of US$3.78 million. On this basis and as illustrated by the table below, the weighted average of the project’s PDO achievement is rated satisfactory. Annex 8.2 provides a comprehensive account of qualitative and quantitative information of outputs realized against both original and revised PDO and Intermediate Output Indicators.


1. Rating Moderately Satisfactory Satisfactory -




5. Final rating - - Satisfactory

299 The national action plan was validated with stakeholders prior its approval by the project’s inter-ministerial Steering Committee and subsequent adoption by the Ministry of Agriculture in June 2012. 300 Diagnostic de gestion et d’utilisation des pesticides, AGRIFORUM – Tunisie/CA17 International (France), December 2009. 301 Trainings regarding use and maintenance of the container press were conducted by FAO in 2011 and 2012.


248


54. The project aimed to directly contribute to the GEO of reducing the effects of POPs on the global environment through: (i) improving the quality of life in poor communities, by reducing environmental risks; (ii) improving environmental protection; and (iii) enhancing the capacity of the agricultural sector to better manage crop pests. GEO achievement is measured by PDO achievement, and, thus, is also rated satisfactory (see Section 3.2).

3.4 EFFICIENCY

55. Efficiency - Rating Substantial. This rating represents a broad estimate, as financial and economic data to support this rating are scarce. At appraisal, an economic or financial analysis was neither undertaken for ASP-P1 at the program nor at the country-level due to non-measurable benefits of reducing the risk of contaminating the environment.

56. Based on the fact that approximately 91% (approximately US$3.44 million) of disbursed GEF funds went towards disposal activities under ASP-P1 Tunisia, the project’s efficiency was primarily assessed by its investment in the disposal of obsolete pesticides and associated waste. During appraisal, the unit cost for disposal was estimated at US$3,400 per tonne (based on approximately US$15 million spent on the disposal of 3,240 tonnes of obsolete pesticides between 1990 and 2004 across Africa). (See Section 3.4 in the program-level ICR for details).

57. At project closing, 1,730 tonnes of obsolete pesticides and associated waste had been disposed of at the cost of approximately US$1,986 per ton, which represents a much lower cost than originally anticipated.302 Given the overall high cost involved, cleanup and safe disposal was prioritized on the basis of a detailed inventory of pesticide stockpiles and associated waste, determining the identity of the contaminant, its quantity, and its proximity to the people and environment. In comparison with the originally estimated US$3,400 per ton, the project achieved its goal with a much higher cost-benefit than originally estimated. In addition, Tunisia was the first ASP country that succeeded in disposing of its inventoried publicly held obsolete pesticides within allocated resources, and establishing a framework for sustainable pest and pesticide management. However, the US$ depreciation since 2004 and an increase in the price of fuel, which has increased the cost of disposal in US$ terms, is not taken into account by this calculation; neither are varying distances from collection centers to harbors, varying distances of shipping routes, and responses to call of bid by different companies.


58. Overall Outcome - Rating Satisfactory: On the basis of the project’s efficiency (substantial), relevance (high), and PDO achievement (satisfactory), the overall outcome is rated satisfactory. The overall outcome rating takes into account that despite substantial implementation challenges (see Section 2), the project successfully reduced the risks that emanated from publicly held obsolete pesticides and associated wastes, and significantly improved measures to reduce future re-accumulations by establishing a sustainable system of pest and pesticide management.

302 Costs include costs associated with repackaging, transport, and incineration of obsolete pesticides and other related activities such as for example the inventory, safeguarding efforts, associated studies, and advisory services.

249


59. Poverty impacts, gender aspects, and social development: The disposal of obsolete pesticides and associated wastes had a direct positive impact on poor and farming communities, species and ecosystems, located in proximity to the identified 205 sites. Baseline data on affected populations, species and ecosystems was not collected at appraisal; exact impacts are therefore difficult to measure.

60. Institutional change/strengthening: The project strengthened the government’s institutional capacity on several levels: (i) the regulatory and legal framework for pest and pesticide management was improved; (ii) the logistics of hazardous waste transportation have been internalized and the knowledge gained can inform future disposal activities; (iii) the experience and skills gained in managing the project, particularly in the areas of procurement and FM, can be easily applied to other projects and subsequently contribute to success; and (iv) knowledge gained on sound pesticide management through the various training courses.

61. Other unintended outcomes and impacts: No unintended outcomes and impacts were noted.


62. A specific completion, and/or ICR mission was not carried out, and thus no stakeholder workshop was held. However, the government provided a completion report, which did not specify any particular issues with stakeholders.


63. Risk to the development outcome – Rating Low. The project substantially strengthened pesticide legislation and national capacity, and together with the various activities agreed upon in the national action plan, the government is well positioned to sustainably manage and reduce the re-accumulation of obsolete pesticides. The government’s commitment to enforcing these regulations and continuing to implement prevention activities has been demonstrated during the past two years of the project, its commitment to eliminating additional obsolete pesticides after project closure under its own financing is likely to continue.



64. World Bank performance in ensuring quality at entry – Rating Moderately Unsatisfactory: ASP-P1 Tunisia was prepared in conjunction with preparatory activities at the program-level, which were lengthy (see Section 2.1 in the program-level ICR). While considerable time was invested in overall preparation, key project preparation documents for ASP-P1 Tunisia only broadly laid out project implementation plans, and lacked country-specific information. The project’s design was characterized by several weaknesses, which translated into substantial project implementation delays (see Section 2.1 in the program-level ICR). FM and procurement assessments were adequate. Noteworthy is the World Bank’s quick reaction in conducting a second FM and procurement

250

assessment, following the change in the legal status of the implementation agency late in the preparation process.

65. Quality of World Bank supervision – Rating Moderately Satisfactory: The project was supervised by the Task Team Leader (TTL) responsible for ASP at the program-level, and a co-TTL specifically assigned to ASP-P1 Tunisia – both based at the World Bank’s headquarters in Washington DC; supervision missions took place regularly, albeit only once a year.

66. During the first three years, World Bank performance was affected by a complex ASP-P1 program-wide supervision arrangement, which contributed to the project’s initial slow progress (see Section 2.2 in the program-level ICR). It should, however, be noted that despite these initial challenges, World Bank performance contributed to Tunisia having been the first country to successfully carry out disposal activities. The World Bank’s collaboration with ASP-P1 partners was mixed. While the World Bank team effectively worked together with representatives from FFEM throughout implementation,303 the collaboration with FAO and CLI prior to mid-term review was challenging, mainly as a result of the loosely defined stakeholder implementation arrangement at the program-level (see Section 2.2 in the program-level ICR). Collaboration with other partners such as PAN-UK and WWF was adequate, however, limited.

67. Following the mid-term review, the World Bank team gradually addressed more effectively the project’s implementation delays more effectively. To compensate for the insufficient support received from the TSU and CLI, and to ensure achievement of the PDO, the team took corrective actions when needed. These included: (i) timely reaction to the mid-term review through the preparation of an additional GEF financing request, extension of the closing date, and the temporary recruitment of a pesticide expert;304 (ii) reallocation of GEF funds to accommodate the eventual recruitment of a TAD (originally supposed to be financed by CLI); and (iii) the approval of a Level 1 restructuringinvolving revision of the PDO to respond to sector realities and country needs, which contributed to PDO achievement.

68. Overall World Bank performance – Rating Moderately Unsatisfactory: The project suffered from shortcomings in project preparation and design at the program level, but following a resulting slow performance during the first half of the project, the World Bank team gradually improved its performance through corrective actions, which contributed to the achievement of the PDO.


69. Government performance – Rating Satisfactory: Government commitment at entry towards the issue of obsolete pesticides was evident through previous steps the government had already undertaken. During preparation there was equally strong support for the project as demonstrated by active participation in preparatory activities. The government’s decision to modify the institutional set-up shortly before negotiations, however, delayed the project’s start-up activities. It took more than seven months before project activities began, as a result of ANGed and the Ministry of Agriculture (DGPCQPA) having had difficulties in agreeing on exact responsibilities. Collaboration difficulties continued thereafter and particularly affected implementation of prevention activities. However, the

303 Representative from FFEM joint all supervision missions. 304 Long negotiations between CLI, the World Bank, and the PMU at the country-level on the financing modalities of a Technical Advisor for Disposal (TAD) caused significant delay in recruiting a TAD. Eventually, negotiations did not lead to an agreement. The TAD was finally recruited in 2010, albeit financed through reallocation of existing GEF funds.

251

two entities recognized the importance of an effective partnership, and gradually improved their collaboration throughout implementation. Reluctance with respect to the World Bank’s recommendation to revise the PDO, contributed to the delayed restructuring of the project. However, thereafter, the government showed strong commitment towards achieving the revised PDO, and to sustain the project’s outcomes. This was evident through the government’s commitment to eliminate under its own financing after project closing 262 tonnes of obsolete pesticides and associated waste that had been identified during disposal activities, and that could not be financed by the project due to time constraints.

70. Implementing Agency performance – Rating Satisfactory: Overall, ANGed’s performance was satisfactory, with gradual improvement throughout project implementation. Initially, day-to-day management of project activities was hampered by the implementing agency’s lack of specialized staff. Until mid-term review, the project coordinator was handling most of the project’s administrative tasks, including procurement and M&E related tasks; the need for technical support was continuously raised. In 2010, the transition associated with the recruitment of a new project coordinator put project implementation temporarily on hold; a handover of responsibilities associated with the management of the prevention component from ANGed to the Ministry of Agriculture further contributed to delays. Particularly during the last two years of project implementation, ANGed demonstrated strong commitment towards achieving the PDO. ANGed complied with all World Bank loan covenants, and executed its fiduciary duties. Delays were noted in submitting Financial Management Reports and implementing World Bank FM recommendations, and the poor collaboration between the financial and accounting department in several occasions; however no mis-procurement was noted during project implementation. ANGed equally ensured safeguarding requirements.

71. Overall borrower performance – Rating Satisfactory: Based on the above presented evaluation, the overall borrower performance is rated satisfactory.

6. LESSONS LEARNED

72. The most significant lessons learned are outlined in Section 6 of the program-level ICR, as the experience they are based on was often reflected across all country-specific projects. Lessons learned that were particularly relevant to ASP-P1 Tunisia include:

• Both, the implementing agency and the World Bank team need to be staffed with adequate technical expertise, which is crucial for sound project preparation and implementation of projects associated with pesticide disposal. As was evident during implementation, the disposal of obsolete pesticides and associated waste required highly specialized technical expertise, particularly in preparation of disposal activities (i.e., procuring the specialized services). The technical support the project received proved to be crucial for PDO achievement.

• The project’s effectiveness and development impact can significantly benefit from correct application of safeguard requirements. OP 4.12 on Involuntary Resettlement was triggered for a storage site in Fathallah, a southern suburb of Tunis that was being used for business purposes by occupants. By complying with the policy, the project successfully promoted the participation of affected individuals in the planning and implementation of their temporary resettlement. The project compensated affected individuals financially for their loss in income during their

252

temporary resettlement, and allowed for improved standards of living following the successful disposal of obsolete pesticides at the site.

• For projects classified as Category A, sufficient time and technical expertise need to be allocated for the preparation of required safeguards instruments. In the case of ASP-P1 Tunisia, the preparation of the CESA involved complex procurement procedures and resulted in being a lengthy process. A realistic timeline that plans for the required procedures needs to be in place to not delay overall project implementation.



8. ANNEXES


a. Project Cost by Component – All sources (in US$ million equivalent) – as of 06/30/2012 Component


equivalent)








Total 5.16 5.51 107%


Co-financing Appraisal Estimate




Borrower Counterpart Funding 0.53 0.67 126%


French Global Environment Facility (FFEM)

Parallel Co-financing/Grant 0.63 1.05 167%

Total 5.16 5.50 107%

305 This is an estimate. Prevention activities funded by FFEM are still being implemented until March 2014.

254





Part A – Obsolete pesticides inventory: Achievements under this component are rated Satisfactory Carrying out of a detailed inventoried of the recipient’s publicly held obsolete pesticides stocks through: a. Collection of data and compilation of a



a. An inventory of publicly held obsolete pesticides in Tunisia was completed and validated in 2007, including 100% data entry into an inventory database which has been under the management and oversight of FAO, as part of their PSMS (Original PDO Indicator #1 – 100%).

b. The PMU and the government’s staff were trained by FAO in using the system (2007), and have been using the information on stocks identified in the database for a risk-based prioritization of the different sites, and as basis for the preparation of the CESA.

c. A CESA identifying mitigation measures to treat possible negative environmental and social impacts that were considered likely to occur during implementation of the project, was completed in February 2010, and found satisfactory by the World Bank (Original PDO Indicator #2 – 100%).





a. A technological assessment of available treatment and disposal options was completed in June 2008.

b. Disposal activities, however, had only started at the time of restructuring, thus implementation of the treatment and/or disposal technology selected was not completed (Original PDO Indicator #3 – 0%).

c. Operations of collection centers were improved through a piloting exercise of empty pesticide container collection in two governorates since 2009. The exercise was evaluated in December 2010.

Part C – Prevention of obsolete pesticide accumulation: Achievements under this component are rated Moderately Satisfactory Carrying out of activities at preventing obsolete pesticide accumulation, including: a. Awareness raising activities on the health and



a. General public awareness on the health and environmental hazards of pesticides was raised through national media campaigns (e.g., television, radio). Awareness was also raised among journalists through a work shop held in June 2006.

b. The regulatory framework governing the sector of agricultural pesticides and its alignment with international conventions was analyzed as part of a review of pesticide management practices (see below). Based on this review, the project subsequently achieved the


255



enactment of two legal regulations by the Ministry of Agriculture (Original PDO Indicator #4 – 100%). These include (i) Decree No. 2973-2010 which was enacted on November 15, 2010, and which established modalities and conditions for homologation, production and storage of pesticides; and (ii) the Executive Order No. 42/2011 which was issued on June 10, 2011, and which set and revised modalities for tax collection related to phytosanitary control analysis, homologation and provisional permit for sale of pesticides.

c. An IPM strategy was prepared in 2011, and subsequently implemented between 12/2011 and 06/2012.

d. A review of the management and use of pesticides in Tunisia was completed in 2009. In addition to a review of the legal and regulatory framework, it included an analysis of the data gathered through the inventory to better understand the causes for accumulation and possible prevention activities, and an examination of pesticide routes beginning with their import and ending with their usage. Implementation of a training plan to raise awareness for sound pesticide management included: (i) Two training sessions provided to NGOs in December 2005, and June 2009; (ii) 22 training session to 23 technicians from the public service sector and NGOs in April 2008; and (iii) 21 training sessions provided to approx. 360 regional technicians from the ministries of agriculture, environment, and health held between April and December 2008. Technical trainings to build capacity were provided to PMU members and representatives from the ministries of environment, agriculture, and health (see achievements under Part D).

Part D – Capacity building: Achievements under this component are rated Satisfactory. Strengthening the capacity of the PMU to implement the project through the provision of training.

Between 2006 and 2010, PMU members and representatives from the ministries of environment, agriculture, and health, received 6 pesticide management trainings. Capacity was strengthened as became evident through application of the acquired knowledge during the inventory, and safeguard requirements (Original PDO Indicators #5 – 100%).

Part E – Project Management: Achievements under this component are rated Moderately Unsatisfactory. Supporting the PMU in carrying out project coordination, implementation, monitoring and evaluation, through the provision of technical advisory services, equipment and operational support.

The highly technical and specialized expertise required for implementation, supervision and monitoring of project activities the PMU was supposed to receive from FAO’s TSU was not delivered to the extent planned. Collaboration issues between FAO and the World Bank hampered timely technical support, particularly in the beginning of implementation. Valuable support was given in terms of equipment through the provision of the PSMS, and associated training. However, limited technical advisory services were given on the pre-qualification, bidding, and execution of the specialized services associated with procuring the selection processes of the CESA and disposal service providers, and M&E. The recruitment of a TAD financed by CLI had been planned for February 2007. However, long negotiations between CLI, the World Bank, and the PMU on the financing modalities, had not led to an agreement. The eventual recruitment in 2010 was financed through reallocation of existing GEF funds, and represented a

256

valuable technical advisory service to the PMU. The PMU subsequently requested continued assistance for the remaining elimination phase which was granted.




Part A – Disposal and/or safeguarding of obsolete pesticides and associated waste: Achievements under this component are rated Moderately Satisfactory. Safeguarding and/or disposal of publicly held obsolete pesticides stocks and associated waste by: a. Carrying out a technological assessment of




The changes reflected the addition of safeguarding as an outcome and clearly defined associated waste as an integral part of the stocks to be treated. a. See Outputs by Original Component A. b. In 2007, a total of 2,036 tonnes of publicly held obsolete pesticides

and associated waste was inventoried, consisting of 1,280 tonnes of obsolete pesticides and 756 tonnes of associated waste. At project closure, 1,730 tonnes of the inventoried stocks had been environmentally and technically sound disposed of overseas or safeguarded in country (Revised PDO Indicator #1 – 85%). The project had focused available funding towards eliminating obsolete pesticides and associated waste located at high-risk sites, while lower risk stocks of associated waste were safeguarded at a lower cost in-country. Specifically this included the disposal of (i) 1,595 tonnes of obsolete pesticides (including 18 tonnes that were safeguarded in country and 315 tonnes that had been identified in addition to the inventoried 1,280 tonnes during disposal activities); and (ii) 135 tonnes of associated waste. The remaining 621 tonnes of associated waste were safeguarded; however, their final status is unclear due to a lack of data verification and the PMU’s suggestion that their treatment is difficult to measure.

c. A consolidation of the pilot's results was not finalized prior to project closing (Intermediate Outcome Indicator #4 – 70%). However, plans to mainstream the pilot exercise are described in Section 2.5.

Part C - Reduction of re-accumulation: Achievements under this component are rated Satisfactory Carrying out of activities aimed at reducing obsolete pesticide and associated waste re-accumulation, including: a. Awareness raising activities on the health and



The change reflected the transition from prevention (zero new accumulation) to reduction of new accumulations, which allowed the project to achieve activities to a satisfactory extent. a. Awareness for sound pesticide management was raised through the

drafting of 2 communication strategies: The first was implemented between April and December 2008 through a workshop held with 23 technicians from the public service sector and NGOs, and 21 training sessions provided to approx. 360 regional technicians from the ministries of agriculture, environment, and health. The second



257

c. Preparation and implementation of an IPM strategy;

d. A review of pesticide management practices and preparation and implementation of a training plan; and

e. Preparation and endorsement of a strategy and/or action plan to sustainably manage and reduce the re-accumulation of obsolete pesticides.

includes plans to raise awareness of approx. 4000 farmers located in five different governorates (Intermediate Outcome Indicator #5 – 100%).

b. See Outputs by Original Component C c. See Outputs by Original Component C d. In addition to previous achievements (see Outputs by Original

Components), representatives from the ministry of agriculture received (i) an additional training in maintaining the PSMS (Intermediate Outcome Indicator #1 – 100%), (ii) two training sessions on using and maintaining a container press that was put in place and used for pressing and rinsing contaminated containers at one of the sites (the knowledge acquired was further used to conduct the piloting exercise on the collection of empty pesticide containers), and (iii) one training in IPM (Intermediate Outcome Indicator #7 – 100%).

e. Furthermore, a national action plan focusing on sustainably managing and reducing the re-accumulation of obsolete pesticides was validated with stakeholders prior to its approval by the project’s inter-ministerial Steering Committee and subsequent adoption by the Ministry of Agriculture in June 2012 (Revised PDO Indicator #2 – 100%).



8.4 WORLD BANK LENDINGA AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS




Abushakra, Hadi Chief Legal Counsel LEGMS

Adebowale, Modupe Senior Finance Officer LOAG2

Allan, Christine Operations Analyst MNSRE

Bakayoko, Siaka Senior Financial Management Specialist MNACS


Chummun, Zakia Language Program Assistant MNSRE

Di Leva, Charles Chief Counsel LEGEN

Ebro, Kristyn Communications Officer ESDVP

Gorman, Steve Lead Environmental Specialist ENVGC


Kranz, Frederick Senior Procurement MNACS

258

Specialist

Krishnakumar, V.S. Manager ASPTC

Lagnaoui, Abdelaziz Senior Environmental Specialist ESDQC

Lotayef, Dahlia Senior GEF Operations Coordinator MNSRE


Mani, Angeline Language Program Assistant MNSRE

Melkonian, Hovsep Senior Finance Officer LOAG2

Newton, Murray Consultant ENVMP

Olowo-Okere, Edward Lead Financial Management Specialist AFTFM

Opsal, Knut Senior Social Development Specialist MNSRE

Parish, Matthew T. Legal Counsel LEGMS

Tynan, Ellen Environmental Specialist ENVMP

Warner, Christopher Senior Environmental Specialist AFTS1

Supervision/ICR


Abudagga, Hussam

Ahmedou, Hamed Lead Procurement Specialist



Charlier, Garry Senior Rural Development Specialist Co-TTL

Clément, Viviane

Dhouibi, Walid Procurement Specialist

Doucouré, Djibril Safeguards Specialist

Gregoire, Gael Senior Policy Officer Co-TTL

Guibert, Yves


Kristensen, Peter Program Manager


M’Baipor, Lucienne Senior Social Development Specialist

Maber, Steve Senior Operations Officer TTL

Mani, Angeline Language Program Assistant

Mehdi, Mohamed Financial Management Specialist

Motammedi, Alaleh

Peled, Ayala Environmental Economist

259







3. Beneficiary survey results are not available, as no survey was conducted. However, the government is planning an independent assessment of outcomes achieved under the prevention component, including an impact evaluation.


4. A specific completion, and/or ICR mission was not carried out before project closure, and thus no stakeholder workshop was held. However, the government is planning to conduct a stakeholder workshop in April or May 2014, following closure of the prevention component that is still being implemented under FFEM co-funding.


5. The recipient submitted a completion report; messages contained therein were integrated into this document.

6. Comments and edits received on the draft ICR were equally incorporated. Additional comments included the view that the World Bank performed better than moderately unsatisfactorily between the mid-term review and project closing. The implementing agency referred to the support the PMU had received and which had been mobilized by the World Bank. This included additional safeguards support, the temporary recruitment of a technical expert to help procuring the disposal contract, the eventual recruitment of a TAD, and continuous effective communication with the World Bank including fast responses regarding non objection letters.


7. Comments on the draft ASP-P1 Tunisia ICR received from FFEM include: a. The PDO # 4 indicator refers to the non-ratification of the Rotterdam Convention. Efforts are

underway towards the ratification but Tunisia does not yet have the mechanisms to apply this Convention (particularly regarding its customs).

b. Revised PDO Indicator #1 on the disposal of obsolete pesticides and associated waste: Due to budgetary constraints, Tunisia prioritized the disposal of obsolete pesticides, and adjourned the disposal of the associated waste (which were mostly buildings and contaminated soil). Tunisia achieved to treat 125% of obsolete pesticides that had originally been inventoried.


260


AT L A N T I C

O C E A N

I N D I A NO C E A N

IBRD 39963

ALGERIA

MAURITANIA

MOROCCO

MALI

TUNISIA

NIGER

LIBYA ARAB REP.OF EGYPT

SUDAN ERITREA

ETHIOPIA

DJIBOUTI

SOMALIA

CENTRALAFRICAN REP.

CHAD

NIGERIA

CAMEROON

BURKINAFASO

GHANA

TOGOBENIN

CÔTED'IVOIRE

SENEGAL

THEGAMBIA

GUINEA-BISSAU GUINEA

SIERRALEONE

LIBERIA

GABONCONGO

SÃO TOMÉ AND PRÍNCIPE

UGANDAKENYA

DEM. REP.OF CONGO

RWANDA

BURUNDI

TANZANIA

COMOROS

ZAMBIA

ZIMBABWE

ANGOLA

NAMIBIA BOTSWANA

SWAZILAND

LESOTHO

MOZAMBIQUE MADAGASCAR

MALAWI

MAURITIUS

WESTERNSAHARA

AFRICASTOCKPILES PROGRAMME

JUNE 2013

ASP-P1 PARTICIPANT COUNTRIES*

*including ASP-P1 Nigeria which was not taken into account for this ICR.

SOUTHSUDAN

SEYCHELLES

EQUATORIAL GUINEA

SOUTHAFRICA

This map was produced by the Map Design Unit of The World Bank.The boundaries, colors, denominations and any other informationshown on this map do not imply, on the part of The World BankGroup, any judgment on the legal status of any territory, or anyendorsement or acceptance of such boundaries.

GSDPMMap Design Unit

world bank document€¦ · document of the world bank report no: icr2682 . implementation...

Documents