The Zambia Access to ACT Initiative (ZAAI)ESW Concept NoteJanuary 18, 2009

1. Context and Motivation The Zambian National Malaria Program has achieved remarkable success in expanding access to preventive services in recent years. The percentage of households owning at least one bednet increased from 48% in 2006 to 72% in 2008, the population covered by IRS increased from 1.2 million to 3.5 million during the same period and the share of mothers taking 2 doses of IPT during their last pregnancy increased from 62% in 2006 to 67% in 2007 (2006 and 2008 Malaria Indicator Surveys [MIS]). The efforts to expand preventive services have translated into a significant drop in number of malaria cases and deaths caused by malaria. The percentage of children with parasitemia decreased from 29% in 2006 to 10% in 2008 and under-five mortality decreased from 168 (2002) to 117 (2007).

While the expansion of preventive services has been extraordinary; malaria case management is substantially lagging behind prevention efforts. According to the results from the 2008 MIS, only 29% of children under the age of five took an antimalarial within 24 hours of onset of symptom. Furthermore, only 11% of children under-five living in urban area and 5% of those in rural areas took Artemisinin-based combination therapy (ACT), the adopted first line treatment for malaria, within the same/next day of onset of fever. The weakness in the area of case management was discussed during the 2008 mid-term review of the Malaria Booster Project. The National Malaria Control Center emphasized that interventions are needed in order to (i) improve access and guarantee availability of antimalarial drugs and (ii) ensure availability of diagnostic tools.

Background studies indicate that the relative underperformance in case management is due to several factors including: i) inefficiencies in the public supply chain of drugs which results in frequent stock-outs of ACTs and Rapid Diagnostic Tests (RDTs) in public health facilities (Picazo, 2006) (ii) price barriers and lack of awareness about ACTs and diagnostics in the private sector, and (iii) limited coverage of the Community Health Workers (CHWs) that are trained in and have access to ACTs and RDTs.1

The Government of the Republic of Zambia (GRZ) has shown remarkable leadership in the fight against malaria, which is one of the key determinants of the success in rolling out prevention efforts. As reflected in the fifth National Development Plan, GRZ is committed to improve case management. It outlines that scaling up the use of AL to the private sector and CHWs is one of the key strategies in malaria prevention for the period 2006-10.Although numerous strategic documents outline ambitious objectives in the area of case management, there are very few details about how to best operationalize this expansion.

The Zambia Access to ACT Initiative (ZAAI) has, upon request by GRZ, been designed to inform the GRZ on how to best increase the access to ACTs and other essential drugs. Accordingly, this ESW will evaluate the effectiveness of a combination of public and private sector driven strategies for improving access to ACTs and diagnostics in Zambia. It is expected that this study will contribute to an evidenced based strategy on how to increase the access to ACTs and other essential drugs in Zambia.

The lack of availability of drugs was initially discussed in the context of the Malaria Booster Program, as it was noticed that deficiencies in antimalarial distribution had an adverse impact on

1 CHWs access ACTs and RDTs directly from public health facilities in Zambia.

1

malaria case management. However, as it was recognized that this was not merely due to program specific issues but resulted from broader weaknesses in the health system, the Bank advocated for a push towards improving the supply chain for all drugs rather than for antimalarials exclusively. Although this uses ACTs as one of twenty five tracer drugs, its underlying aim is,to provide lessons learned for the supply of all essential drugs.

ZAAI has been designed in close collaboration between the GRZ and several Cooperating Partners (CP’s) and will be co-financed by DFID and USAID. One of the major achievements in the preparation of this ESW was the collaboration with other donors which led to a move away from funding a parallel supply chain for antimalarials (as had been done in the case of HIV/AIDS commodities) to pool their funding to contribute to ZAAI, that takes a comprehensive approach to deal with the deficiencies with the overall supply chain of drugs. By aligning CPs’ funding as well as planned activities and focusing on the needs of the health system, it has been possible to prevent duplication of efforts and a vertical focus on the supply chain of antimalarials. This last aspect is particularly important in the Zambian context, where a move towards vertical disease specific supply chains, particularly for HIV/AIDS commodities, has been prominent in recent years. As for now, the Bank has only contributed with technical input and staff time to ZAAI. According to plan, the Bank will not make any monetary contributions to ZAAI, which will be funded by money that have been leveraged from other donors (please see Section 5.2 for more information).

This ESW fits well with the CAS priorities. ZAAI will be an important contributor to the Bank's policy dialogue on malaria and health system strengthening, as it will inform both the GRZ and CPs on how to best allocate resources to improve access to ACTs and other essential drugs. The evidence produced through this study, will also contribute to the national scale-up of the most effective strategies to improve distribution and supply chain management of drugs in the public and private sectors.

2. Objectives

The development objectives of ZAAI are to assist the GRZ to:

Identify the most cost effective way to enhance the public sector supply chain of drugs in order to reduce drug outages in the public sector. This objective will be achieved through the public sector intervention of ZAAI, which seeks to evaluate the effectiveness of two different distribution models in 16 districts during a 12 month period.

Determine to what extent GRZ can make use of the private sector in order to increase the access2 to ACTs and diagnostics of malaria through Rapid Diagnostic Tests (RDTs). To reach this objective, the private sector intervention of ZAAI will assess the effectiveness of a combined ACT and RDT subsidy as well as a training/accreditation program for private sector providers in 4 districts during a 12 months period.

Quantify the relative effect of the public and private interventions as well as their combined effect on household decision making and treatment seeking behavior.

To effectively disseminate the results from this ESW and contribute with strategic advice to GRZ on how to most cost-effectively improve distribution of essential drugs, including ACTs, in the country.

2 The World Health Organization defines access to essential medicines as encompassing higher availability, reduced prices, better geographical access and higher acceptability of the product (WHO, 2004).

2

3. Scope and Methodology

This section provides a detail description of the public and private sector interventions that will be evaluated under ZAAI.

3.1 Public Sector Intervention

3.1.1. BackgroundThe MOH and its partners have invested substantial amounts of money in the public sector supply chain in recent years. Despite these efforts, health facilities across Zambia continue to face difficulties accessing drugs and medical supplies in appropriate quantities. The 2006 Public Expenditure Tracking Study (Picazo, 2006) reported that 44% of urban health facilities (HFs), 29% of rural HFs and 15% of hospitals did not have ACTs available at the time of the visit. Average stock out time of ACTs was 9.5 weeks for rural HFs and 6 weeks for urban HFs. An in-depth assessment of the public-sector supply chain for essential medicines in May 2007 (Beer, 2007) revealed that although the central to district distribution has recently improved; the district to HF distribution has bottlenecks resulting in the observed stock-outs. Currently Medical Stores Ltd. (MSL) is responsible for the primary distribution of drugs, from the central warehouse to district stores, while the secondary distribution falls under the responsibility of the District Health Management Teams (DHMTs). Beer (2007) concluded that expanding the mandate of MSL to include secondary distribution may improve system performance.

3.1.2 Description of the Public Sector Intervention To ensure adequate stocking at the facility level, experts in supply chain management proposed to evaluate a number of intervention pilots to redesign the distribution system. There are two primary channels for system innovation. These include:

1. Through resource reconfiguration in the existing distribution system (e.g. allocating more planning capacity and transport resources to the district level). Operationally this can be achieved through the introduction of a Commodity Planner (CP) at the district level who is responsible for stock management, coordinating information from HFs, providing technical assistance to enhance stock flows and obtain improved and timely data from HFs.

2. Through change in the physical and operating structure of the distribution network, in particular by introducing a cross-docking system that eliminates the need to hold stocks at the district level. This design, in addition to the improving flow of stocks, and eliminating decision power as the district level, is expected to reduce the scope for leakages out of the public system.

The public sector intervention will focus on evaluating the comparative cost effectiveness of two supply chain models. It will also provide an opportunity for operational learning as the design of the distribution models will be tested before being scaled up nationally. Each of the two systems will be implemented in 8 districts, during a 12 month period. Performance data will be collected and analyzed. The analysis will inform the re-design of the supply chain and data capturing systems for essential drugs (including anti-malaria products) across the country.

3

The two distribution models that will be evaluated are listed below:

SYSTEM A SYSTEM BName District store with CP District converted to Cross-Dock with CPDescription This system represents the smallest

incremental change from the existing arrangements. The district store is maintained as a stockholding point. The DHMTs, supported by a CP, order monthly to MSL, which make monthly deliveries to the districts. Each order from the district to MSL will be based on aggregate consumption data collected from all HFs in the districts. The HFs will report to their DHMT on a monthly basis, and will receive one or two deliveries per month from the district dependent on the availability of storage space at each HF and vehicles and drivers at the district level. Thus, the main change is that a CP, employed by MSL, will be placed at the district store to support the DHMT in all supply activities

This system converts district stores to cross-docking points. No stock will be held at the store; rather, it will act as a pass through for consignments already packed and labeled for HFs. The district cross-dock will collect reports monthly from all HFs, convert them into orders and forward them to MSL. MSL will pick and pack orders for each HF, and deliver to the district cross-dock once per month. A CP will also be added to this system. The CP will deliver, once or twice a month, down to each HF, based on the availability of space at each HF and vehicles and drivers at the district level.

Purpose To evaluate the impact of the introduction of the CP and the new ordering/reporting system on the availability of drugs and medical supplies at service delivery points.

To evaluate the impact of converting district stores to cross-docking points (with MSL packing for individual HFs), adding a CP, and introducing a new ordering/reporting system, on the availability of drugs and medical supplies at service delivery points.

Note: For a diagram representing system A and B, please see Appendix 1 Figure 1 and 2 respectively.

The conclusions from the public sector study will be communicated to the GRZ and CPs through various dissemination activities. It is expected that the most effective system will be scaled up nationwide at the end of the study..

3.1.3 Implementation Arrangements

Two firms have been contracted as key implementing partners for the public sector intervention. John Snow Inc (JSI). will be responsible for the training of CPs and the collection of baseline data from health facilities across the country. Crown Agents, will provide technical support to MSL to handle the day-to-day management of the distribution models. The firm’s responsibilities include recruitment of staff, ongoing management, operation and supervision of the pilot systems, placement of CPs. A Project Secretariat will be formed, comprising representatives from the MOH, MSL, NMCC, USAID, JSI and the Bank. This secretariat will be chaired by the MOH in collaboration with MSL, and will be responsible for the technical assistance and overall project management.

3.2 Private Sector Component

3.2.1 BackgroundThe role of the private sector in malaria case management in Zambia is significant. As illustrated in Table 1, roughly 42% of Zambians who seek care for fever do so in the private sector (LCMS

4

2004). Private sector care is not merely the domain of wealthy households; about equal proportions of individuals in each expenditure quintile access anti-malarials from private outlets.

Table 1. Percent of individuals seeking treatment in the non-public sector for suspected fever/malariaAge category and data source

Urban Rural Expenditure quintileBottom 2nd 3rd 4th Top Total N

All individuals, 2004 LCMS

42.0% 35.4% 36.9% 41.4% 36.8%

32.2% 45.5% 3203

Children under 7, 2004 LCMS

38.7% 36.7% 34.3% 50.5% 49.2%

23.6% 35.8% 316

Note: Results for 2006 MIS do not distinguish by rural/urban or expenditure quintile. Non-public sector does not include mission and mining facility.

While work by Yadav (2007) provides insight into the top level of the private supply chain for anti-malarials in Zambia, little is known about the semi-formal/informal retail outlets that provide treatment to patients. To inform the design of the private sector component of the ZAAI a rapid analysis was conducted of private retail and wholesale outlets in six districts (CHAI and UNZA, 2008). The study revealed that Sulfadoxine-Pyrimethamine (SP) is the highest volume product (61%) in the private sector3, and that only 25% of private sector outlets stock ACTs (which sell for almost 15 times the price of SP).4 According to the study, the largest barriers to ACT utilization are:

i. Stocking: In all surveyed districts, SP has the highest penetration among private sector outlets stocking anti-malarials.

ii. Price and Business Practice: The high price of ACTs5 is a principal factor limiting the private sector uptake. The continued prescription of SP from public facilities also appears to contribute to the high utilization of this drug.

iii. Perception of Demand: Private outlets perceive the demand for ACTs to be low. Shop keepers cited price as the primary factor for not stocking ACTs, while low demand and lack of information appear to limit ACT and RDT uptake.

The private sector intervention of ZAAI will evaluate the effect of a price subsidy of ACTs and RDTs as well as several accompanying intervention (e.g. repackaging, suggested retail price, public awareness campaigns, incentives to wholesalers, and a training and accreditation program for drugstore dispensers) on the stocking and dispensing behavior of ACTs and household demand for ACTs. The subsidization of RDTs is motivated by the fact that the declining occurrence of malaria incidence in Zambia creates an urgent need for proper diagnosis of all febrile cases as it is no longer possible to consider any fever case to be malaria. The national malaria control program has started deploying RDTs in the public sector, however, diagnosis capacity is almost non-existent in the private sector. In addition to improved biological targeting, the use of RDTs will also increase the cost effectiveness of the private intervention as it is more costly to subsidize ACTs than RDTs. 3 SP has shown resistance levels of 26% in Zambia (Heico et al. 2000). 4 ACTs are classified as a prescription-only medications and are therefore not sold legally through unregistered private sector providers, their availability has therefore remained largely limited to registered pharmacies and private clinics in the private sector. 5 Overall, ACTs and Artemisinin monotherapies (AMTs) were the most expensive anti-malarials observed in sampled outlets at average reported rates of $8.48 and $7.92, respectively. CQ and SP were the least expensive anti-malarials observed, with average retail prices of $0.38 and $0.59. Anti-malarial retail prices varied across and within districts. The greatest variations were witnessed in ACT pricing, with, on average, outlets in urban areas charging much higher prices ($10.58 per course) than those in rural areas ($3.53-3.70 per course).

5

The private sector intervention is much smaller than the public sector intervention. This is primarily motivated by the fact that this is the first time the MOH engages more actively with the private sector. The necessary initial preparation for this engagement will include strengthening of the Public Regulatory Agency, the design of an accreditation scheme for drugstore and a training program for drug store owners.

3.2.2. Description of the interventionEffectiveness of the private sector intervention will be measured by evaluating to what extent the private sector intervention: (i) increases the affordability and access to ACTs, (ii) decreases distribution of ineffective antimalarials (e.g. CQ, SP, and artemisinin mono-therapy [AMT]) through the private sector and, (iii) increases diagnostic capacity in the private sector.The intervention has two principal components: Subsidization and Distribution: The ACTs6 and RDTs are procured from the manufacture at its public sector price and then sold at a subsidized price to pharmaceutical wholesalers, which deliver the products, through their normal distribution channels to eligible drug outlets in the designated districts. Implementation of supporting activities: A number of supporting interventions will be implemented. These interventions seek to increase demand for the subsidized ACTs and improve the quality of care received by patients in the private sector.

One of the greatest challenges to the private sector intervention is to determine an adequate level of subsidy for the RDTs and ACTs as well as their respective suggested retail prices (SRPs). The level of subsidy and SRP for the ACT will be guided by experiences with similar interventions in other countries (e.g. Tanzania, Kenya and Uganda). These prices will be set so as to be the lowest possible prices for the end patient and competitive compared the price of CQ, SP and AMT. At the same time it will be important to maintaining a healthy incentive for the wholesalers and dispensing outlets to continue distributing the product efficiently.7 The level of co-payment to the RDT will be set in such way that the cost of diagnosis to the end-patient is minimal or zero, in order to induce the use of RDTs prior to taking the ACT. A willingness-to-pay study is currently being developed to assess the appropriate price for RDTs. To ensure that the wholesaler and the dispensing outlets have adequate incentives for dispensing ACTs only to malaria positive patients, the margins and incentives for them to dispense RDTs will be set a little higher than the margin to dispense ACTs.

The private sector intervention will be implemented in four districts during a 12 months period. During this period the Pharmaceutical Regulatory Authority (PRA) will grant provisional over-the-counter status (OTC) for AL in the four intervention districts. The selection of districts will ensure that communities with a high malaria burden and with poor access to ACTs and RDTs will benefit from the pilot. All private outlets that meet the eligibility criteria (certain minimum conditions for drug storage, dispensing, willingness to comply with contractual obligations of the pilot, willingness to undergo training etc.) will enroll in the accreditation program and be qualified to sell the subsidized ACTs and RDTs.

6 The private sector component will focus on enhancing access to the fixed dose combination of AL which is currently the recommended first-line treatment in Zambia.7 While designing the impact evaluation it has been discussed whether it would be possible to pilot two different subsidy levels in the four intervention districts in order to discern the most cost effective subsidy level. This idea is still being discussed.

6

The results from the private sector intervention will be disseminated in Zambia and global lessons learned will be published and communicated in strategic locations. If the evaluation from the private sector intervention reveals that this is a promising approach for increasing access to ACTs and RDTs in Zambia, the GRZ has the option to apply for funding from the Affordable Medicines Facility-malaria (AMFm) for a sustainable national scale-up of the ACT subsidy.

3.2.3. Implementation ArrangementsThe ACTs and RDTs will be procured through the Drugs Supply Budget Line (DSBL), according to MOH’s regular procurement procedures. DSBL will also, with support from technical partners, negotiate with manufactures the special price for RDTs and ALs used in the pilot. The drugs will then be sold at a subsidized price to established national private wholesalers (distributors) who can demonstrate sufficient reach and coverage to the dispensing outlets in the four selected districts. The wholesalers will use existing distribution channels, including sales to regional distributors, to deliver the ACTs to eligible outlets in the target districts.

Prior to distribution, the drugs will be placed in user-friendly packages with a SRP. These and other supporting activities will be contracted out to a suitable firm/organization with experience in social marketing. The PRA will be a key contributor to the private sector intervention. In addition to providing provisional OTC status, the agency will be responsible for pharmacovigilence and post market surveillance in the four districts. Furthermore, the it will support the project with regards to regulatory matters as far as the private antimalarial market is concerned.

4. Impact Evaluation A rigorous impact evaluation has been designed to evaluate the effectiveness of the public and private sector interventions. This section outlines the design of the impact evaluation.

4.1 Research QuestionsThe impact evaluation of ZAAI is designed in such way that the public and private interventions (discussed above) can be evaluated separately as well as conjointly. Thus, there will be three different treatments; (i) the public sector intervention only, (ii) the private sector intervention only, and (iii) the interaction between the public and private sector interventions, which are evaluated against the control (i.e. the current state, neither the public nor private sector interventions introduced). This set-up would permit the analysis of a range of interesting research questions as listed below by intervention:

Public Intervention1. How does strengthening the supply chain in the public sector affect priority health outcomes

through the increased availability of essential drugs for households?2. Which one, if any, of the supply chain models introduced in the public sector is the most cost

effective in ensuring adequate supplies of essential medicines at the facility and community level?

Private Intervention3. How does a flat subsidy of ACTs and RDTs in the private sector, coupled with supporting

interventions (e.g. suggested retail price or BBC activities), impact household demand for ACTs in private sector outlets?

4. What is the pass-through rate of subsidies to the end user?5. To what extent is it feasible to provide quality diagnostics of malaria through private sector

providers?

7

6. To what extent do households switch from conventional malaria drugs (CQ, SP, AMTs) to ACTs as ACT price is lowered to a more competitive level?

Interaction between Public and Private Interventions7. Under the two interventions, what is the extent of switches in health demand from public to

private sector, or from private to public sector, for malaria related treatment seeking behavior?

8. What are the key characteristics driving choice of care between the public and private sectors?

9. Which of the three interventions (i) public supply chain, (ii) private subsidy, or (iii) both is the most cost effective for achieving priority health outcomes?

4.2 Identification Strategy and District SelectionRigorous evaluation demands that any observed change in outcomes in areas where the interventions/treatments are implemented is compared with a valid counterfactual representing the course of events that would have occurred in the absence of the intervention. There are various methodological approaches to construct a valid counterfactual, but by far the most rigorous evaluation design is an experimental design where treatment/control status is assigned to a locale on a randomized basis. Randomization assures that all units have an equal chance of control or treatment status and satisfies the conditions of a valid counterfactual comparison.8

The public sector intervention is implemented at the district level. A total of 16 out of 58 peri-urban and rural districts will receive one of the two public sector intervention. Urban districts are not included because of the relatively high performance of the health system in urban areas as well as relatively low malaria burden. Districts will be randomly selected from strata (defined below) to ensure a balance across treatment arms in regards to district characteristics.

An additional eight districts will be selected in the public sector study to serve as controls. Maintaining a relative balance between peri-urban and rural districts, this implies the selection of 8 peri-urban and 16 rural districts among the total 24 selected. The treatment arms and control status will be randomly assigned within this sample of 24, stratified by rural or peri-urban status as well as by the three macro-regions of the country.9

Facility data collected in 2006 indicates there are 4 significant predictors of ACT stock outage at the facility level (in addition to rural/peri-urban status). These predictors are: malaria incidence (a positive relationship), likelihood of phone at facility level (a negative relationship), district population (negative relationship), and average catchment area of facility (positive relationship). Together these predictors account for about 15% of the variation in observed ACT outages in the non-urban districts.

Given these findings, districts within each region and peri-urban or rural category are further grouped into “high risk” districts (those that have either 3 or all 4 of these risk factors) and “low risk” districts (those with 2 or fewer risk factors). An equal proportion of high and low risk districts are randomly selected into the study sample. As a final step, the assignation of districts to

8 That is (1) all pre-intervention factors/characteristics will be on average equal between groups and (2) the only difference in observed outcomes is due to the intervention and not to other observed or unobserved factors.9 Districts are stratified by region to ensure a geographic balance in the selected districts and to further control for possible region wide influences such as general weather patterns.

8

System A (district stores plus CP), B (cross-docking plus CP), or Control (existing system) are also randomly determined.

This method results in the selection of districts listed in Annex 1 Table 1.

The private sector intervention is geographically targeted to four districts. Limiting the private pilot’s scale (compared to the public intervention) is required due to the costly nature of the ACT and RDT subsidy. Because of the cross-over nature of the study, two of the districts with the private intervention will also receive some version of the public sector intervention.

Again, only rural and peri-urban districts will be randomly selected into the private sector intervention since the malaria burden in urban districts is comparatively low.

The four selected private sector districts are listed in Annex 1 Table 2.

4.3 Data sources and key indictorsAs the proposed ESW has the objective to inform GRZ on a variety of issues, information needs to be collected across a range of topics stretching from process inputs to ultimate health outcomes. Some of the key indicators that will be collected include: Percentage of public facilities reporting stock-outs of key commodities (including ACTs) Percentage of fever cases treated with ACT within 24 Hours Availability and price of ACT in the private sector Parasite prevalence Reduction in malaria incidence for children under five Reduction in malaria mortality for children under five Number of days of school and work lost to illness Indicators of public sector supply chain reporting

Data will be collected from a number of sources as listed in Table 2 by intervention.

Table 2. Data sources by interventionPublic Sector Intervention Private Sector Intervention Additional Sources* A dedicated facility survey

(baseline and follow-up) Administrative data,

routinely reported through the HMIS and to MSL

Exit interviews with patients

A dedicated private facility survey (baseline and follow-up)

Audits, including use of ACT tracing through uniquely labeled packages

Exit interviews with patients Mystery Shoppers

Household Survey (baseline and follow up)

Complementary Data (e.g. monthly weather statistics, community survey)

Costing Data

Note: * The additional sources will be used to (a) provide a comprehensive picture of health seeking behavior at the household level, (b) analyze the equity implications of each intervention, (c) control for potential confounders, and (d) measure the costs, relative costs and cost effectiveness of the proposed interventions.

For more information on the contents of the data sources listed above and the motivation for collecting the datasets please see Annex 3. Please refer to Annex 2 for more details on sample selection and power calculations.

9

5. Resources

5.1 Task team The ESW team consists of Monique Vledder, Senior Health Specialist AFTH1 (TTL); Rosemary Sunkutu, Senior PHN Specialist AFTH1; Jed Friedman, Senior Economist DECRG (Principal Investigator); Prashant Yadav, Prof. in Supply Chain Management, MIT-Zaragoza; Mirja Sjöblom, Consultant AFTH1; and Edit Velenyi, Consultant DECRG.

5.2 BudgetA preliminary estimate indicates the total cost for ZAAI will amount to approximately US$ 4.7 million for FY09 and FY10. For a more detailed budget see Table 3.

Table 3. Preliminary Budget for the ZAAIComponents Amount (in US$) Percent of totalPublic Sector Intervention 2,365,232 50%Private Sector Intervention 1,000,000 21%Impact Evaluation 1,365,800 29%TOTAL 4,731,032

DFID contributed with a US$3 million grant, which was provided through the External Funded Output (EFO) mechanism. USAID supports ZAAI with technical assistance and approximately US$900K for the public sector intervention. GRZ will contribute US$200K for the public sector training. The private sector intervention is primarily financed by the EFO from DFID. The impact evaluation has received funding in the amount of US$250K from the Spanish Trust Fund for Impact Evaluation and Results-based Management in Human Development Sectors (SIEF).

5.3 Peer-ReviewersThe peer reviewers for this ESW as well as their area of expertise (in brackets) are listed below: Sangeeta Raja, Consultant (Supply Chain Management); Olusoji O. Adeyi, Coordinator (Malaria and Health System), Alessandro Tarozzi (Evaluation Research).

6. Timetable

Date Milestone 2009January Concept Note Review MeetingFebruary Recruitment of CP’s and trainingMarch Implementation startMarch/ April Baseline Household Survey

2010March/April Follow up household surveyJune Decision MeetingJune Presentation and discussion of results (MOH and partners)July, August Dissemination Activities in Zambia and elsewhere

10

7. ReferencesBeer, K.O. (2007) “Zambia Malaria Commodity Security Assessment”.Unpublished Report,

Human Development Department, World Bank. . Clinton Foundation/University of Zambia (2008) “Understanding the Private Sector Anti-

Malarial Market in Zambia”, Unpublished report, Zambia National Malaria Control Centre, and Human Development Department, World Bank

Heico M. Bijl, Joost Kager, David W. Koetsier, Tjip S. Werf (2000) “Chloroquine- and sulfadoxine-pyrimethamine-resistant falciparum malaria in vivo; a pilot study in rural Zambia”, Tropical Medicine & International Health, 5 (10): 962-695.

Picazo, O. 2006. Zambia Public Expenditure Tracking and Quality of Service Delivery Survey in the Health Sector: Findings and Implications. Unpublished Report, Zambia Ministry of Health, University of Zambia, Swedish International Development and Cooperation Agency and Human Development Department, World Bank.

Yadav (2007) “Analysis of the Public, Private and Mission Sector Supply Chains for Essential Drugs in Zambia”, Unpublished Report, Medicines Transparency Alliance (MeTA),.

Zambia Central Statistical Office (2004) “Living Conditions Measurement Survey” Zambia National Malaria Control Centre “2006 Malaria Indicator Survey” Zambia National Malaria Control Centre “2008 Malaria Indicator Survey”

11

ANNEX 1Annex Figure 1: System A: District Stores Cross Dock plus Commodity Planner (CP)

12

Annex Figure 2: System B: Cross Dock plus Commodity Planner (CP)

.

Annex Table 1. Selected District for Public Sector InterventionDistrict Province System A - District stores plus commodity plannerMwense LuapulaMilenge LuapulaKafue LusakaNakonde NorthernMufumbwe NorthwesternKabompo NorthwesternChoma SouthernShangombo WesternSystem B - Cross dock plus commodity plannerMkushi CentralChama EasternKasama NorthernMungwi NorthernChavuma NorthwesternMwinilunga NorthwesternMongu WesternKaoma WesternControl Group- No public sector interventionSerenje CentralLundazi EasternKaputa NorthernChinsali NorthernMazabuka SouthernGwembe SouthernNamwala SouthernLukulu Western

Annex Table 2. Selected District for Private Sector Intervention District Province Kasama NothernMilenge LuapulaLundazi EasternChinsali Northern

14

ANNEX 2

ANNEX: SAMPLING AND POWER CALCULATION

According to the 2006 PETS data, 81% of facilities experienced some outage of ACTs in the 3 moths prior to survey. A study of 8 districts per arm is sufficient to identify a decrease of 33 percentage points in the stock outage rate, at a standard significance level of .05 and a power of .8, given the following assumptions:

15% of observed variation in facility stock-out rates is attributable to observed district characteristics.

An average of 15 facilities per district. A hypothesized intra-cluster correlation of .35 (the observed PETS data correlation is .24)

If instead we use the observed intra-cluster correlation (.24), then we would be able to identify a 28 percentage point reduction in the stock outage rate at the same levels of precision and power.

Distinguishing the relative improvements in stock outages across different models will be more difficult at standard levels of precision if the performances of each pilot intervention are expected to be more similar to each other in relation to the control group. However each pilot should lead to a great deal of procedural learning which will help to discern the effectiveness and costs of each intervention.

With regard to the household survey the sample will be composed of households from 8 districts. This includes the two districts that receive both the public and private sector interventions, the additional two that receive the private sector intervention alone, and two districts that receive the public sector intervention (one type A district and one type B district in order to match the two cross-over districts), and two controls. The four sampled districts that do not receive the private sector intervention will be purposively chosen to match the four private sector districts in terms of peri-urban or rural location, region, and epidemiologic profile.

According to the 2006 LCMS, 7.6% of the population (all ages) self-report symptoms of fever/malaria in the 4 weeks before survey. Of this number, 90% either seek care at a facility or self-medicate. Given the lack of more specific data it is difficult to determine the number of fever cases that received AL within 24 hours of onset of symptoms – the 2006 MIS reports that 8% of children under 5 had done so. We choose the deliberately conservative supposition that by 2008 20% of fever cases had received AL within 24 hours of onset of fewer (here a conservative value is a higher value).

A survey of 4800 households comprising 240 communities (20 sampled households per community) in eight districts is sufficient to identify a increase of 20 percentage points in the proportion of the population either receiving AL or a diagnostic test within 24 hours of onset of fever in any of the treatment arms (public sector intervention, private sector intervention, or both), at a significance level of .01 and a power of .99. This is based on the following assumptions:

An average household size of 5 individuals. Health seeking behavior at baseline as per the above discussion. A hypothesized intra-cluster (within community) correlation of .10 with regards to

individual health seeking behaviors.

15

The same sample size and assumptions are also sufficient to discern a difference of 10 percentage points in the proportion of the population either receiving AL or a diagnostic test within 24 hours of onset of fever between any two treatment arms at a standard significance level of .05 and a power of .8.

16

ANNEX 3ANNEX: RATIONALE AND PLAN FOR COLLECTION OF DATA

Regular administrative data on facilities and supply chain management, such as the HMIS and MSL reporting, does not cover all key variables necessary to track and evaluate the described interventions. To complement the existing administrative data streams, the Bank is collaborating with the MOH, NMCC, MSL, the University of Zambia and CPs to upgrade the reporting system and develop the instruments required to capture the effect of the interventions in the public and private sectors.

To evaluate the public sector intervention, the following data will be collected:

Administrative Data: Monthly reports and summaries from MSL will be produced to allow tracking of drugs. All partners involved in the ZAAI have agreed on the list of tracer products and other data elements to be collected. In addition, the first stage baseline data will be used to monitor the performance in terms of stock outages at the facility level.

Dedicated Surveys: Key data provided through surveys of public facilities and patients include: i) facility type; ii) access to electricity / water / sanitation / telephone; iii) personnel per capita; iv) HR (absenteeism, ghost workers, staff education, qualified staff) v) working equipment; vi) material (stock of drugs and commodities, stock outs of tracer items, etc.); vii) incentives (compensation, salary retention, delays in payment, supervision and its frequency, public resources share in revenues, public resources per capita, external resources, donor support, user fees, user fee’s share of client income, informal payments); viii) service outputs (number of consultations, etc.); ix) provider KAP for malaria related illness; and x) consumer satisfaction (accessibility, affordability, quality, timeliness, information value, etc.).

In the private sector, the combined ACT and RDT subsidy intervention necessitates improved mapping and stock taking of various levels of private outlets, and improving the tracking of drugs, specifically fever/malaria related drug stocks and diagnostic capacity. The evaluation of the private sector intervention will require:

a. A dedicated private facility and wholesaler survey (baseline and follow-up)

b. Audits, including use of ACT tracing through uniquely labeled packagesc. Exit Interviewsd. Mystery Shopping

The (a) dedicated survey will provide a mapping of the outlets (location, catchment, concentration, accessibility, etc.) as well as information on stocking, prices and malaria related knowledge of retailers. (b) Audits will enable measuring, for example, the effect of volume incentives by recording sales. (c-d) Exit interviews and mystery shopping will shed light on consumer choice factors, the availability of ACT and diagnostics, and ascertain mark-up information.

Administrative data and facility surveys suffer from well know limitations, such as i) concerns regarding quality (e.g. reporting system capacity, potential for data tempering); ii) representativeness (biased estimates due to sample selection; e.g. the difference in

17

results/variables for those who seek care at facilities and those who self-treat can be systematic); and iii) little complementary data on important covariates (e.g. household socio-economic status, education, health status, labor productivity, etc.), as well as the effect of the intervention on these household variables over time.

To offset these limitations, the evaluation will include a household survey. A combined malaria indicator and socioeconomic household survey will be administered in randomly selected households in both the control and intervention districts prior to the intervention (baseline), and one year following the baseline survey (follow up).

The household survey modules will provide data on: household composition (age, gender, etc.), consumption, assets, education, labor supply, health seeking behavior, fever/malaria episode-related KAP, history of malaria within the household, treatment seeking behavior, WTP for anti-malarials, fever/malaria related expenditures, and opportunity costs of illness.

In addition, the survey will collect biomarker tests: parasite prevalence, hemoglobin, and anthropometry will be collected from all household residents. Upon consent from the household member or his/her guardian, parasite prevalence will be tested using rapid diagnostic test kits (RDTs). The procedure is mildly intrusive, whereby a small sample of blood is taken by standard finger-prick methods using a sterile lancet (the same sample will be used for hemoglobin assessment). Trained public health technicians will be responsible for all blood collections.

All surveys will be performed according to the international guidelines for human experimentation in clinical research. Ethical clearance for the surveys will be obtained from the MOH prior to fielding the surveys.

Complementary data will also be collected to track and understand the effect of potential confounders, and to, ideally, ensure lack of contamination between treatments and control groups, or, if unavoidable, to best mitigate these effects during the analytical work. Complementary data includes: i) monthly weather statistics; ii) community factors, including changes in behavior communication related to fever/malaria prevention and treatment, etc.; and iii) specialized agency consultations (MOH, NMCC, NRA, MLS, DHMT, etc.) to track/control for confounding interventions, such as introduction of new programs (e.g. additional preventive intervention ITNs, IRS etc. through other donors; changes in the regulatory regime, etc.)

In addition to tracking operational progress and the impact of the interventions on the population, the study includes a rigorous costing and cost effectiveness component. The cost-effectiveness analysis will provide evidence on the relative costs and consequences of different interventions in order to assist in priority-setting and budget allocation. Costing will inform on accounting and economic costs of the interventions. Cost effectiveness will inform on the gross (incremental cost of intervention only) and net costs (incorporating potential cost savings as a result of the intervention, measured e.g. as cases averted, reduction in productivity loss, etc.) of the interventions.

Specifically, Cost data will be analyzed to assess the unit and incremental costs of the alternative

supply chain management modalities (public sector intervention), and the ACT and diagnostic subsidy (private sector intervention). These will be compared to the baseline case of the existing system and alternative treatment regimes.

18

An ingredients approach will be used to calculate unit costs. Resource categories will be assessed independently to identify the more costly components of the given public supply chain system.

The primary resource categories include: personnel, equipment and supplies (e.g., ACT, RDT, etc.), vehicles and transportation, training, community sensitization, initial capital investments (for buildings, hiring staff), and overhead. Identifying the relative contribution of resource categories to the total cost of each intervention is useful for improving resource allocation decisions and overall program efficiency. Both financial (e.g. reflecting actual market prices) and economic costs (reflecting the true opportunity costs of donated inputs and volunteer time) will be analyzed and compared to assess issues of affordability and sustainability. All cost analyses will be divided to illustrate the costs incurred at the start of the project versus maintenance costs after program has been operational for a period of time. The analysis will emphasize the provider perspective, but we will also assess the direct (e.g., savings to individuals and households from medical treatment averted) and indirect costs (e.g., the value of lost work time from illness and care-seeking) to individuals and households.

19