![Page 1: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/1.jpg)
Plugging the Swazi Sink
SWAZILAND COUNTRY TEAM
BONGILE KANYA BONGANI MOTSA QUMARUL HASAN MARTINA BJORKMAN NYQVIST KELLY ZHANG ANNA WIELOGORSKA (TA)
ieGovern Impact Evaluation Workshop Istanbul, Turkey January 27-30, 2015
: #ieGovern
![Page 2: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/2.jpg)
2
What is the problem?
![Page 3: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/3.jpg)
What is the problem?
Intervention(s) Health sector - Training of personnel dealing with procurement on how
to use and comply with the framework agreements and consequences of non-compliance
- Auditing adherence to the procurement procedure
1
![Page 4: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/4.jpg)
Research questions • What is the effect of training and auditing on value
for money in procurement? Mechanisms • Training improves efficiency and compliance. • Auditing improves compliance. Results Chain • Auditing and Training in procurement -> improves
compliance -> improves efficiency -> improves service delivery
Research Questions
2
![Page 5: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/5.jpg)
Method • An RCT with 3 treatment arms and 1 control arm
• Audit only, Training only, Audit and Training, and control group
Clusters / sample size • Randomization at the health center level • 215 health centers
Other • Within the audit treatment arm we will randomize
independent and internal auditors. • Within the training arm a randomly selected subset will
also receive training in demand forecasting.
Methods & Design
3
![Page 6: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/6.jpg)
Main outcomes
• Prices paid for supplies
• Availability of supplies
• Number of call-off orders
• Transaction cost
Unit(s) of observation for survey:
• Health centers
• Households
• Suppliers
Secondary outcomes
• Utilization of health facility
• Quality of health services
• Child health
Outcomes & Measurement
4
![Page 7: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/7.jpg)
#ieGovern Impact Evaluation Workshop
Istanbul, Turkey January 27-30, 2015
Thank You!
facebook.com/ieKnow
#impacteval
blogs.worldbank.org/impactevaluations
microdata.worldbank.org/index.php/catalog/impact_evaluation
http://dime.worldbank.org WEB
![Page 8: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/8.jpg)
“Compras Manaus” - DPL WITH MANAUS, AMAZONAS BRAZIL MUNICIPALITY OF MANAUS
ieGovern Impact Evaluation Workshop Istanbul, Turkey January 27-30, 2015
: #ieGovern
![Page 9: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/9.jpg)
What is the problem?
What is the problem? - High price discrepancy of goods and services relative to the State (Amazonas) and the market. - Bureaucratic processes slows down the speed of procurement - Weak contract control often results in mismatches between goods and services procured and those delivered. Intervention(s) Types: 1.) Informational interventions on inventory and its impact on delivery to end users. 2.) Procedural interventions modifying aspects of procurement processes to test affect on price, time and quality.
1
![Page 10: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/10.jpg)
Research questions • How does use of the electronic procurement system improve
efficiency, speed, and transparency in procurement processes (i.e bid to fulfillment)?
• Mechanisms: 1. Increase access to information on government purchases via different electronic methods 2. Simplify paperwork through electronic tools 3. Increase competitiveness through electronic reverse auction
Research Questions
2
![Page 11: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/11.jpg)
Method Randomized Control Trials (RCT) We will randomize… - venue of auction (in person vs. electronic) - location of public disclosure (newspaper vs. website vs. email) - processes that use electronic signature
Clusters / sample size
Randomization at the transactional level Sample size - 1000 observations per year
Methods & Design
3
![Page 12: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/12.jpg)
Outcomes:
- Reduction in price spread between State and Municipality
- Increase in use of procured goods and services
- Improvements in speed of procurement processes
Unit(s) of observation:
- A procurement process
Outcomes & Measurement
4
![Page 13: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/13.jpg)
#ieGovern Impact Evaluation Workshop
Istanbul, Turkey January 27-30, 2015
Thank You!
facebook.com/ieKnow
#impacteval
blogs.worldbank.org/impactevaluations
microdata.worldbank.org/index.php/catalog/impact_evaluation
http://dime.worldbank.org WEB
![Page 14: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/14.jpg)
Bangladesh: Transforming Public Procurement with e-GP
ieGovern Impact Evaluation Workshop Istanbul, Turkey January 27-30, 2015
: #ieGovern
Md. Faruque Hossain Director General Central Procurement Technical Unit (CPTU) Bangladesh
![Page 15: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/15.jpg)
What is the problem?
What is the problem? - Poor utilization of public funds
• Inadequate transparency • Low efficiency • Issue of integrity in Public Proc
Intervention(s) - Electronic Government Procurement (E-GP) - Capacity development for sectoral agencies - Information disclosure to citizens and civil society
1
![Page 16: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/16.jpg)
Research questions
• What changes in procurement performance are achieved by the
implementation of E-GP? (EGP efficiency of processes, transparency, competition economy/savings)
• What changes in procurement performance are achieved by the
implementation of capacity development program?
• What changes in citizen behavior are achieved by enhanced information disclosure and awareness towards social accountability?
Research Questions
2
![Page 17: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/17.jpg)
Phase I - Retrospective Method
Statistical matching Clusters / sample size
Clusters : procuring entity (about 1000) Unit of observation : tender (contract) Sample size : 2000 - 4000
Phase II - Prospective Method Randomized Control Trial Clusters:
Clusters: procuring entity (000s) Unit of observation: tender Sample size: 000s
Other
Spillovers: administrators, bidders move around
Methods & Design
3
![Page 18: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/18.jpg)
Outcomes:
- Improved utilization of public resources
• Improved lead time
• Reduced processing cost
• Better price
• Improved competition (e.g. number of bidders)
• Enhanced fairness
• Increased transparency
Outcomes & Measurement
4
![Page 19: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/19.jpg)
#ieGovern Impact Evaluation Workshop
Istanbul, Turkey January 27-30, 2015
Thank You!
facebook.com/ieKnow
#impacteval
blogs.worldbank.org/impactevaluations
microdata.worldbank.org/index.php/catalog/impact_evaluation
http://dime.worldbank.org WEB
![Page 20: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/20.jpg)
PROCURING FOOD FOR THOUGHT PACA ZULETA CRISTINA GUTIERREZ GABRIEL TOUREK JURGEN BLUM MICHAEL ROSCITT COLOMBIA COMPRA EFICIENTE
ieGovern Impact Evaluation Workshop Istanbul, Turkey January 27-30, 2015
: #ieGovern
a a
![Page 21: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/21.jpg)
What is the problem?
What is the problem?
Intervention(s) Introducing a Framework Agreement negotıated by Colombia Compra Efıcıente as the procuring mechanism for buying school meals.
1
Measure % of children between 5 and 9 years
Chronic Malnutrition
9%
Underweight 2.6%
Overweight 18.9%
Anemia 8%
1. The Colombian Government spends 500 Million USD
per year on the “Programa de Alimentacion Escolar”; 2. But while data is lacking, there is concern that many
students do not get meals; 3. Education Secretariats current sign contracts (i) with
local NGOs for schools meals through direct contracting: (ii) public tenders: and (iii) auctions;
4. Many kids drop out of school because food is not delivered;
5. Symptoms of malnutrition are widespread.
![Page 22: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/22.jpg)
Research questions
• How does purchasing school meals through the FAs helps delivering PAE (price, quality, timeliness, nutrition and education)?
Theory of Change
Research Questions
2
Purchasing through
Framework Agreements
Cuts through exıstıng patronage
↑ transparency
Aggregrated demand
Standardızed bidding procedures
↑ supervisıon
↑ quality of ınformatıon
Self selection of established fırms
↑ quality
↓ price
↑ provision / timelıness
Bundling of attractive & unattractive schools
(↑ equity)
↑ nutrition & health
↓ drop outs & ↑ attendance
![Page 23: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/23.jpg)
Method RCT at the school level
Clusters / sample size • 4 ETCs (school zones) • 180 schools and 76,000 students + Bogota
Other
Methods & Design
3
Potential Spillovers How to capture / mitigate
ETCs procuring wıth providers of the framework agreement for control schools
All sales by FA providers captured ın CCE system
Use information (price, logistics, etc.) from framework agreements to procure for control schools
(i) Check system ıf treatment schools buying more food than students; (ii) Capture relationships between school administrators; (iii) Outside ETCs as additional controls
![Page 24: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/24.jpg)
Outcomes & Unıt of Observation:
Outcomes & Measurement
4
ETC level School level Student level
Price % of students receivıng meals
Dropouts
Clarity in the contractual process
Meal quality Height / Weıght / BMI
Characteristıcs of provıders
Meal Value Delivered a % of Negotiated ( (# of
meals delivered x Px of Quality )/ (# of students / Px negotiated))
Test scores
# of bidders
• Measurement ıs goıng to be the prımary challenge (for cost & logistıcal reasons)
![Page 25: Plugging the Swazi Sink - World Banksiteresources.worldbank.org/INTDEVIMPEVAINI/Resources/Procurement... · Plugging the Swazi Sink SWAZILAND COUNTRY TEAM BONGILE KANYA BONGANI MOTSA](https://reader030.vdocuments.site/reader030/viewer/2022041222/5e0b87204238d0026118e6fc/html5/thumbnails/25.jpg)
#ieGovern Impact Evaluation Workshop
Istanbul, Turkey January 27-30, 2015
¡Muchas gracias!
facebook.com/ieKnow
#impacteval
blogs.worldbank.org/impactevaluations
microdata.worldbank.org/index.php/catalog/impact_evaluation
http://dime.worldbank.org WEB