World Bank “Detailed Methodology for Procurement Country Systems Piloting Program” (2007)

Evaluation and Comments

10 December 2007

Development Finance International, Inc.

Public procurement is the governmentactivity most vulnerable to corruption.- OECD (Integrity in Public Procurement, 2007)

The multiplicity of rules may have a negative impact on transparency and lead to legal uncertainties and high transaction costs- OECD (Bribery in Public Procurement, 2007)

Procurement agencies may also purposely use and abuse the regulatory diversity. For instance, they may privilege firms by opting for tendering procedures which require no controls. They may also formulate requirements which favour specific firms and constrain market access to specific suppliers.- OECD (Bribery in Public Procurement, 2007)

Industry – WB Shared ObjectiveIndustry – WB Shared Objective: Strengthen the national procurement systems of Bank borrower countries.

Concerns: The proposed pilot methodology and process does not meet

the standards of the World Bank’s current procurement system and 50+ years of improving practices in (i) measuring “equivalence” in selection of pilots and (ii) building local capacity.

The many loopholes and omissions in the current proposal run counter to, and conflict with, the Bank’s strong position against corruption and for increased transparency.

Recommendation:

Proposal and tools should be redrawn to preserve and maintain international best practice in procurement

The WB should convene a technical working group of world-class experts to develop effective tools to assess a country’s adoption and implementation of international best practice in procurement.

The working group should include experienced US and European firms who have been actively involved in analysis of the World Bank’s country system methodologies and the OECD/DAC’s assessment tools and methodologies.

Basis for Detailed Comments:

World Bank procurement guidelines and Standard Bidding Documents side-by-side with:

- Detailed Methodology for Procurement Country Systems Piloting Program, Draft for Consultations and related PowerPoint presentations

- OECD/DAC Version 4 Methodology for Assessment of National Procurement Systems (laws and regs only)

- CPAR data available (WB website)

- Loan agreements where available (limited via WB website data; NCB side letters not available to public)

- 2005 Country Systems proposal indicators

* Capacity Assessment, Risk Assessment tools/details are not available for review

Fails to articulate, maintain, and encourage international best practices

Reduces transparency and accountability, increasing potential for corruption

Undermines reliability and accessibility of procurement information

Decreases protections for biddersin bidding and tendering processes

The 2007 Proposal for Country Systems Pilot Program

Start country-levelQualitative

Assessment Process

CPAR action plan implemented?

OECD/DACCompleted &

Validated

Complete OECD/DAC

NO

YES Review NCB side letter

Flowchart 1: Country Level Qualitative Assessment Process (partial only – see Annex 1)

Mandatory score On

Critical sub-indicators?

Acceptable score on

other sub-indicators

YES

YES

Country is cleared for

equivalency

Fails to articulate and encourage international best practices

Currently

International ProcurementBest Practices

Under Country Systems Proposal

ICB Guidelines largely omitted

UNCITRAL Model Law

not reflected

UNCITRAL Model Law

World Bank ICB Guidelines

Standard Bidding Documents (SBDs)

“…the Bank has developed world class standards, bidding documents and selection procedures that maximize the bidding opportunities and provide for a fair and open playing field for all eligible participants” - WB Detailed Methodology, p. 5

WTO work on transparency and

multilateral guidelines is not considered

New York Convention on Foreign Arbitration

WTOAgreement on Government Procurement

Standard Bidding Documents waived

and replaced by “suggested

minimum content”

“suggested minimum content of bidding

documents” (OECD)– undefined

Reduces transparency and accountability, increasing probability of corruption

Under the proposal, a pilot country would not be required to:

• Apply Bank’s Standard Bidding Documents to international tenders (2E)

• Adhere to Bank norms on fraud and corruption and demonstrate a strong anti-corruption enforcement record (12B)

• Ensure competitively selection of country procurement personnel based upon procurement knowledge and skills (6A)

• Ensure that compliance auditors are sufficiently informed about procurement to conduct quality audits that contribute to compliance with regular reporting of issues to management (6A)

• Conduct effective procurement data collection, analysis, reporting (5B)

• Make Codes of Conduct / Ethics obligatory or enforceable (12G)

• Disclose procurement system deficiencies until after loan execution (and then only in loan agreements) (n/a)

Undermines reliability, transparency, and accessibility of procurement information

Under the proposal, a pilot country would not be required to:

• Make all procurement information publicly available free of charge and both comprehensive and easy to understand (5A)

• Specify time frames for disseminating procurement information (invitations, procurement notices or awards) (5A)

• Use or regulate model tender documents – where documents exist, only a minimum set of clauses is required (2B)

• Regularly update and make transparent implementing regulations for the country’s procurement systems (2A)

• Possess a procurement manual for contracting entities (2E)

• Maintain procurement documents in accordance with international standards (6)

Decreases protections for biddersin bidding and tendering process

Under the proposal, a pilot country would not be required to:

• Expressly state all evaluation criteria and stipulate lowest evaluated cost as the sole award criteria (1F)

• Provide a clear method for evaluating non-price criteria in goods and works tenders (1F)

• Specify the amount of time for bid preparation (1E)

• Supply prospective bidders with all information pertaining to a bid

• Ensure that all pre-qualified applicants are allowed to bid (2C)

• Supply all bidders with the record of bid opening and explicitly deny bids received after the close date or not read publicly (1G)

• Guarantee formal recourse for complaints to an independent body such as the World Bank (1H)

• Specify in advance in the bid documents how currency and payment will be handled (1E)

Public Procurement Legislative and Regulatory Framework

Existence of Implementing Regulations / Documentation

Public Procurement System Integrated with Governance System

Country has a Functioning Regulatory Body

Existence of Institutional Development Capacity

Country Procurement Operations and Practices are Efficient

Functionality of the Public Procurement Market

Existence of Contract Admin. and Dispute Resolution

Country has EffectiveControl and Audit Systems

Efficiency of Appeals Mechanism

Degree of Access to Information

Country has in place Ethics andAnticorruption Measures

1a 1b1c 1d 1e 1f 1g 1h

2a 2b2c 2d 2e2f

3a 3b 3c 3d

4a4b 4c 4d

5a 5b 5c 5d

6a 6b6c 6d

7a 7b7c 7d/9e

8a 8b 8c

9a 9b 9c 9d

10a 10b 10c 10d 10e

11a

12a 12b 12d 12e12f 12g12c

Requires 3

Requires 2 + Capacity

Development Plan* Requires 2

OECD/DAC Tool vis-à-vis 2005 WB

Proposal* Set forth in Loan Agreement only; no methodology for monitoring; consequences for non-compliance unclear

2005 proposal: Withdrawn amidst concerns that procurement standards were “set too low” by making only 35% sub-indicators “mandatory”

2007 proposal: While 40% of sub-indicators now require “full compliance” for country systems pilot program eligibility, some have been downgraded, others watered down. Many are less rigorous than WB’s ICB.

1a 1b1c 1d 1e 1f 1g 1h

2a 2b2c 2d 2e2f

3a 3b 3c 3d

4a4b 4c 4d

5a 5b 5c 5d

6a 6b6c 6d

7a 7b7c7d/9e

8a 8b 8c

9a 9b 9c 9d

10a 10b 10c 10d 10e

11a

12a 12b 12d 12e12f 12g12c

Requires 3

Requires 2 + Capacity

Development Plan Requires 2

40%

60% OR

Check OECD/DAC Tool Against WB Rated Indicators

Equivalence Methodology Comments

“Detailed” Methodology is vague on key procedures

• OECD/DAC baseline indicators are substantially more ambiguous than current ICB Guidelines; even “critical” indicators do not require the same degree of rigor as the current ICB Guidelines

• OECD/DAC v.4 lacks meaningful operational assessment tools or methodology

• Unclear how OECD/DAC v. 4 Benchmarking Tool will be applied or how it will be validated (NB: Will it remain a country self-assessment and self-validation as it is today?)

• Unclear why only a select few deficiencies require improvement through a Capacity Development Plan. Further the deficiencies will be included in the loan agreement only post-effectiveness. Today, loan agreements are withheld by the Bank pending effectiveness and are not easily available to the public, if at all, on Bank’s website.

• Unclear why certain sub-indicators are “critical” meaning required to qualify for pilot, others are “critical” but not required to qualify for pilot, and others “non-critical”

“Detailed” Methodology is vague on key procedures

• Unclear how the Bank’s CPAR, NCB Side Letter, and OECD/DAC assessments are factored into a measurement of “equivalence”

• Lacks comprehensive details of the “step-by-step analysis” to determine “equivalence” (who is involved, how it proceeds, how it is assessed and validated, and to whom testing results will be made available)

• Unspecified time frame for implementation of agreed upon Capacity Development Plan or guidance regarding assessment and validation

• No evident procedures for assessing progress required by the Loan Agreement, or specifics should a country fail to take effective actions.

• No detail provided for new project risk assessment or capacity assessment tools and methodology

Summary: ICB Guidelines Acceptably Covered by Required Sub-Indicators

GL 2.7-2.8: Notification and Advertising*

GL 1.10: Joint Ventures

GL 2.43: Dispute Settlement*GL 1.15: Fraud Provisions / Bid Documents of Large Contracts**

GL 1.12: Misprocurement

GL 2.19-2.20: Standards and Brand Names

Summary: ICB Guidelines Acceptably Covered by Required Sub-Indicators

GL 2.7-2.8: Notification and Advertising*

GL 1.10: Joint Ventures

GL 2.43: Dispute Settlement*GL 1.15: Fraud Provisions / Bid Documents of Large Contracts**

GL 1.12: Misprocurement

GL 2.19-2.20: Standards and Brand Names

GL 1.16: Procurement Plan

GL 2.13-2.14: Bid Validity and Security

GL 2.21-2.23: Pricing

ICB guidelinesacceptably covered

12%

ICB guidelinesNOT acceptably covered

88%

Conclusions

• The 2007 methodology for assessing laws / regulations does not improve upon the withdrawn 2005 proposal.

• The proposal offers no methodology to assess day-to-day operational procurement practices.

• The proposal does not measure equivalence with current Bank procurement rules and practice.

• Essential elements of the proposal are omitted (compliance and risk assessment tools)

Fails to articulate, maintain, and encourage international best practice

Reduces transparency and accountability, increasing potential for corruption

Undermines reliability and accessibility of procurement information

Decreases protections for biddersin bidding and tendering processes

A step backwards in effective and accountable international procurement

Counter to good governance and sustainable economic growth

The 2007 Proposal for Country Systems Pilot Program

=