1

Annual Review - Summary Sheet

Summary of Programme Performance

Year 2016 2017

Programme Score A B

Risk Rating Moderate Major

Summary of progress and lessons learnt since last review The Pollution Management and Environmental Health (PMEH) programme is a Multi-Donor Trust Fund which assists selected countries in reducing their air, land, and marine pollution and emission levels by generating an improved understanding of the pollution and its health impacts. It will also promote increased awareness amongst policy makers of the opportunities to reduce greenhouse gas (GHG) emissions and short lived carbon pollutants (SLCPs) while improving public health. In the air quality management (AQM) component of the programme, recipient countries are currently Egypt, China, Ghana, India, Nigeria, South Africa and Vietnam, though more may be added later. The programme is delivered by the World Bank with contributions from the UK, Norwegian and German governments. Other donors are expected to join the programme in the future. BEIS have contributed £12.5m (34% of $47.8m total current donor pledges), primarily for purchasing air quality monitoring equipment, with a further £7.6m contributed by DFID for the research elements of the programme. The BEIS-funded element of the programme has been delayed this year, causing many milestones to be missed, and the reduction in the programme’s score. The delay has primarily been caused by a procurement issue, which has taken most of this year to resolve: originally recipient countries were going to purchase the PMEH-funded air quality monitoring equipment themselves, but the Secretariat realised this would take up to two years, so have instead been given permission for the World Bank to directly procure the equipment. PMEH will act as a pilot programme for this type of procurement within the World Bank. There has also been a lack of capacity in the Secretariat and in the World Bank country offices that has contributed to the delay. While no air quality monitoring or lab equipment has been purchased this year, the Secretariat used the delay to continue providing technical assistance to recipient countries (particularly China and Nigeria), and to gather more evidence about the equipment the countries need to install to achieve the programme’s objectives. The Desert Research Institute (DRI), which has internationally recognised expertise in air quality measurement, was contracted to conduct scoping studies in all seven recipient countries, looking at what monitoring arrangements and lab facilities they currently have and making recommendations for where PMEH funds could best be used. The DRI has found that many of the cities have reasonable ambient air quality monitoring capacity (which provides on-going data on pollution concentration levels), but that “source apportionment” technology (which provides detailed analysis of air samples so that the source of pollutants can be identified) is much less developed. The PMEH Secretariat must now make sure that each city plan is re-evaluated based on the DRI’s findings, and funding is focused in the places where support is most required and on activities that offer most value for money. Given the on-going delays, the programme’s risk level has increased. The next year will be crucial, as the programme will have to ramp up and make rapid progress as it enters its implementation phase to make sure targets are met before the end of the programme in 2020. It is important that the Secretariat builds strong relationships with recipient governments, and draws up clear and achievable delivery plans to guide the work in each country.

Title: Pollution Management & Environmental Heath (PMEH)

Programme Value: £12.5m (34% of $47.8m total current donor pledges)

Review Date: December 2017

Start Date: 19th December 2015

End Date: December 2020

2

This year there have been challenges in the relationship between the World Bank and donors, as governance arrangements and the role of the Steering Committee have not been properly defined. We are working through these issues at the moment. Communication between the World Bank and donors has also not always been clear, often because of a lack of understanding of donors’ priorities but also because of a lack of resources in the Secretariat. Improvements have been made, which have been welcome, and we are keen to continue to work in partnership with the Bank to find solutions, as we enter the challenging next stage of the programme. Summary of recommendations for the next year

The programme must now complete the planning stage and move to implementation very quickly

so that equipment in each country can be purchased in the next year. To do this, revised

administration arrangement documents, reflecting the changes required to allow procurement

to go forward, must be presented to donors. The version for BEIS must include a revised

encashment schedule, reflecting the new project timeline. Deadline for completion – end of

December 2017. Responsibility – head of the PMEH Secretariat, Jostein Nygard.

The governance structure must be agreed for the programme. This must be based on

consensus decision making by the Steering Committee using the World Bank’s “B2” governance

model, in which the Steering Committee endorses annual work plans. This must be set out in the

committee’s terms of reference. Date for completion – December 2017. Responsibility – head of

the PMEH Secretariat.

The Secretariat must produce an annual work plan for the programme each year, which can

then be endorsed by the Steering Committee. This should include an overarching timeframe for

the programme, a Gantt chart (or similar) listing activities that will take place each month, a list of

implementing partners in each country, budget forecasting, and information on how the plan for

the AQM component will reflect climate co-benefits. This will not mean that the Steering

Committee will need to approve budgets on an annual basis (as the programme requires multi-

year contracts), but that we can see forecasts to understand how the money we have committed

is being spent and when further drawdowns will be needed. Date for completion of first plan –

December 2017 Steering Committee meeting, with potential for a revised version in May 2018

reflecting developments in the programme. Responsibility – head of the PMEH Secretariat.

The Secretariat must finalise detailed city work plans. These work plans need to:

o Be informed by the technical scoping work of the DRI

o Include details of the air quality monitoring equipment and lab facilities that are required,

show plans for how they will be staffed and maintained on an ongoing basis, and outline

which organisations will be involved in the projects

o Show how the plans will improve the measurement of SLCPs and other GHGs, and also

how they will support the integration of climate change considerations into air quality

management policy (for example by involving those who work on climate change in the

cities or countries)

o Have the technical specification of equipment reviewed by the Technical Advisory Group

(TAG)

o Be agreed with relevant local and national authorities, and endorsed by the PMEH

Steering Committee before going forward.

Date for completion – December 2017 Steering Committee meeting, but for Egypt, the process

should be completed by email by end of January 2018. Responsibility – head of the PMEH

Secretariat.

Given what has been learnt in the last year about existing capacities in cities and delays to the

programme, the Secretariat and the Steering Committee must review the city work plans and

3

consider whether the programme should be restructured to create maximum value for money and

impact before the 2020 deadline. The Steering Committee must agree a set of principles that set

the framework for project funding and can guide this work. These should at least cover evidence

of continued need for ODA finance, that there is continued local government engagement, and

continued confidence in the delivery of the project in the time available. Where this is not the

case, budgets would require readjusting so that BEIS’s capital funds are concentrated in the

countries that require the most support. Date for completion – end of February 2018.

Responsibility – the Steering Committee.

Because of the delivery risk of the programme and the amount of progress that needs to happen

in the first six months of this year, an interim review will be held in the first half of 2018. This

will be lighter touch than the annual review. The Secretariat, BEIS and other donors will review

progress in each country and the programme overall, and consider whether further

recommendations or restructuring are required to ensure success. Date for completion – May

2018. Responsibility – BEIS programme lead Laura Aylett.

To facilitate planning and implementation of this complex programme, the World Bank must

increase staff resources working on the programme in the Secretariat and country offices,

particularly technical experts. This will free up the time of the programme managers and co-

ordinators. We recommend that a senior “operational officer” role is identified in the team, who

has responsibility for pulling together work plans, and ensuring smooth co-ordination and

communication between the different elements of the programme. The Secretariat should put

forward a resourcing paper to the Steering Committee, showing the roles they need. Date for

completion – April 2018. Responsibility – head of the PMEH Secretariat.

The Secretariat must develop an “operational manual” that is agreed with the Steering

Committee. It should include:

o the terms of reference of the Steering Committee and the Technical Advisory Group

o details of how meetings are run and how quickly meeting minutes are produced

afterwards (we recommend rotating the role of chair between the members of the

committee)

o standards for communicating with donors (for example, responding to enquiries within five

working days and providing quarterly updates on progress)

o a set of principles to govern the work of PMEH (for example transparency, additionality,

deliverability, and local ownership)

o a process for risk and issue management that would include proactively raising high-level

risks and issues with the Steering Committee and agreeing mitigation approaches and

resolutions. These would be discussed at Steering Committee meetings and in-between

as and when risks or issues materialise

Date for completion – April 2018. Responsibility – head of the PMEH Secretariat.

Because of the critical nature of this year, the Secretariat should hold monthly teleconferences

with donors to provide updates on progress. Date for completion – monthly from January to

December 2017. Responsibility – head of the PMEH Secretariat.

Following the approval of the overarching work plan, the programme’s logframe must be

revised so that output indicators and milestones better reflect the new timeline following the

procurement delay, and the detailed plans in each city. Outcome and impact indicators should

also be reviewed to consider whether they are the most relevant measures of success. The

theory of change should also be reviewed to see if it should be updated based on more recent

evidence. Date for completion – January 2018.

An evaluation plan must be completed for the programme, including the timing of the mid-term

evaluation (which will need to be delayed slightly to reflect the delayed timeline of the

programme), and evaluation questions. This should be agreed with the Steering Committee. Date

4

for completion – Steering Committee meeting in December 2017. Responsibility – head of the

PMEH Secretariat.

British Embassies and High Commissions in the seven recipient countries should be further

engaged in the PMEH programme to help monitor progress, share experience, improve UK

visibility on the project (potentially through the use of UK branding), and to see if there are

commercial opportunities that could be advertised to UK companies. Date for completion – this

has already begun, but should continue throughout the year. Responsibility – BEIS programme

lead.

Links should be made to related ICF programmes by introducing the PMEH team to the

people involved and encouraging collaboration. Programmes include the 2050 Calculator (which

has modelling projects in China, India, Vietnam, South Africa and Nigeria), and the engagement

with cities through C40. Date for completion – January 2018. Responsibility – BEIS programme

lead.

5

A. Introduction and Context

Link to Business

Case:

http://www.aidstream.org/files/documents/PMEH-Business-Case-

redacted-version.pdf

Link to Log frame: https://aidstream.org/files/documents/PMEH-logframe-December-

2017-20180105080132.pdf

Link to previous

Annual Review (if

appropriate)

https://aidstream.org/files/documents/PMEH-Annual-Review-

November2015-November2016-20170206030218.pdf

Outline of the programme

BEIS has pledged a total of £12.5 million over five years (from FY 2015-16 to FY 2019-20) to fund the air quality and GHG emissions-linking component of a World Bank-executed Multi-Donor Trust Fund (MDTF) programme on Pollution Management and Environmental Health (PMEH). Other donors are currently Norway’s Ministry of Foreign Affairs, the United Kingdom’s Department for International Development and Germany’s Ministry for the Environment, Nature Conservation, Building and Nuclear Safety whose total commitment’s takes the MDTF to $47.8m. The BEIS pledge therefore totals 34%. The PMEH programme has five components. Components 1-3 focus on implementing pollution abatement and remediation activities in the air, on land and in the marine environment respectively. Component 4 (currently solely funded by DfID) is research-based and focuses on generating new academic and scientific knowledge, as well as researching and developing science-based solutions to tackle and reduce pollution levels in developing countries. Component 5 provides ongoing communications support for Components 1–4. BEIS’s contribution is principally funding activities under component 1, which focuses on air pollution. The funding will be used to provide high quality, comprehensive, air quality monitoring and lab equipment to a number of cities and city clusters in highly polluted low-income and middle-income countries. This equipment will monitor a range of pollutants depending on the exact local requirements (including PM10, PM2.5, black carbon, carbon monoxide, SOx, NOx, ozone, and potentially methane). It will cover both ambient monitoring stations, so that air pollution concentrations can be tracked in real time, and sampling stations accompanied by advanced analysis that shows the source of different pollutants (for example if they come from transport, industry, power stations, agriculture or households), and their contribution to pollution concentrations. The equipment will become part of an integrated AQM planning process, which follows the PMEH framework. This is made up of ten key steps, including ambient air quality monitoring, sampling, chemical composition analysis, emissions inventory, source apportionment, and health impact assessments. A small portion of the BEIS donation will go towards Component 5, funding monitoring “hubs”, which will live-stream pollution data online to raise local and global awareness, as well as knowledge sharing and dissemination activities to allow countries to learn from each other. As many of the air pollutants that have negative health impacts share common sources with greenhouse gases, including SLCPs, solutions to improve air quality can lead to the reduction of other GHG emissions and vice versa – this programme will allow policy makers to make an explicit link between air pollution, public health, and climate change mitigation. The demonstrated linkages between the sources of air pollutants and GHG emissions, alongside cost effective actions to mitigate them both, will enable improved policy action in recipient cities and the wider countries.

6

B: PERFORMANCE AND CONCLUSIONS

Annual outcome assessment The outcome we are expecting, as described in the logframe, is “improved access to evidence of the

level and scale of air pollution and their linkages to GHG emissions amongst policy makers and general

public in participating countries”. Although concrete progress in the BEIS-funded part of the PMEH air

pollution programme to deliver this has been slow this year (see output score section below for more

details), the Secretariat has used the time to find out where best to use funds to ensure value for money.

Through this process, the PMEH team has learnt that cities are already improving their access to air

pollution evidence, as many are expanding their network of ambient air quality monitors without PMEH

support.

Because of this, the PMEH programme is shifting emphasis to focus on more advanced source

apportionment techniques, as the cities have low capacities for this currently. International expertise is

required to advise on the equipment needed and to share the skills required for this analysis. This has

been an important process of developing the programme’s “niche” and finding where it can add most

value. The logframe will need to be adjusted to reflect this change, so that the impact of the re-focused

programme can be properly monitored.

The logframe’s impact section focuses on the development of full AQM plans in each city, which will be

informed by data from BEIS-funded equipment. The successful completion of these plans will depend

not just on the installation of the equipment, but also on the wider PMEH air quality programme, which

includes technical assistance workshops, the introduction of new modelling techniques, and knowledge

sharing between countries. Progress in these areas has continued this year, and is less behind schedule

than the BEIS-funded components. However, the Secretariat must continue to ramp up pace in this area

as well, engaging relevant authorities in all the cities and ensuing they are bought into the process. This

is a very complex programme, and so far the level of progress has varied a lot between cities.

PMEH also aims to have an impact by mobilising public finance. Progress has been made in this area.

The World Bank has started a $500m lending project in China using their “Program-for-results” (PforR)

financing instrument, which will run from 2016 to 2018. PforR provides loans to countries to support their

own programmes that strengthen institutional capacity, and disburses funds when results are achieved

and verified. This PforR project focuses on Hebei province, which is the main source of pollution in the

Jingjinji region surrounding Beijing, and will focus on implementing air pollution control measures like

introducing cleaner busses, replacing cooking stoves in households and developing an emissions

inventory system. The group of international experts who developed the project with the Government of

China were recruited by the PMEH programme, and this PforR funding will be integrated into the PMEH

project in Jingjinji, supporting the implementation of the region’s AQM plan. While this project is clearly

linked to PMEH, we are investigating whether the programme had a formal role in mobilising the finance.

So progress is being made towards the programme’s aims, but the pace needs to ramp up significantly

in the next year to ensure that everything is completed before the 2020 deadline. This will require strong

programme management from the Secretariat and implementation by the World Bank’s country offices,

which currently have low levels of AQM capacity.

Overall output score and description

Score: B

This year progress on PMEH has slowed, primarily due to a corporate-level procurement issue – in late 2016 it was discovered that using recipient-executed (RE) funds for air quality monitoring equipment would lead to severe delays in the programme. RE means that the cities or countries would purchase the

7

air quality monitoring equipment themselves through grant agreements between the Bank and host countries. It was discovered that in most cases it would take from 18 months to two years to prepare such agreements, usually because of lengthy internal project development and clearance processes in host countries. The Secretariat therefore asked for an exception to the Bank’s procurement policies so that they could use Bank-executed (BE) funds instead, meaning that World Bank contractors would procure the equipment for all seven countries, either centrally or, more likely, by the World Bank’s country offices. By procuring centrally or regionally, this may also allow PMEH to benefit from economies of scale, and ensure that the equipment purchased is uniform in quality and that the software is compatible across countries. In September 2017, the World Bank Procurement Committee formally approved that PMEH can use BE funds, and will act as a pilot in the Bank for using this approach for capital goods in complex projects like this. The equipment will be owned by contractors during the period of the programme, and will then transfer to the local authorities at the end of the contract period. The Secretariat is now in the process of revising the administrative arrangement documents between the Bank and each donor to allow our donations to be spent in this way. While it is positive that this issue is nearly sorted out, and that the committee has shown trust in the PMEH programme to pilot the new BE procurement scheme, it has delayed the programme by around a year. Consequently, no further BEIS funds have been released to the World Bank this year and all output milestones regarding the installation of monitoring and lab equipment (Outputs 1-2) have been missed. They have therefore both scored C. The PMEH Secretariat has used this delay to do more technical scoping work. They have contracted internationally recognised experts on AQM from the Desert Research Institute (DRI) to perform “deep dives” into the current capacities of recipient cities and to make recommendations for the PMEH funding. The DRI is working with the University of Nevada, Las Vegas (UNLV) on this scoping. This has meant that the plans for each city are becoming much clearer, as well as the potential benefits for their populations. For example, Lagos in Nigeria is the most populated city in Africa but has no air quality monitoring in place at the moment. The PMEH programme will be funding a first network of ambient air quality monitors to provide up-to-date information on air quality to the general public and policy makers. The Secretariat has also continued with the technical assistance and communication parts of the programme during the delay, particularly in China and Nigeria. This is important as it means that momentum behind the projects can continue, even if the equipment is delayed. Scores are therefore higher in Outputs 4 and 5 than the other outputs. However, work on the regional and global hubs to share air quality data online (Output 3) has largely stopped while the other issues are resolved. The Secretariat plans to come back to this issue next year. This Output scored a C for this reason. This year there has been a shift in emphasis in the programme from the original business case proposition, because of the technical assessment by the DRI. They have found that several cities have already got quite developed networks of ambient air quality monitors. PMEH still has an important role to play, however, as the DRI team has found that most cities have very limited (if any) analytical capacity to identify and quantify the sources of pollution. This source apportionment information is crucial for guiding policy, as effective policies can be designed that focus on the major sources of pollution, and progress can be tracked more accurately. When combined with the technical assistance components of the PMEH programme, which will leverage international expertise, this technology is designed to feed into the development of cost-effective, multi-sector AQM plans for these cities and city clusters. The PMEH programme is therefore finding a niche where it can add most value, and can build on the existing capacities of the cities. While work this year has given us a clearer picture of where funds should go, the programme is not yet ready to disperse funds to all cities as soon as the procurement issue is fully resolved. Each city is unique, and it has taken longer than expected to engage local government agencies and formally set up the projects in many countries. This has been particularly true in countries where PMEH aims to work with city clusters, which bring together neighbouring city and provincial authorities, as the number of organisations to involve can be quite large. Work in India has been particularly slow, but in the last few months, progress has been made by working directly with the Delhi city authorities and establishing direct communication with the Central Pollution Control Board regarding comprehensive support by the World Bank through the PMEH agenda, not only in Delhi but also in the wider National Capital Region

8

(NCR). Care must be taken to ensure that the programme is not overly ambitious, given the time available. The budget allocated to each country varies quite a lot. In Egypt, Ghana and Vietnam, only $500,000 has been allocated to each country from BEIS’s contribution to pay for monitoring and lab equipment, compared to $4 million allocated to India, $3.5 million to Nigeria, and $2 million to China. The DRI feedback so far is that this will not cover all the equipment they would recommend. Given what has been learnt this year, it is now time to take stock and consider whether the funding allocations should be changed to focus resources where they can have the most impact and are most needed, based on a set of key principles. The PMEH programme should also consider whether they can work with other agencies to multiply the programme’s impact. For example, the German development agency GIZ is donating €3 million from the German International Climate Initiative to Egypt and Vietnam, which they will use for technical assistance in conjunction with the PMEH programme. This will help with the technical assistance part, though it cannot be used for capital goods. The relationship between climate change and air pollution is a sensitive issue in some of the recipient countries, and responsibility for them are often held in different parts of local and national governments. The PMEH Secretariat has therefore decided to position greenhouse gas reductions as a co-benefit of tackling pollution when approaching governments, rather than the primary goal of the programme. As well as the monitoring equipment funded by PMEH, which will often measure SLCPs, one of the main ways that climate considerations are being considered in the programme is through the introducing the Greenhouse Gas-Air Pollution Interactions and Synergies (GAINS) model for cost-effective abatement options analysis, which allows policy makers to explore how both air pollution and greenhouse gases can be reduced in each city. However, other options should be explored, like involving the local and national agencies responsible for climate change mitigation in the PMEH-funded AQM work, and making sure that key SLCPs (e.g. black carbon, brown carbon and methane) are measured in each city. The links between the PMEH programme and climate must be made more explicit as the programme moves into its operational phase.

Key actions In addition to the recommendations in the summary sheet, the experience of this year has shown that technical scoping is an essential pre-curser to interventions like this. We recommend that the World Bank and other ICF delivery partners use this approach at the beginning of any similar programmes to get an expert view on what is required in the different country contexts, as well as working closely with recipient governments to meet their needs.

Has the logframe been updated since the last review?

As recommended in the last annual review, the World Bank completed the logframe by filling in and updating baselines for all indicators in early 2017. No milestones were changed, except a few 2016 milestones to reflect the revised baseline data as the starting points.

9

C: DETAILED OUTPUT SCORING

Output Title

Procurement, installation and maintenance of continuous and manual air monitoring equipment

Output number per LF 1 Output Score C

Risk rating (Minor, Moderate, Major, Severe)

Major Impact weighting (%): 30%

Risk revised since last AR?

N/A Impact weighting % revised since last AR?

N/A

Key Points

All milestones have been missed because work has been delayed by the procurement issue, as outlined

above in section B, which is in the final process of being resolved following the World Bank Procurement

Committee’s approval to change the procurement method used, acting as a pilot for the World Bank. The

Secretariat have used the delay to do further assessment of what equipment could most benefit the

recipient countries, which will benefit the programme in the long run and has been a very positive

development. However, the country-level plans have not been fully developed yet, so are not ready to go

as soon as the procurement issue is resolved. The procurement process is now expected to begin in the

first quarter of 2018.

Over spring, summer and autumn 2017, the World Bank has been carrying out deep dives in the

programme’s recipient countries. They have contracted the DRI to review current air quality

monitoring and lab capacity in the countries and to make tailored recommendations of what

additional equipment and training would most benefit them. The DRI is internationally recognised

for its expertise in air quality monitoring, and were recommended by the United States

Environmental Protection Agency (US-EPA), the clean air office within Directorate-General of

Environment in the EU Commission, and the Norwegian Institute of Air Research (NILU). They

have completed research missions in six of the seven countries so far, with a visit to the final one,

Egypt, due to take place in December 2017. These recommendations will feed into the full PMEH

country plans for 2018 onwards.

Indicator(s) Milestones from Logframe

Progress

1.1 – % of cities that have procured, installed and are operating all (continuous and manual) air monitoring equipment according to agreed plans.

0 (2016) 70-75% (2017)

Missed – No cities have installed any PMEH-funded equipment yet. Implementation has been delayed because of the shift in procurement execution arrangement.

1.2 – # of (fully-functioning) network hubs installed in participating countries.

10 (2016) 18 (2017)

Missed – No network hubs have been installed by the PMEH programme yet due to the shift in procurement execution arrangement. However, three of the cities have installed hubs this year, increasing the baseline to 13. This indicator needs to be revised next year.

1.3 – % of targeted pollutants being monitored according to agreed plans in participating cities.

0% (2016) 45-50% (2017)

Missed – 10-15% of cities are monitoring the agreed pollutants using PMEH protocols, as China has introduced these to their existing monitoring equipment.

10

The DRI research has found that the air quality monitoring capacities of recipient cities vary

hugely – from Lagos in Nigeria where there is no monitoring network at all, to parts of the

expanded Jingjinji region in China where there is a large network of state-of-the-art ambient

monitors. In all cases, however, they found a need for further equipment and capacity building

support.

Overall, they have found that ambient monitoring is more widespread than the World Bank

thought when they started PMEH, and some cities are currently taking steps to improve their

networks further (largely using their own funds). The programme is therefore changing emphasis,

and focusing more on background (in which pollution concentrations are measured away from

known sources to give a baseline level) and SLCP ambient monitoring, and particularly on

“source apportionment” technology. Source apportionment means sampling the air and

performing detailed analysis of the compounds within it to work out their source (for example,

whether they come from transport, industry, power stations, households or agriculture) rather

than just the total amount of a few select pollutants. This information is extremely important for

policy makers, as it can be used to design effective policies targeted at the actual sources of

pollution. As this is state-of-the-art work, the PMEH programme can add value here by not only

providing the equipment needed, but also training from international experts on how to use it

properly, and support to use the data for effective policy making through its inclusion in the

complete PMEH AQM framework. In some countries, they are also looking to add value to

existing ambient monitoring networks by increasing the number of pollutants being tracked,

including SLCPs.

Three countries that were added late to the programme – Egypt, Vietnam and Ghana – have

been allocated only $500k of capital funds each for ambient monitoring and source

apportionment equipment (taken from BEIS’s donation). This is limiting the scope of what can be

achieved in these countries with PMEH alone (for example, in Ghana the DRI is recommending

that no source apportionment equipment is installed because of the budget restrictions). GIZ are

providing €3m that can be used in parallel to PMEH for technical assistance in Vietnam and

Egypt, and in Ghana, Accra can benefit from the “Centre of Excellence” lab which PMEH will fund

in South Africa, Ghana or Nigeria that will be able to carry out source apportionment analysis.

However, given the information that has been learnt this year from DRI’s scoping, the Secretariat

and the Steering Committee should consider whether the funding allocation to each country is

still appropriate, and whether it would be appropriate to move funds to concentrate on the

projects that are most achievable in the time available and can add most value.

The Secretariat has been made aware of some good practice in India which can be learnt from

across PMEH. The Delhi Government has signed a contract for 20 advanced ambient air quality

monitoring stations that includes operations and maintenance of the equipment for 10 years, and

short-term back-up energy supply at a very favourable price ($6m, so $300k per station). This

can be used as a reference for other cities and states in India, and in other countries.

Summary of responses to issues raised in previous annual reviews

Confirm neutral baseline data is accurate as soon as is practicable – the baselines in the logframe were revised by the World Bank by April 2017. More information has been learnt since then about the current capacity of recipient cities, through the detailed scoping studies of the DRI. It may be necessary to revise these baselines further, and this will take place as part of a wider review of the logframe in December 2017 and January 2018.

Progress recipient country work plans to an implementation phase as soon as is practicable – work plans for six of the seven recipient countries (not including India) were reviewed by the Technical Advisory Group (TAG) and presented at the steering committee meeting in March 2017. While the TAG found that these matched the PMEH AQM framework, they did not have sufficient detail to move to the full implementation phase. The delays caused by the shift from RE to BE procurement meant that implementation has not been possible since

11

then, and the Secretariat has used this time over the spring, summer and autumn to do more detailed scoping work in conjunction with the DRI. The Secretariat aims to present full work plans, including a list of equipment that needs to be procured, for six of the seven countries at the Steering Committee meeting in December (Egypt will follow a little later, as the DRI has not visited for their scoping trip yet).

Establish and commission TAG as soon as practicable so that procurement

proposals/work plans and other recipient country specifics can be scrutinised via a formal

and uniform independent process – the TAG was established in early 2017, and they

presented their findings on the initial work plans of six countries at the Steering Committee

meeting in March. As discussed above, these plans did not include full details of how the AQM

work will be carried out in each country, and since then DRI have done further research to inform

the Secretariat’s recommendations.

Recommendations – reflected in summary sheet

Work plans must be agreed with the relevant local and national authorities prior to their final

approval, to ensure that they meet their needs and are willing to support the process. A

memorandum of understanding (MOU) or a technical agreement should be signed in between the

World Bank and the host country/city in each case, prior to procurement negotiations.

The Secretariat must present updated work plans for all seven countries to the Steering

Committee for endorsement, which should include more detailed plans of what equipment will be

installed in each city and where, a list of stakeholders who will be involved, and how the

measurement of SLCPs and other GHGs will be improved through this plan. This should be

informed by the recommendations of the DRI. Date for completion – December 2017 Steering

Committee meeting, but for Egypt, the process should be completed by email by end of January

2018.

Given what has been learnt in the last year about existing capacities in cities and delays to the

programme, the Secretariat and Steering Committee must review the city work plans and

consider whether the programme should be restructured to create maximum value for money and

impact before the 2020 deadline. This is an opportunity to have a strategic reassessment before

contracts are signed. Date for completion – end of February 2018.

The TAG should review the technical specifications of equipment in the updated work plans and

provide feedback to the Steering Committee on whether they are in line with the PMEH AQM

framework and have scientific rigour. Date for completion – December 2017 Steering Committee

Meeting.

After plans have been approved, PMEH must move to the implementation phase as quickly as

possible. Revised administration arrangement documents reflecting the change from RE to BE

should be presented to donors as soon as possible so that work can begin.

After work plans for each country have been agreed, the indicators and milestones for this output

must be revised to better reflect the new timeline following the procurement delay, and the

detailed plans in each city based on the recent scoping work. Date for completion – January

2018.

In Sub-Saharan Africa, off-grid solar should be considered as an energy source for monitoring

equipment because of the lack of reliable grid capacity, and to reduce the carbon footprint of the

programme. BEIS can put the Secretariat and country offices into contact with some programmes

in DFID who have experience in this area and could offer advice.

The on-going maintenance of the equipment must be guaranteed for the duration of the PMEH

programme (and ideally beyond) through agreements with local authorities or in on-going

maintenance contracts held by World Bank, as is most appropriate. The World Bank should

introduce a process to monitor the operability of labs so that problems can be picked up. Date for

completion – October 2018.

12

Output Title

Procurement, installation and maintenance of lab-based analytical equipment.

Output number per LF 2 Output Score C

Risk rating (Minor, Moderate, Major, Severe)

Major Impact weighting (%): 30%

Risk revised since last AR?

N/A Impact weighting % revised since last AR?

N/A

Key Points

Both milestones have been missed because of the procurement delay, as described above in section B

and Output 1, which has affected the purchase of lab equipment as well as air quality monitoring

equipment. The output therefore scores a C for this year. The procurement issue is now in the final

stages of being resolved. The Secretariat have used the delay to engage stakeholders and to do further

scoping on what lab facilities the countries currently have, what they could benefit from, and where they

should be based, which will benefit the programme in the long run.

DRI researchers have visited six of the seven countries to visit their current lab facilities, as well

as institutions that could potentially host labs in the future. They are making recommendations

about the location and function of labs in each city, which will feed into the overall PMEH work

plans.

These visits have showed that the lab capacity in each city varies a lot, and so a bespoke

approach needs to be taken. In some cases perhaps only some equipment needs to be

upgraded, but in others there is no existing capacity and it will be built from scratch. In those

cases, DRI have looked at what other analytical capacity is available in universities and

government agencies to find the best location to build the new labs.

As part of its research, the DRI is considering where best to place the regional “centre of

excellence” lab in Sub-Saharan Africa, which will provide services to South Africa, Ghana and

Nigeria in the PMEH programme, and potentially other countries too. They will be making a

recommendation in the next few months, but early indications are that they will recommend that

some lab capacity is built up in all three countries, so that a “community of practice” can be

formed, rather than just investing in one lab.

Summary of responses to issues raised in previous annual reviews (where relevant)

Indicator(s) Milestones from Logframe

Progress

2.1 – % of lab-based analytical facilities (servicing participating cities) that have procured, installed and are operating analytical equipment according to agreed plans.

0% (2016) 75% (2017)

Missed – No lab facilities have been installed by PMEH yet due to the shift in procurement execution arrangement.

2.2 – # of lab-based analytical facilities (fully-functioning) and operating at international standards, and servicing the participating cities (as being stated in the PMEH protocol in WP1).

0 (2016) 2 (2017)

Missed - No lab facilities have been installed by PMEH yet due to the shift in procurement execution arrangement.

13

Confirm baseline data as soon as is practicable. The quality of existing analytical labs in participating countries is yet to be fully assessed against international standards. This must be completed and data collated before it will be possible to judge the progress and ambition that will need to be made on these indicators without first undertaking this assessment – The DRI has visited six of the seven countries to assess each city’s analytical capacity, and are making recommendations based on this. The final visit to Egypt is due to take place in December 2017.

The decision on where to site the lab of excellence needs to be made as soon as can be agreed as a multiple of recipient countries will need to base their analytical lab work around this focus point and complicated logistics arranged in order for the chosen solution to work for all countries. This should be achieved in the most collaborative way possible and rapidly to ensure this does not delay implementation – DRI researchers have visited Ghana, Nigeria and South Africa to review their lab facilities, and will make recommendations about where the regional centre of excellence would be best placed.

The ongoing maintenance and sustainable long-term operation of lab based facilities must be written into the respective recipient country plans where it does not currently exist in adequate detail. These facilities have the capability to deliver a long-term diagnostic service to recipient countries. However, ensuring they are funded and maintained will be key to that success – In each country or city, PMEH will agree an MOU or technical agreement with the relevant authorities that will include the conditions that each country will provide to support the programme. This could include land, office space, energy supply and personnel working in labs if required. However, in some cases the on-going staffing and maintenance of the facilities will be included in contracts with vendors until at least the end of the PMEH programme.

Recommendations

Recommendations on lab facilities should be presented to the Steering Committee for

endorsement with the air quality monitoring equipment plans. These should also be reviewed

by the TAG. Consideration should be given to whether the funding allocation should be

changed based on the information that has been learnt this year about each country’s

requirements. After approval, the Secretariat should work towards implementing these plans

and building up lab capacity as quickly as possible. – Steering Committee meeting in

December 2017.

The location and technical specification of labs must be agreed with the relevant local and

national authorities prior to their final approval, to ensure that they meet their needs and are

willing to support the process. Date for completion – April 2018.

The idea of one “centre of excellence” lab should be reconsidered based on what has been

learnt in the last year about the needs of Nigeria, Ghana and South Africa. The Secretariat

should consider whether the funding that has been ear-marked for it should be spent on lab

facilities in each country, or whether it would offer better value for money to spend it on

additional monitors.

After work plans for each country have been agreed, the indicators and milestones for this

output should be revised to better reflect the new timeline following the procurement delay, and

the content of the actual plans (for example, in some countries lab capacities might just need

an upgrade, while in others whole new labs will need to be built – this is not currently reflected

in the logframe). Date for completion – January 2018.

The on-going staffing and maintenance of lab facilities must be guaranteed for the duration of

the PMEH programme (and ideally beyond) through agreements with local authorities or in on-

going maintenance contracts held by World Bank, as is most appropriate. The World Bank

should introduce a process to monitor the operability of labs so that problems can be picked up.

Date for completion – October 2018.

14

Output Title

Develop the capacity of global hubs by adding data from PMEH-focused cities and support the establishment of more local hubs displaying data alongside SLCPs.

Output number per LF 3 Output Score C

Risk rating (Minor, Moderate, Major, Severe)

Major Impact weighting (%): 10%

Risk revised since last AR?

N/A Impact weighting % revised since last AR?

N/A

Key Points

The work on PMEH-funded global and local hubs to collect and publish air quality data has been paused

while the procurement issue (as described above) is sorted out. However, over the last year the

Secretariat have learned that more countries have monitoring hubs than previously thought, so the

baselines need to be revised. Because this extra capacity was not due to the PMEH programme, both

milestones have been missed. While plans for monitoring and lab equipment have been improved during

this pause, the work on hubs is further behind and decisions still need to be made about the best

approach to take.

Last year, the existing online hub www.aqicn.org was identified as a possible host for PMEH

data, but no formal assessment has been conducted as to its suitability. There is also the

possibility that a new PMEH online platform could be created instead, but the practicalities and

costs of this have not been fully explored yet.

The Secretariat have decided to delay making a decision about how to stream the data until more

research has been done into how information is being shared at the moment, and until the AQM

plans in each country are further developed.

There remains uncertainty about whether real-time monitoring of SLCPs is possible through local

hubs, as described in indicator 3.2. IIASA and the DRI have been asked to look into this and are

due to report at the Steering Committee meeting in December 2017.

Summary of responses to issues raised in previous annual reviews

Secretariat to assess whether chosen global monitoring hub is the best suited to dissemination and display of PMEH data and to establish whether the development of own

Indicator(s) Milestones from Logframe

Progress

3.1 – # of monitoring hubs in participating cities streaming data into the existing global monitoring hubs.

13 (2016) 15 (2017)

Missed – No new hubs have been installed by the PMEH programme over the last year because of procurement delays. This year three cities in South Africa introduced hubs, so the baseline has increased to 16. This indicator needs to be revised next year.

3.2 – % of local hubs in participating cities having real-time information on air pollution and SLCPs (x, x).

20%, 0% (2016) 35%, 0% (2017)

Missed –The programme has discovered that the baseline needs to be revised, as more cities have hubs than previously thought. 68% of the cities now have hubs reporting information on air pollution. However, none of these are outputs of the PMEH programme. This indicator needs to be revised next year.

15

PMEH-focused hub would be more suitable as a standalone communication and awareness tool – This recommendation is still relevant. The Secretariat will first make country-specific recommendations of what monitoring and lab equipment is required, and then will consider whether communication of the data is best done through existing reporting tools like AQICN.org, or if the creation of a new PMEH hub is required, or a series of hubs covering different regional, national or local areas.

Confirm baseline data as soon as is practicable – Baselines were updated in the logframe at the beginning of 2017. However, progress is continuing in many countries so this may need to be revised in the future.

BEIS lead and PMEH secretariat to work together to determine how achievable the full

practicalities of sub-indicator 3.2 are to reach – This will need to be revisited. Even if it is

possible, it may be that setting up new reporting systems is not the most valuable use of PMEH

funds in all cities.

Recommendations

The Secretariat must consider whether the original plans around global and local monitoring hubs are still relevant, and if PMEH funding is still required in this area, given the information that has been learned over the last few years about growing hub capacity around the world. Date for completion – June 2018.

If it is decided to go ahead, the Secretariat should assess whether the chosen global monitoring hub (aqicn.org) is best suited to dissemination and display of PMEH data and to establish whether the development of a new PMEH-focused hub would be more suitable as a standalone communication and awareness tool. They should consider the current methods being used in each city to report and share information so that they do not “reinvent the wheel” unnecessarily. Date for completion – June 2018.

The indicators, baselines and targets in this output need to be revised to reflect recent findings

from the DRI, and to better reflect the revised aims of the programme once a plan is decided.

Date for completion – January 2018 for initial revisions, with possible revisions later in the year.

16

Output Title

Improved awareness (among policy makers and other stakeholders) of air pollution and related AQM issues in participating countries.

Output number per LF 4 Output Score A+

Risk rating (Minor, Moderate, Major, Severe)

Moderate Impact weighting (%): 15%

Risk revised since last AR?

N/A Impact weighting % revised since last AR?

N/A

Key Points

The Secretariat has continued to promote the work of PMEH and to raise awareness of pollution this

year.

A number of learning events have been held by the PMEH programme this year around the

world, focused on key stakeholders (see Output 5). The Secretariat’s communications team has

shared over 50 documents from these events through the PMEH website and their DropBox

portal, so that they are available to a wider audience.

Indicator(s) Milestones from Logframe

Progress

4.1 – # of policy-makers whose awareness on AQM issues has been raised through knowledge-sharing and awareness-raising events.

100 (2016) 200 (2017)

Surpassed – At least 510 have attended events so far, including the Business Week event in December 2016, and workshops in China and Nigeria this year. Survey results have been over 90% positive about their relevance and usefulness.

4.2 – # of page views of the PMEH Programme website (website hits), and programmatic videos, press releases, blogs and other static sources. (x, x).

10,000, 1 (2016) 20,000, 20(2017)

Surpassed – There have been 44,713 page views from 1

st October 2015 to 30

th

September 2017.

4.3 – # of media events (newspaper articles, radio programs, TV features, films, magazines, etc.), in the participating countries, linked to the information and outreach from the PMEH programme.

1 (2016) 11 (2017)

Surpassed – At least 36 media events in total (at least 35 articles have been published online about the work in China, including Reuters, Business Week and China Daily, and the Story of Yesterday musical play was also held in China).

4.4 – # of Bank publications and peer-reviewed articles in scientific journals linked to information and data generated by the PMEH programme.

0 (2016) 1 (2017)

Achieved – A report called The Cost of Air Pollution: Strengthening the Case for Action was published in 2016, funded by the wider PMEH programme

4.5 – # of products produced for public communication purposes from AQM work (either self-standing or integrated).

3 (2016) 10 (2017)

Surpassed – At least 19. 15 were produced by the end of 2016. Another annual report is due at the end of this year. A shorter summary document will also be ready before the end of the year. An AQM training programme is due to launch on the World Bank website in December. The Story of Yesterday play was broadcast online and will be made available online.

17

The World Bank has developed a standard approach to evaluating learning events over the last

year, which they have used to survey participants after their workshops and other events. The

World Bank’s benchmark for such international events is that 85% of participants should find the

event useful and relevant. PMEH events have always scored 90% or higher.

Although no peer-reviewed research articles have been published yet that use data produced by

BEIS-funded monitoring equipment, the PMEH programme has produced one research paper

this year on air pollution, as part of the DFID-funded research component. This is Filling the

Gaps: Improving Measurement of Air Quality in Low and Middle Income Countries and is the

result of a technical workshop that was held jointly with the US Environmental Protection Agency

(EPA) in July. Representatives of developing and developed countries attended. This report will

serve as the core document during 3rd Session of the United Nations Environment Assembly

(UNEA-3) in December 2017 to bring low- and middle-income countries together to agree upon

common air quality measurement methodologies.

The programme has also done some innovative communications work in the form of a musical

play performed in Beijing in December 2016. The play, called The Story of Yesterday, is an

adaptation of Ibsen’s An Enemy of the People, and shows a small town dealing with pollution.

The play attracted 600,000 online viewers, and is now being prepared for wider dissemination in

China. Please note that this play was not funded by BEIS.

PMEH has been promoting wider awareness of pollution issues through Twitter and blog posts.

For example, over summer 2017 they supported the Global Alliance on Health and Pollution’s

#WeCanFightPollution campaign.

The Secretariat has been the main contributor to the production of the World Bank’s forthcoming

“Introduction to Air Quality Management online training course, which is scheduled to be made

available in December 2017, with a full launch at the World Bank Sustainable Development Week

in March 2018.

Summary of responses to issues raised in previous annual reviews (where relevant)

Implement active surveying of policy makers at knowledge-sharing and awareness-raising events in order to gauge level and direction of travel of their awareness of AQM and emission sources in order to attribute their figures to indicator 4.1 – The World Bank has developed a standard approach to evaluating learning events, as described above, which includes asking if the event was relevant and useful to them, and how they will apply the information they learnt in their work. However, this does not specifically ask if participants’ awareness of AQM issues has been raised.

Reassess the ambition of 4.2 page views figure – given surpassing this figure in multiple already. Consider modelling out future reach of the website and other sources that will be counted under this sub-indicator (second half of the milestone) – This has not been revised this year, and the recommendation remains relevant.

Reassess the ambition of 4.3 given unambitious milestone setting having already been surpassed – This has not been revised this year, and the recommendation remains relevant.

Reassess the ambition of 4.5 given milestones have already been surpassed – This has not been revised this year, and the recommendation remains relevant.

PMEH secretariat in collaboration with BEIS lead advisor should consider whether sub-indicators 4.3 and 4.5 should be merged together as they both cover similar indications. Or whether they should be better positioned as demonstrating one thing or another. World Bank-led publications/publically led publications and other interest – After discussing this with the Secretariat, we have agreed that the two indicators are different and should be treated separately for now. However, there are issues with indicators 4.4 and 4.5, as they are very similar and the word “products” is not defined.

18

Recommendations

Change indicator 4.1 to reflect the questions that are being asked of participants. For example: “# of policy-makers who have attended PMEH-organised AQM knowledge-sharing and awareness-raising events and report that it was useful.” Date for completion – January 2018.

Reassess the ambition of indicators 4.2, 4.3 and 4.5 in the logframe, as they have already been surpassed. Date for completion – January 2018.

Consider whether to combine indicators 4.4 and 4.5, and to clarify what counts as a “product”. Date for completion – June 2018.

Given delays to date and the need to accelerate delivery, the Secretariat should consider division of effort and whether they should prioritise action on delivering Outputs 1 and 2 in 2018, and do less on Output 4 until the air quality equipment is installed. Milestones could be revised to reflect this phasing of the work. Date for completion – January 2018.

19

Output Title

Improved collaboration between the Bank and participating countries as well as among participating countries on AQM issues.

Output number per LF 5 Output Score A+

Risk rating (Minor, Moderate, Major, Severe)

Minor Impact weighting (%): 15%

Risk revised since last AR?

N/A Impact weighting % revised since last AR?

N/A

Key Points

In 2017, the PMEH programme has carried out nine missions to build relationships between the World

Bank and participant countries and to set up projects. Some of these have included training workshops.

There have also been two “study tours” taking officials from recipient countries and regions to visit

experts elsewhere. Note that these workshops and study tours have not been funded by BEIS, but by

other PMEH donors.

A policy study tour of Germany, the Netherlands and Austria was held for 24 Chinese officials

from 30th May to 8th June, so that they could learn from European policy makers and academics.

The delegates came from the Ministry of Environmental Protection (MEP) and the three

provinces making up the Jingjinji region (Beijing, Tianjin and Hebei), and the Jingjinji Co-

ordination Committee (JCC). They received presentations from the Mayor of London’s office, the

University of the West of England, the German and Dutch ministries, and the International

Institute for Applied Systems Analysis (IIASA) on how to use the GAINS model. Officials from

Jingjinji and the MEP were able to learn from both the historical and current European

experience, and to see practical examples like how a steel mill in Germany is reducing the

Indicator(s) Milestones from Logframe

Progress

5.1 – # of knowledge-sharing and solution development technical workshops on AQM, organized among the participating cities and countries, supported by PMEH.

1 (2016) 2 (2017)

Surpassed – Five AQM workshops have been held this year – in Nigeria, Vietnam and three in China – taking the total to eight.

5.2 – # of exchanges to share AQM plan drafts between participating cities and countries.

7 (2016) 14 (2017)

Missed – No countries have shared their draft AQM plans as they have not been fully developed yet. However, China has been sharing their wider experience with Nigeria this year. The Beijing Business Week in 2016 also brought 18 countries together (some of whom are not participating in PMEH at the moment) and gave them opportunities to share their knowledge.

5.3 – # of documents uploaded to the "AQM technology-sharing" portal.

5 (2016) 10 (2017)

Surpassed – Over 100 documents have been shared on the PMEH website and DropBox system.

20

amount of particulate matter it produces. Officials are now creating an on-the-job training

programme for managers working in similar fields.

The PMEH programme has also facilitated South-South co-operation. For example, workshops

were held in the cities of Guangzhou and Yancheng on 22nd and 25th September to share the

AQM experience of the Jingjinji region, Europe and America with two other major regions in

China. Over 120 officials from the lower Yangtze Delta and the Pearl River Delta participated.

The Secretariat has developed a programme of knowledge sharing between China and Nigeria,

following the strong interest of Nigerian officials during the PMEH Business Week in Beijing in

December 2016. The work will focus on technical assistance on AQM planning, World Bank

lending for AQM in Nigeria, and steering larger-scale Chinese infrastructure lending in the

country which can support AQM targets. A delegation from Jingjinji visited Lagos in July, and a

tour of China for Nigerian officials will take place in November 2017.

The PMEH team has worked actively with the organisers of the World Bank, China Development

Bank and China Ministry of Foreign Affairs on the Investing in Africa Forum (IAF), to promote air

quality considerations in investment decisions, initially in Lagos, Nigeria. These events bring

together representatives of China and African countries, private sector companies, and

development partners to increase investment in Africa. The 3rd IAF was held in Dakar, Senegal in

September and the PMEH programme is now developing a specific PMEH session at the 4th IAF

to be held in Shenzhen, China in September 2018. It is planned that a specific MOU will be

signed on collaboration and investments into AQM planning, building upon the current

collaboration between China and Nigeria on AQM through the PMEH program.

Summary of responses to issues raised in previous annual reviews (where relevant)

BEIS project lead and secretariat to arrive at a better definition for what constitutes an ‘exchange’ of AQM drafts between participating cities and country regions. At Beijing conference it is likely that multiple discussions were held between relevant individuals however this should not be deemed as an in-depth analysis of the successes and failures of AQM plans – the logframe has not been revised significantly this year, and this issue has not been addressed. It remains relevant.

Establish sharing portal and ensure the unification of the AQM products for ease of collaborative exchange between cities/country regions – The Secretariat have been sharing documents from their training and knowledge sharing events on the PMEH website and through DropBox, but a separate and permanent online portal has not been established yet. The Secretariat will consider the best way of doing this after the country plans are further developed in early 2018. The Secretariat are looking at if this could link to the existing PMEH website and the Chinese Ministry of Environmental Protection’s 3iPET International Platform for Environmental Technology, but other options are also being considered.

Recommendations

BEIS project lead and Secretariat to provide a better way to measure knowledge sharing than the current “exchange of AQM drafts” indicator that reflects the PMEH work plan and range of South-South co-operation activities, and build that into the revised logframe. Indicators 5.1 and 5.3 should also be reviewed to make sure they are easily measurable and to make sure that the milestones are stretching. Date for completion – January 2018.

Establish sharing portal and ensure the unification of the AQM products for ease of collaborative exchange between cities/country regions. Date for completion – October 2018.

21

D: FUND PERFORMANCE NOT CAPTURED BY OUTPUTS

Over the last year, the plans for PMEH in each of the seven recipient countries have become clearer.

Here is a summary of progress made in each country, and the plans over the next few years:

China:

The work in China focuses on the expanded “Jingjinji” region, the metropolitan region that covers

Beijing and Tianjin municipalities, the surrounding Hebei province, as well as parts of Inner

Mongolia, Shanxi, Henan and Shandong provinces. The expanded Jingjinji region includes major

cities and also large industrial areas, but because different authorities represent the different

areas, it has been hard for them to co-ordinate policy and work towards solutions together. The

budget for the work is $2m of capital spend for ambient air quality monitoring and source

apportionment equipment, and $1.5m for capacity building.

Technical assessments by DRI have shown that Beijing has a large network of ambient air quality

monitors and advanced source apportionment capacity. Following consultation with the Jingjinji

Coordination Committee (JCC), the PMEH programme proposes to focus on Hebei, a heavily

industrialised province that produces around 25% of the world’s steel and much of the country’s

cement. The DRI found that the province has a good ambient monitoring network and source

apportionment capacity, but have recommended that they receive equipment to perform more

advanced source apportionment analysis, to set up more sampling sites, and to introduce

ambient monitoring of SLCPs. It will be important for the Secretariat to demonstrate the

additionality of this work before funding is provided for the equipment. The technical assistance

part of the programme will cover the expanded Jingjinji region, however, so that officials can

share information between the different parts of the region and build a cost-effective and

sustainable plan for the future.

Setting up this project has been complicated because of the number of different local authorities

involved. Formal agreements have not been signed yet, but Hebei Province’s Environmental

Protection Bureau (EPB) and the EPBs of seven of the 11 most polluted municipalities within

Hebei are keen to take part. In addition, the PMEH program is finalising agreements with the

seven provinces in the expanded Jingjinji region about developing the GAINS model.

A lot of knowledge sharing has been carried out this year, with the technical mission to Germany,

Austria and the Netherlands (which included speakers from the UK) a particular highlight (see

Output 5 above). This included six participants each from the national Ministry of Environmental

Protection (MEP), the Beijing, Tianjin and Hebei EPBs, and also local administrative offices within

the provinces. This was the first time these three provinces and the MEP have carried out a

technical mission abroad in this way.

Egypt:

The Cairo project is one of the three that is receiving a smaller amount of money from PMEH

($500k on equipment and $1m for capacity building) along with the Vietnam and Ghana projects.

However, the German development agency GIZ is providing €3 million of funds that can be used

alongside PMEH for more technical assistance in Egypt and Vietnam.

Egypt has a strict clearance procedure, and the World Bank assistance is still awaiting clearance

from the Ministry of International Cooperation. However, a common understanding on ways

forward was reached with the Ministry of Environment (EEAA) in September 2017. This has

meant that the concept note for the project has not been confirmed yet, but is due to be finalised

during missions to Cairo in November and December.

The Cairo metropolitan area has 44 air quality monitoring stations, mostly focused on PM10, but

the DRI has not yet visited to examine their quality and determine possible gaps. Their research

mission will be held in December 2017, as well as a visit by IIASA to discuss the application of

22

the GAINS model. Discussions with the EEAA so far have shown that they are very interested in

improving their source apportionment capacity.

India:

The process of setting up the PMEH project in India has perhaps been the slowest of the seven

countries. While the national government was initially very keen to work with the programme,

progress slowed in both 2015 and 2016, but has picked up in the last few months. The World

Bank is now working directly with the Delhi Pollution Control Committee (DPCC) and the national

Central Pollution Control Board (CPCB) to get access to collaboration with the surrounding

provinces (Haryana, Uttar Pradesh and Rajasthan) which make up the National Capital Region

(NCR). The budget for the project is $4m of capital funds for equipment, and $1m for capacity

building.

As with China’s Jingjinji region, the NCR has large urban areas surrounded by industry and

farmland, and concentrations of PM10 and PM2.5 regularly exceed National Ambient Air Quality

Standards (NAAQS) in Delhi. Using the GAINS model through collaboration between IIASA and

India’s National Environmental Engineering Research Institute (NEERI), it has been estimated

that up to 60% of PM2.5 pollution in Delhi city comes from surrounding regions. Although the

authorities acknowledge the need for co-operation, it can be difficult for the states to work

together to tackle this problem.

Visits to Delhi by the PMEH team and DRI have shown that Delhi has an extensive and growing

ambient air quality monitoring network. However, conversations with the DPCC have identified

capacity building needs that the PMEH programme can support, particularly around air quality

forecasting and also developing cost-effective abatement options. In terms of equipment needed

for this, the DRI has recommended the upgrading of a number of ambient monitoring stations so

that they can take samples suitable for source apportionment of PM2.5, ammonia, black carbon,

and eventually other SLCPs. No technical assessments have been made yet into the monitoring

capacity in the other regions that make up the NCR, but that is due to take place in upcoming

missions. Lab capacity also needs to be upgraded. The DRI report recommends that a new joint

Indian Institute of Technology Delhi (IITD) and DPCC lab is set up for the region, which will be

able to perform detailed analysis of PM2.5 samples. A technical assistance programme will also

be built around the city and region’s objectives.

PMEH is still in discussions with the CPCB. The technical assistance part of the programme

could support them to improve their air quality monitoring standards, help them understand the

sources of pollution in the NCR, provide international expertise for research on the health impact

of pollution in select cities within the NCR, and potentially extend the PMEH AQM framework to

more cities within the NCR and in other parts of India. The World Bank believes that collaboration

with CPCB will provide access to the overall NCR, so that AQM planning can take place across

the region.

Ghana:

The PMEH project in Ghana is centred on the city of Accra. It has the lowest level of funding of

the seven countries, with $500k budgeted for capital goods, and $250k for capacity building. The

programme is partnering primarily with Ghana Environmental Protection Agency (EPA), but is

involving many other organisations including the US EPA, the Global Alliance for Clean

Cookstoves (GACC), the Climate and Clean Air Coalition (CCAC).

A scoping mission by the DRI has taken place, which found that there are currently 14 ambient

air quality monitors in the city, but PM2.5 is not consistently monitored. The city lacks a

standardised air quality management process. They have recommended that at least two new

ambient monitoring sites are added to the network, which will measure PM2.5, NO2, SO2 and

black carbon. Because of the limited budget, they have not recommended installing source

23

apportionment monitors at this time, but the PMEH programme will aim to support this in the

future, probably using the “centre of excellence” (COE) lab that will be funded by the PMEH

programme.

$1.5m of BEIS funding will be used to set up this COE lab, which will be used by Ghana, Nigeria

and South Africa, with and additional $750k of funding for technical assistance to support it. It has

not yet been decided how the split will be made between labs in the three countries, but the DRI

have recommended that all three countries receive some upgrade of their existing lab capacity

using some of this funding.

Nigeria:

The project in Nigeria is centred on Lagos, the city with the largest population in Africa. The

budget is set at $3.5m of capital funds for equipment, and $1.25m for capacity building. The

Lagos project is one of the most advanced in the PMEH programme in terms of planning.

This year the Secretariat and the country offices have organised three technical missions to

Nigeria, including two visits by DRI, which has made recommendations. They confirmed that

Lagos itself currently has no ambient air quality monitors (except for two simple dust monitors), or

any capacity to do source apportionment. While the lack of ground level air quality data makes it

difficult to estimate the level of impact on health, the PMEH fact-finding mission conducted in

December 2015 suggested an extremely high level of particulate matter concentration across

Lagos city.

PMEH is planning to create a network of monitoring stations located in different land use areas.

The exact number and location of monitors is still to be decided, but recommended to be

between five and eight. They have also identified some existing labs that could be upgraded,

including a lab in Ede, north east of Lagos which is already supported by the World Bank as the

centre for Ebola research in West Africa, a lab in Ife close to Ede that have some experiences in

analyses of PM2.5 samples, and a Lagos State Environmental Protection Agency (LASEPA) lab,

which currently has limited analytical capacity. Attempts have been made to set up a monitoring

network before, but failed due to lack of staff, power, and maintenance. It will therefore be critical

to combine the equipment with operational support to ensure that an electricity supply is

maintained and vandalism is prevented, and long-term arrangements for staffing.

Government support for the programme appears to be very high. It has been agreed that

LASEPA will be the focal point agency for the project, with support from Lagos State Ministry of

Environment (LSMOE), though Lagos State Ministry of Economic Planning and Budget

(LSMEPB) will provide oversight and will chair the project’s steering committee. Lagos State

Ministry of Health (LSMOH) will be involved in the health impact assessment part of the

programme, and Lagos Metropolitan Area Transport Authority (LAMATA) will be involved. The

Federal Ministry of Environment (FME) is the national focal point for pollution management and

will provide the national policies, guidance and enabling legislation for the implementation of the

pollution management plan.

The federal government sees the work in Lagos as a pilot, which could be extended to other

cities and states in the future, particularly the 12 states in Nigeria that have cities with more than

1 million inhabitants. During the mission by the PMEH team in July, an AQM symposium was

held that included participants from six other states who are interested in joining the PMEH

beyond Lagos.

As discussed above under Output 5, Nigeria is keen to learn from the experience of other

countries, particularly China, and the PMEH team is facilitating this South-South co-operation. A

delegation from China’s MEP visited Lagos in July, and a tour of China for Nigerian officials is

now organised for 13th to 18th November. The PMEH team held is co-operating with the China

Development Bank, the China Ministry of Finance and China MEP to promote AQM collaboration

between China and Africa – and particularly Nigeria and Lagos at the Investing in Africa Forum

24

(IAF) IV in Chengsha, China during autumn 2018. Based upon the PMEH China-Nigeria dialogue

on AQM, it is planned that an MOU for this collaboration will be signed prior or during IAF IV.

South Africa:

The project in South Africa will be working in the Greater Johannesburg Area, which includes the

city of Johannesburg and the neighbouring municipalities of Ekurhuleni and Tshwane (the

municipality that contains Pretoria). The area is undergoing economic growth and experiencing

fast rates of industrial expansion (about 40% of South Africa’s total GDP is generated in GJA).

The budget is $1.25m of capital funds for equipment, and $1m for AQM planning.

The DRI confirmed that Johannesburg already has a fairly well developed ambient air quality

monitoring network, but has limited PM2.5 monitoring capacity. The DRI has recommended

enhancing the PM2.5 capacity, and to upgrade the current network by incorporating black and

brown carbon monitors as well. They also discussed the installation of samplers to allow for

source apportionment, so that this can help them identify cost-effective abatement options. The

location of these samples is under discussion with the Government. North-West University has a

lab that has the right instruments and capacity (with minor upgrades) to do the required analytical

work. The DRI will soon provide a more detailed list of what equipment is required.

The main partner in South Africa is the Department for Environmental Affairs (DEA), which has

confirmed its involvement by submitting an official request to the World Bank through their

Treasury. The DEA is keen to share its experience of AQM with other PMEH countries.

When considering where to place the “centre of excellence” lab that will most likely serve Ghana,

Nigeria and South Africa, after a recommendation by the DEA, the DRI reviewed lab capacity in

North West University (NWU), close to Johannesburg, that would be suitable. The exact

collaboration scheme and function of the proposed NWU’s regional centre of excellence involves

political and cultural sensitivities and is still under discussion. The DRI will be writing a separate

report to make recommendations on the centre of excellence from a technical feasibility

perspective, after visiting labs in three countries.

Vietnam:

The project in Vietnam is centred on the Larger Hanoi Metropolitan Area (LHMA), made up of

Hanoi and the neighbouring provinces of Hung Yen and Bac Ninh. These neighbouring provinces

are important as they contain a large number of “craft villages” that potentially contribute large

amounts of industrial emissions to the region. The aim is to support the development of a full-

scale AQM plan for LHMA. The budget for the project is $500k of capital funds for equipment,

and $1m for capacity building.

The main partner for this project is the Pollution Control Department (PCD) within the Ministry of

Natural Resources and Environment (MONRE). The Departments of Natural Resources

Environment (DONREs) of each city are also involved.

While both the PCD and DONREs of the three cities are very enthusiastic about improving

region’s AQM, the Vietnamese Government has quite a complex internal approval system for

development projects, which has slowed down this initial stage of the project.

As with many other PMEH cities, ambient air quality monitoring is quite widespread in LHMA, and

the DRI has found this to be mostly sufficient. However, there is very limited source

apportionment monitoring capacity so they have recommended that the project focuses on

supporting the development of these monitoring stations and of lab capacity. They have

recommended locations for the samplers, and have identified a laboratory that can be upgraded.

Several technical workshops on AQM have been held in 2016 and 2017 involving the PCD, the

Hanoi Environmental Protection Agency (EPA), representatives from neighbouring provinces,

and representatives from academic and research institutions. PMEH has brought in experts from

25

IIASA to work with the Vietnamese agencies to apply the GAINS model and on the development

of an emissions inventory.

Other impacts of the programme this year are:

The PMEH Secretariat represented the World Bank’s Global Practice for Environment and

Natural Resource Development at the first Advisory Expert Working Group for the Global Urban

Air Pollution Observatory (GUAPO) in Paris on 27th September. The GUAPO initiative, founded

by the Mayor of Paris, will promote the sharing of best practice in AQM between cities and to

evaluate the impact of different interventions. PMEH will play a major role by sharing its AQM

methodology with member cities, and may lead the GUAPO work stream on sharing AQM actions

and strategies. The World Bank through the PMEH program is also participating in the official

launch of GUAPO in Paris on 16th November 2017.

The PMEH Secretariat has continued its dialogue with other potential donors in the last few

months, which may be able to contribute to the air pollution component and other components of

the programme. The total programme may need to be extended by another few years to

accommodate this new work, but the BEIS-funded elements would still be completed by 2020.

As the World Bank has agreed to make PMEH a pilot for using Bank-executed funds in

programmes with significant capital goods, it has the potential to have quite a large impact on

similar programmes in the future. When technical equipment is being purchased across multiple

countries, as it is in PMEH, the procurement may become quicker and the quality of equipment

can be assured.

26

E: VALUE FOR MONEY & FINANCIAL PERFORMANCE

Key cost drivers and performance

As the BEIS-funded elements of the programme have been largely paused this year, no further encashment has been made or money spent. The combined management and administrative costs of the programme to BEIS across the life of the programme will remain the same despite this delay: up to 7% of our funding, made up of a standard fee of 2% for the MDTF, with a variable fee of up to 5% for managing and operating costs. This compares is similar to other World Bank projects funded by the ICF that have been set up over the last few years, for example the World Bank Carbon Capture and Storage programme charges 9% in total, and the Transformative Carbon Asset Facility (TCAF) charges 5% for its first $50 million dollars committed. The new standard fee at the Bank for MDTFs is 17%. Donors that joined the programme after us have to pay this higher rate, but as our funding was confirmed before this higher rate came in, we are able to continue using the lower rate. The devaluation of sterling that occurred last year continues to have a negative effect on the amount of funding available for the programme compared to when the administrative arrangement was signed, as the World Bank holds the funds in US dollars. The World Bank has kept the allocation for each of the seven countries the same despite this, so it will not have an impact on the projects in China, India, Egypt, Ghana, Nigeria, South Africa and Vietnam. However, funds have been set aside for expansion beyond these countries, and this budget will be lower than originally planned if the exchange rate does not improve, reducing the scope of this work and its likely impact.

VfM performance compared to the original VfM proposition in the business case As noted above, there has been a shift in emphasis in the programme from ambient air quality monitoring to more source apportionment, which has the potential to improve value for money for the programme overall. This is because PMEH will now be focusing on the more advanced technologies, where international involvement can have more of an impact as the recipient countries have less direct experience. This change may also have wider value for money impacts, as understanding the source of pollution can help countries determine cost-effective investments by knowing the critical sources to address. To assess the value for money of a programme, we use the “three Es framework”. Although the BEIS funded part of the programme has not started we can still provide an assessment of this:

Economy (Are we or our agents buying inputs of the appropriate quality at the right price?): This is a relatively new programme that has not started buying inputs yet. Given this, we have little experience to be able to compare its procurement modality to good practice seen in the past. This year the procurement modality that will be used has changed compared to the original business case, as the World Bank will now be procuring the monitoring equipment rather than the countries and cities themselves, using BE rather than RE funds. This procurement method was not considered at the business case stage because of World Bank procurement policy, but the Secretariat has received special permission to use it. We do believe that this change has the potential to increase the programme’s value for money proposition compared to the original business case, and will deliver the best possible results:

o Using BE funds means that the procurement can go ahead more quickly, as the complex internal approval systems in recipient countries can be avoided. Because the programme is working across multiple local authorities in each country, this would have been a very difficult process.

o The World Bank is currently refining the procurement plan, and the balance between global, regional and country-specific procurement. If global or regional procurement is used, the programme could benefit from economies of scale compared to the original plan (in which each city would purchase their equipment separately). This has the potential to improve value for money for the programme.

o Using BE funds also ensures that high quality equipment will be purchased in each country, as the World Bank will be able to use its own expertise directly, and the

27

recommendations of the DRI. The risk that unsuitable equipment is purchased has decreased. The World Bank will also be able to ensure that equipment is compatible across countries if needed, which could help with knowledge and data sharing internationally.

o As the programme is a pilot for the World Bank in using BE funds in this way, their procurement team will be working particularly closely with the Secretariat and country offices to ensure that value for money is maintained.

o Some of the benefits of using the original procurement modality have been reduced, though. Most concerning is that we cannot guarantee automatically that the recipient cities and countries approve of the specifications of the equipment that is being purchased, and that it meets their needs. To mitigate this risk, the World Bank will work closely with the authorities in each city, and agree the list of equipment before procurement begins. They plan to closely consult host countries when drawing up terms of reference and final contracts for the procurement. Also, the ownership of the equipment will not be as simple under the new procurement arrangements: equipment will be owned by contractors until the end of the programme, and then will transfer to the recipient governments. It will be important to work closely with governments so that they are ready to take this on and are more likely to continue using it. MoUs and similar technical agreements should be agreed with recipient governments to formalise this and ensure the long-term impact of our investment.

Efficiency (How well do we or our agents convert inputs into outputs?): This cannot be assessed fully at this stage. Progress in the programme has been slow so far and below our expectations, due to the unforeseen procurement delays and extra time taken to set up the projects in each country. However, the purchase of equipment is due to begin in early 2018 using the World Bank’s procurement expertise. Work on capacity building and communications has been running to schedule.

Effectiveness (How well are the outputs from an intervention achieving the desired outcomes?): Although it is too early to assess actual results and outputs as the programme is still at an early stage, the programme is disappointingly behind schedule and has faced problems in terms of effective delivery against desired outcomes. The drop in the value of the pound has reduced the impact of the programme somewhat as the programme currently has less funding set aside to expand into new countries and for contingencies. However, overall the programme is still on track to have the desired outcomes in the agreed recipient countries, albeit on a more challenging timeframe. The equipment funded by BEIS will be part of the wider PMEH AQM programme in each of the seven countries, which covers technical assistance specifically around developing long-term AQM plans and is on-going. This will make it more likely that the outputs funded by BEIS will have an impact on policy, as seen in the logframe.

Assessment of whether the programme continues to represent value for money The business case identifies a few specific drivers of value for money. One is that PMEH will pool the resources of different donors and will ensure that all parts of the programme complement each other, reducing duplication of effort, maximising geographical coverage, and producing economies of scale. This continues to be the case. While capital spending using BEIS funds has been paused this year while the World Bank worked on changing the procurement modality, the other parts of the AQM programme (funded by Germany and Norway) have continued, laying the groundwork for the next stage and also building capacity in recipient countries through training. The use of the common PMEH AQM framework across all the countries ensures that the different activities are complementary and build towards a common goal. Economies of scale have been achieved in the contract with the DRI, who are assessing all seven countries. The programme will start to really benefit from this as the procurement process begins next year. The programme will also be able to leverage further resources because of its size. This year, GIZ has agreed to use €3m of its funds in line with PMEH to support projects in Vietnam and Egypt, and there are other donors that are likely to join the programme next year. In terms of delivering results and demonstrating value for money, it is too early to fully assess PMEH, as the BEIS-funded components of the programme are still at the preparation stage. The delays represent a severe risk to delivery, which would negatively impact the programme’s value for money, but the

28

changes to the procurement modality being used have the potential to increase value for money. Overall the programme continues to represent value for money. However, to fully achieve this, progress must be made rapidly next year to operationalise spending and start installing the equipment.

Quality of financial management

The World Bank manages many of our multilateral funds and we are currently happy with their financial management given their reputation and track record on other projects. Their financial management and reporting processes for the multi-donor trust fund have demonstrated themselves as very good The World Bank uses proprietary reporting systems that follow strict requirements, equivalent to BEIS’s own. Given that the programme is entering a period of implementation following a delay, it is imperative to ensure the highest levels of transparency and scrutiny are enforced in all financial management. To help with this, donors have requested that the Secretariat provides the Steering Committee with a work plan for the following year that includes spending plans and a timeline so that progress can be monitored.

Date of last narrative financial report 31 November 2017

Date of last audited annual statement 30 September 2016

29

F: RISK

Overall risk rating: Major

Overview of programme risk

Over the last year, the risk level of the programme as reflected in the logframe has increased because of the delays, and as the complexity of the programme has become clearer. However, the change from RE to BE fund will increase the control of the World Bank over the procurement process, so the risk has gone down in this area. Here is an updated risk register for the programme, now using the DFID risk matrix to determine the ratings based on their likelihood and potential impact:

Risk description Rating last year

Current rating

Update

Risk 1: Potential for recipient country governments to either not acknowledge the extent of local air pollution and emissions or not take the requisite mitigation action in a coordinated and effective manner.

Moderate Major This has increased because of an acknowledgement of the severe impact this risk would have on the success of the programme if it came to pass. This risk will always exist as it is up to the governments and local authorities how much action they will take. When working across so many cities and provinces, co-ordination is challenging. It is too early to tell what action they will take, but the programme has received commitment letters from each government, showing their commitment. To mitigate the risk, the PMEH programme is organising capacity building in each country aimed at improving their capability so that they understand the impacts of pollution and can use the PMEH framework to make effective AQM plans. They are also engaging the countries in each element of the PMEH AQM framework as much as possible to reduce this risk.

Risk 2: Host countries are not committed to the project and so do not continue monitoring air quality and maintaining the equipment. This would severely affect the long-term sustainability of the project.

Moderate Major This has increased because of an acknowledgement of the severe impact this risk would have on the success of the programme. Each country has signed a commitment letter, showing their engagement in the programme. Countries have raised the importance of on-going maintenance themselves to the World Bank. The Secretariat is planning to build the maintenance and running of the equipment into contracts wherever possible. They are also having MOU or technical agreement with the relevant authorities to show their commitment.

Risk 3: Exchange rate fluctuations reduce the value of BEIS's contribution. This has taken place in 2016 and 2017 as the pound weakened against the dollar compared to when the

Moderate Major This has increased this year to acknowledge that this risk has actually taken place over the last few years and will likely have an impact. The Secretariat has decided to keep allocations for each recipient country the same as originally planned, to lower that impact. Instead, the budget held in reserve for potential expansion

30

administrative agreement was signed.

into more countries has been reduced. Going forward, BEIS will work with the Secretariat to monitor rates and adjust encashment schedules to minimise the impact. The Secretariat is also raising funds from other donors.

Risk 4: Management and prioritisation of the programme at the World Bank (both in the Secretariat and the country offices) is not sufficiently strong to deal with the size and complexity of the PMEH programme, and the different elements of the programme are therefore not carried out and co-ordinated together to achieve maximum impact and value for money.

Major Severe The risk rating has increased because the programme has been further delayed this year. It will need a significant ramp up of activity to get everything done before the 2020 deadline, which will require strong programme management skills, as the programme is working across so many countries. The World Bank has taken on three more staff to work on the programme, which is positive. BEIS has made recommendations through this annual review and that of the previous year to support the Secretariat to improve programme management and reporting.

Outstanding actions from risk assessment

The following actions were identified in the last annual review:

The BEIS lead advisor and PMEH secretariat will monitor exchange rates and adjust the indicative schedule of encashment with an aim to minimise any funding discrepancy – no encashment took place this year. The recommendation remains relevant, and will be followed when the revised encashment schedule is drawn up.

The secretariat will consider how to increase the efficiency of procuring of equipment, potentially through central procurement, in order to minimise the impact of currency fluctuations – this recommendation was followed through, but because of potential delays to the programme rather than exchange rate considerations.

The PMEH Secretariat will review its management and leadership capacity to meet the future demands of the programme in terms of programme management and implementation of activities. Donors will continue to monitor this closely and offer advice as needed – the Secretariat has secured three extra posts in the last year (an air quality a management planning expert to work on this part of the programme, an environmental engineer to work on the components of the programme looking at toxic sites and solid waste in marine environments, and a communications expert). The programme is very complex, however, so we have recommended that further support is found so that the programme management elements of the programme can be prioritised.

All risk levels will be kept under review and any other additional actions will be considered to ensure most significant impacts are mitigated – BEIS has monitored the risks to the programme over the last year, particularly the issues over procurement. We have made a number of recommendations in this annual review to mitigate the remaining risk.

BEIS and DFID leads have offered their support at future Steering Committee meetings to leverage interest in the programme by prospective donors and speed up accession process to the programme – No potential new donors have been introduced to BEIS and DFID this year. The offer remains open, as potential donors are being identified by the Secretariat.

31

G: COMMERCIAL CONSIDERATIONS (½ page)

Delivery against planned timeframe The programme is now significantly behind schedule, as shown in the logframe and as explained in section B above. 70% of cities were supposed to have had air quality monitoring equipment installed by the end of 2017, but the purchase of this equipment has not started yet. This is not expected to have an impact on the cost of the programme or expected results, but it does make delivery much more challenging for the Secretariat and the country offices, as it gives less time to complete all the work before the end of 2020. The Secretariat have used the time it has taken to change the procurement modality to build capacity, research what each city requires, and set up the projects in each country, but more work needs to be done.

Performance of partnership (s)

BEIS’s work on the PMEH programme depends on strong partnerships with DFID, other donors and the World Bank. DFID’s partnership with BEIS has been excellent from the start and overall one of the most successful aspects of the programme so far. This partnership was not only fundamental to the drafting and approval of two complex and innovative business cases, covering areas of work that neither DFID nor BEIS have been historically engaged with, it has also played a constructive role in managing the relationships between other donors and the World Bank. This year, DFID and BEIS have also built stronger relationships with the Norwegian and German donors to the programme, working closely to make recommendations while the procurement and governance arrangements in the programme have been uncertain. Our partnership with the World Bank related to this programme has been challenging at times. This was mostly due to a lack of understanding of each other’s key priorities and the need for effective communication between the PMEH Secretariat and the donors. There has also been a lack of clarity regarding the governance of the programme and the role of the Steering Committee, and we are concerned about a lack of staff resource in the Secretariat and country offices to manage this complex programme. We have worked with other donors, particularly DFID, to flag issues and make recommendations. This has resulted in the introduction of a quarterly report for donors. It should also be noted that three new members of staff have started work in the Secretariat. Progress has been made, but we are waiting for questions about the governance and reporting of the programme to be fully resolved. We understand that the World Bank is working on these issues. In the recommendations section of this annual review, we have made recommendations that we believe would improve things, including the introduction of an operational manual for the programme to clarify roles and expectations. Because of the shift from RE to BE funds for the procurement of capital goods, the formal administrative arrangement documents between the World Bank and the donors will need to be updated. This should be a relatively simple process, and the World Bank is drafting these changes now. It may be necessary to also reflect some other changes to the governance arrangements for the programme in this document, once they are agreed.

Asset monitoring and control

Originally the assets purchased with our funding would have been the property of the recipient

organisations, but because of the change in the procurement method used, this will not be the case

initially. The World Bank will be running the procurement of the equipment, but they will not own it – it will

be owned by the contractor or contractors that the World Bank will hire, and then will be transferred to

the host country or city at the end of the contract period. The contracts will specify that monitoring and

maintenance of the assets are covered, and the Secretariat will be working closely with recipient country

32

and city authorities to ensure that the equipment meets their needs and that they are able to maintain it

after this point. The World Bank procurement team will be very involved in this process, particularly

because it is a pilot, so there will be scrutiny of all contracts to make sure that the assets are being

properly managed.

33

H: MONITORING & EVALUATION

Evidence and evaluation

The following evidence was used to inform this annual review:

Logframe produced by the PMEH Secretariat

Discussions with the Secretariat in Washington DC and with country leads in all seven countries,

in person or on the phone

Quarterly reports produced by the Secretariat

Reports produced by the DRI following their mission to recipient countries

Reports produced by the PMEH Secretariat following their missions held by the PMEH

Secretariat

Discussions with Posts at the British High Commissions and Embassies in the recipient countries

An evaluation plan for the programme has not been agreed yet. Originally the evaluation itself was due

to take place in early 2018, but this will need to be pushed back given the delays in the programme. The

Secretariat has suggested that the evaluation should take place at the end of 2018 or beginning of 2019.

The timing and commissioning process of this should be discussed and endorsed by the Steering

Committee as part of a larger evaluation plan. It will be important to agree in advance the commissioning

process of the evaluation and the evaluation questions, and for BEIS and DFID to be able to feed into

this process.

Monitoring progress throughout the review period

This year the PMEH Secretariat has introduced a quarterly report to update donors on their progress,

following requests from donors for more information. During the process of writing this annual review,

BEIS has fed back thoughts on how to make this as clear as possible so that it is useful for monitoring

progress, and improvements have been made. Working with other donors, BEIS has also recommended

that the Secretariat provide an annual plan that includes clear timelines, to make it easier to track the

programme.

The work plan for the programme has changed based on evidence gathered this year, changing

emphasis more towards supporting source apportionment technology. More information about specific

country plans is also now available. We have therefore recommended in this annual review that the

theory of change be reviewed to see if it needs updating, and that the logframe should be updated to

reflect the outputs the programme will be focusing on.

34

I: TRANSFORMATIONAL CHANGE

Rating

1: No evidence yet available - too soon to revise assessment in business case

Evidence and evaluation

The BEIS-funded elements of the PMEH programme were “paused” for most of this year because of a

procurement issue. The Secretariat received formal approval to change the procurement method used to

get around this in the autumn, which is positive, but it has resulted in missing most milestones. Scoping

work has taken place this year that will feed into the plans for each city, which should help make them

more transformational by linking them to the needs of the countries. However, these work plans are still

in development, so it is difficult to judge how successful individual projects are likely to be at this stage.

The programme has scored well on indicators relating to engagement and technical assistance, as

training and communication activities have continued over the last year. In fact, the programme has

exceeded many milestones in this area.

The next year will be crucial for the programme, as it must move to implementation very quickly and

make up for lost time. This is a multi-faceted programme working across many different cities and

municipalities, so it is challenging. While some good work has been done that can be built on, overall at

this stage not enough evidence is available to judge whether the programme will meet all its targets and

produce transformational change.