performance azerbaijan: east–west highway …. chronology of major events 29 4. ... ars was...

EvaluationIndependent

Performance Evaluation

Report

Raising development impact through evaluation

Azerbaijan: East–West Highway Improvement Project

Reference Number: PPE:AZE 2017-10 Project Number: 35457-013 Loan Numbers: 2205/2206 Independent Evaluation: PE-797

Performance Evaluation Report August 2017

Azerbaijan: East–West Highway Improvement

Project

This document is being disclosed to the public in accordance with ADB's Public Communications Policy 2011.

NOTES

(i) In this report, “$” refers to US dollars. (ii) The fiscal year (FY) of the government ends on 31 December. (iii) For an explanation of rating descriptions used in Asian Development Bank evaluation

reports, see Asian Development Bank. 2016. Guidelines for the Evaluation of Public Sector Operations. Manila.

In preparing any evaluation report, or by making any designation of or reference to a particular territory or geographic area in this document, the Independent Evaluation Department (IED) does not intend to make any judgments as to the legal or other status of any territory or area.

The guidelines formally adopted by IED on avoiding conflict of interest in its independent evaluations were observed in the preparation of this report. To the knowledge of the management of IED, there were no conflicts of interest of the persons preparing, reviewing, or approving this report.

Director General M. Taylor-Dormond, Independent Evaluation Department (IED) Director N. Subramaniam, Sector and Project Division, IED Team leader Y. Ono, Senior Evaluation Specialist, IED Team members I. Garganta, Associate Evaluation Analyst, IED

C. Regodon, Evaluation Assistant, IED

Abbreviations ADB – Asian Development Bank AMS – Asset Management System ARS – Azer Road Service CAREC – Central Asia Regional Economic Cooperation DMF – design and monitoring framework EBRD – European Bank for Reconstruction and Development EIRR – economic internal rate of return FIRR – financial internal rate of return IDB – Islamic Development Bank IEE – initial environmental examination IEM – independent evaluation mission IRI – international roughness index MOT – Ministry of Transport PCR – project completion report PIU – project implementation unit PPER – project performance evaluation report SFD – Saudi Fund for Development SPPRED – State Program for Poverty Reduction and Economic Development VOC – vehicle operating cost VoT – value of travel time savings WOP – without project WP – with project

Currency Equivalents Currency Unit –Azerbaijan manat (AZN)

At Appraisal (1 November 2005)

At Completion (31 March 2010)

At Independent

Evaluation (16 May 2016)

AZN1.00 = $1.16 $1.25 $1.51 $1.00 = AZN0.86 AZN0.80 AZN0.66

Contents

Page

Acknowledgments vii Basic Data ix Map xi Executive Summary xiii Chapter 1: Introduction 1 A. Project Description and Objectives 1 B. Evaluation Purpose and Process 1

Chapter 2: Design and Implementation 3 A. Rationale 3 B. Formulation 3 C. Costs and Financing 4 D. Project Schedule and Implementation Arrangements 4 E. Consultants and Procurement 5 F. Safeguards 6 G. Loan Covenants, Monitoring, and Reporting Arrangements 7 Chapter 3: Performance Assessment 8 A. Relevance 8 B. Effectiveness 9 C. Efficiency 11 D. Sustainability 12 Chapter 4: Other Assessments 14 A. Development Impact 14 B. Performance of the Asian Development Bank 15 C. Performance of the Borrower and the Executing Agency 16 Chapter 5: Overall Assessment, Issue, Lessons, and Recommendation 17 A. Overall Assessment 17 B. Issue 17 C. Lessons 18 D. Recommendation 18

Appendixes

1. Project Design and Monitoring Framework 20 2. Project Costs and Financing Plan 27 3. Chronology of Major Events 29 4. Organizational Structure for Project Implementation 30 5. Status of Compliance with Major Loan Covenants 31

6. Economic Reevaluation 35

Acknowledgments

A team of staff and consultants from the Independent Evaluation Department prepared this study. The team is led by Yuji Ono (Senior Evaluation Specialist), and supported by Irene Garganta (Associate Evaluation Analyst), and Charina Regodon (Evaluation Assistant). John Miller and Shahin Zulfugarov were the consultants. Peer reviewer Toshiyuki Yokota (Senior Evaluation Specialist) provided valuable comments to strengthen the report. The report benefited from the overall guidance of Nathan Subramaniam, Director, Sector and Project Division.

The team would like to thank the Azeravtoyol officials and representatives interviewed in Azerbaijan for their time and opinions. The team expresses its appreciation for time and support in the field provided by local offices of Azeravtoyol and local governments, in particular, the officers and staff of Yevlakh Region Government and Gazakh Region Government; Goranboy Region Office of Azeravtoyol; the World Bank; European Bank for Reconstruction and Development; and Azerbaijan International Road Carriers Association. Finally, the team would like to acknowledge the Asian Development Bank staff from Azerbaijan Resident Mission and the Central and West Asia Department for facilitating the field mission and engaging constructively in reviewing and commenting on the report.

Basic Data

East–West Highway Improvement Project (Loans 2205-AZE/2206-AZE [SF])

Project Preparation

Key Project Data ($ million) As per ADB Loan Documents Actual

Total Project cost 93.2 53.4 Total ADB loan amount (utilization) 52.0 40.4 ADB loan amount (Loan 2205) ADB loan amount (cancellation) Loan 2205

49.0 39.4 9.6

ADB loan amount (Loan 2206) 3.0 1.0 ADB loan amount (cancellation) Loan 2206 2.0

Key Dates Expected Actual

Fact finding 29 January–4 February 2005 Board approval 8 December 2005 Loan agreement 1 June 2006 Loan effectiveness 27 October 2006 Loan closing 31 May 2010 15 June 2010 Project completion 3 September 2010

Borrower Republic of Azerbaijan

Executing Agency Ministry of Transport

Mission Data

Type of Mission No. of Missions No. of Person-Days Fact-finding 1 99 Appraisal 1 42 Inception 1 12 Review 6 50 Midterm Review 1 9 Completion Review 1 27 Independent Evaluation 1 28

Executive Summary

The east–west highway corridor forms part of the Central Asia Regional Economic Cooperation (CAREC) corridor. However, owing to lack of resources for maintenance and overloading of trucks, the east–west highway, which had been developed primarily under the former Soviet Union, was in poor condition at appraisal. The poor condition of the road network was a major impediment to economic development in the six regions of west Azerbaijan (Agstafa, Ganja, Goranboy, Gazakh, Goygol, and Yevlakh). In addition, improvement of the east–west highway corridor was expected to improve international cargo transport. The East–West Highway Improvement Project was approved by the Asian Development Bank (ADB) with a lending amount of $49 million (Loan 2205-AZE) and $3 million equivalent (Loan 2206-AZE) in December 2005. The objective of the project was to increase traffic and improve the quality and efficiency of transport on the east–west highway and selected local roads connected to the highway. The project intended to accelerate socioeconomic development in the six regions of west Azerbaijan and enhance regional cooperation. Scope of Evaluation and Methodology

The purpose of this project performance evaluation report (PPER) is to assess the performance of the project and identify lessons for future projects. The project was evaluated 6 years after completion as this timeframe provides adequate time to assess progress in achieving effectiveness, efficiency, sustainability, and impact of the project. The independent evaluation mission (IEM) was fielded in May 2016 to prepare the PPER. The IEM had meetings with stakeholders to obtain data on impact, outcomes, and outputs. However, the IEM was not able to discuss in depth the implementation of the project, as the project management manager and key officials involved in the project have left the executing agency. Hence, the PPER was largely dependent upon the project completion report (PCR) in the analysis of the implementation, including safeguards. Project Implementation Highlights

At appraisal, the project envisaged four outputs: (i) improvement of the east–west highway, (ii) improvement of local roads, (iii) support for policy reform and improved governance in the road subsector through institutional strengthening, and (iv) cross-border facilitation. The total project cost, including civil works, physical and price contingencies, interest, and other charges during construction of the 127 km of a two-lane paved road on sections of the Yevlakh–Ganja and Gazakh–Georgian borders and 10 local roads totaling to about 10 km in the project area, was estimated at $93.2 million equivalent at appraisal. During detailed design, after project approval, the cost estimates for the civil works component were found to be about two times higher than the estimates at appraisal. Accordingly, at the request of the Government of Azerbaijan, ADB approved the change in project scope in November 2007, which excluded the (i) 55.0 kilometer (km) Yevlakh–Ganja section, (ii) 41.15 km Ganja bypass road, (iii) local roads, and (iv) cross-border facilitation components. The Yevlakh–Ganja section was financed by the Islamic Development Bank (IDB), the Saudi Fund for Development (SFD), and the Government of Azerbaijan; and was completed for two lanes in September 2010 and for four lanes in December 2011. The Ganja bypass road and the local roads were financed by ADB under the multitranche financing facility (MFF): Road Network Development

xiv Azerbaijan: East–West Highway Improvement Project

Program (Loan 2354 and Loan 2433-AZE). The cross-border facilitation component was financed by the government. Consequently, the revised project scope included the 39 km road on Gazakh–Georgian border section and the institutional strengthening for the road subsector component. At completion, the total cost of the project was $53.4 million, about $39.8 million lower than the $93.2 million estimated at appraisal. However, the civil works cost for the Gazakh–Georgian border section was $45.0 million (268% increase), about $28.2 million higher, compared to $16.8 million estimated at appraisal. The project was originally envisaged to be implemented over 4 years from 2006 to 2009. Detailed design started in December 2006 and was completed in November 2007. Procurement for the civil works packages was delayed by repeated tendering. The civil works for the Gazakh–Georgian border section started in February 2008; and were completed in March 2010. The institutional strengthening included subcomponents of improving (i) road maintenance management, (ii) regulation on axle load control and enforcement of avoiding overloading, (iii) accident monitoring and an awareness campaign for road safety, and (iv) capacity development for project management and implementation. The overseas study tour, training, and seminars under the project assisted and enhanced the capacity building for road project management and implementation in the country. At appraisal, the Road Transport Service Department of the Ministry of Transport (MOT) was the executing agency (EA). In 2007, the department was transformed into Azer Road Service (ARS), a semiautonomous open joint-stock company reporting to the MOT. The ARS was reorganized to manage the implementation of road projects in Azerbaijan. It also acted as the implementing agency for the project. The MOT was dissolved in December 2015 and was not functioning at the time the IEM was fielded. The ARS, which was under the MOT, became independent in December 2015 and was transformed to Azeravtoyol in March 2016. Performance Assessment

Relevance. The project was aligned with the government’s State Program for Poverty Reduction and Economic Development (SPPRED) (2003−2005) and was a priority in the Government’s transport development program (2005−2014). The project was included in the country assistance program of ADB for 2005, and was consistent with ADB country strategy. The project also forms part of the CAREC Transport and Trade Facilitation Strategy 2020, supported by ADB. The scope change resulted in substantial reduction of the project scope and the originally envisaged output was not achieved (however, the revised envisaged output under the change in scope was achieved). The scope change was mainly due to sharp increases in the cost of fuel, utilities, and road construction materials. Azerbaijan’s gross domestic product (GDP) growth recorded average annual growth of 29.8% in 2006 and 2007, and inflation rates soared up to 16.8% and 29.8% in 2007 and 2008, respectively. The price escalation was remarkably high when the detailed cost was estimated in December 2006 and was, to a large extent, unforeseeable. The scope change was thus not counted as a major negative point. However, at the time of the scope change, the design and monitoring framework (DMF) should have been modified. Although the Yevlakh–Ganja section was no longer within the scope, due diligence for the section was continued ambiguously. The project scope was confusing because the DMF was not updated. Another weakness was the lack of due diligence on the utilization of the $11.6 million (22 % of the approved amount) undisbursed amount. Of the seven impact indicators, only one indicator was available, and of the five outcome indicators (excluding one that was dropped as a result of the scope change), only three indicators were available or relevant. There was weakness in the selection of indicators

Executive Summary xv

in the DMF. As a positive feature, considering that the project was the first road project financed by ADB in Azerbaijan, substantial experience was gained and lessons were learned to the benefit of ongoing and future design and implementation of road projects. The project is assessed as relevant. Effectiveness. The traffic forecast at appraisal on the Gazakh–Georgian border section was 3,800 vehicles per day in 2011 (first year of full operation); and actual traffic was lower at 3,565 (6% lower than the target). The targeted traffic was not achieved. The target outcome on international roughness index (IRI) was achieved as the actual IRI was much better than the target. The outcome indicator on reduction of vehicle operating cost (VOC) was assessed under efficiency, to avoid double counting. The outcome indicator on road accidents was considered to be achieved as the number of road accidents was reduced by 50% in the Gazakh–Georgian border section. Of the six outcome indicators, two were achieved, one was dropped by the scope change, one was irrelevant, and two were not achieved (but one was assessed under efficiency, to avoid double counting). Although the outcome target for traffic was not achieved, traffic was only 6% lower than the target, and other two outcome targets were achieved. All the envisaged outputs have been achieved except components that were dropped by the scope change. The project is assessed as effective. Efficiency. The reevaluated economic internal rate of return (EIRR) for the project was calculated for the Gazakh–Georgian border section. The recalculated EIRR was 5.9%, compared with 12.2% estimated at appraisal and 21.5% estimated in the PCR. The lower traffic volumes and lower speed increase in the Gazakh–Georgian border section were major reasons for the lower recalculated EIRR. Traffic forecast is linked to GDP growth, and recent economic downturn after the project completion and expected lower GDP growth affected the recent actual and future estimated traffic volumes. In particular, the average GDP growth of the Gazakh region (project area) was lower than the national average during the project implementation and after the project completion. In 2016, the traffic volume was 5% lower than the appraisal estimate and 26% lower than the PCR estimate. The PPER counted accident reduction benefits which are not always counted for road projects and estimated maintenance cost optimistically, and the recalculated EIRR still did not exceed 12%, which is the threshold of economic viability. Another factor that made it difficult to make comparisons among the economic analyses in the appraisal, the PCR, and this PPER is the lack of detailed data provided in the appraisal and the PCR. In terms of process efficiency, the project was originally envisaged to be implemented over 4 years from 2006 to 2009, but the civil works for the Gazakh–Georgian border section were completed only in March 2010, nearly one year later than scheduled. The delay was considered to be modest. The project is rated less than efficient. Sustainability. Since the project is non-revenue generating, the financial internal rate of return (FIRR) was not calculated. Maintenance funds are allocated based on actual needs. The maintenance of national roads is financed through a road fund administered by the Ministry of Finance. If traffic increases, government revenues will increase through user charges such as road taxes and be a financial source of increasing maintenance needs. Overloading of trucks was a serious problem at appraisal, but prohibition of overloading has been stringently enforced in recent years, with a shift of enforcement authority to the traffic police and the installation of weigh stations. The road asset management system (AMS) was intended to be introduced but it was not successful. Instead, performance-based contracts, which the maintenance payment would be based on output rather than input, are planned to be introduced to increase the efficiency of the maintenance units of Azeravtoyol. Given the situation that the road surface conditions have not deteriorated for 5 years since completion of the project, there is a certain level

xvi Azerbaijan: East–West Highway Improvement Project

of institutional capability to maintain roads sustainably. The project is assessed as likely sustainable.

Impact. The project has likely contributed to socioeconomic development in the Gazakh region. However, average GDP growth of the Gazakh region was 10.4% during project implementation (2005–2010) and 5.2% after project completion (2010−2014), which were lower than national averages of 17.2% during project implementation and 6.5% after project completion. Average GDP growth of the six regions which were originally intended project areas was 19.2% during project implementation (2005–2010) and 13.1% after project completion (2010−2014). GDP growth of the six regions was higher than the national averages. If the project impact area was extended to the six regions because the project is corridor development, the project might have made significant contribution to socioeconomic development. Poverty in the project area was reduced through (i) improved access of agricultural and industrial products to the markets by better conditions of the roads, (ii) improved access to employment, (iii) improved access to social services and education, and (iv) lower transport costs. Business enterprises for textiles, cement factories, furniture shops, petrol stations, and service areas were established along the project highway after the project completion. Increased international traffic brought a significant benefit to the economy in Azerbaijan. As the project is likely to have substantial positive socioeconomic development impacts, the project impact is assessed as significant.

Issue

The EIRR largely depends on the economic situation at the time of recalculation. The IEM found that the recalculated EIRR of less than 12% was mainly due to recent economic downturn and lower expected future GDP growth in Azerbaijan. In contrast, the PCR recalculated the EIRR when the economy was booming in Azerbaijan and thus resulted in the higher EIRR. Economic analysis of infrastructure projects is strongly affected by economic forecasts. If the Azerbaijan economy becomes better than forecasted, the efficiency rating of the project could be improved.

Lessons

A large amount of unused loan should be avoided. Although the actual civil works cost

for the project section increased by about 170%, the project still ended with unused loan proceeds of $9.6 million (Loan 2205) and SDR1.4 million (Loan 2206). These were sufficient to finance the local roads, which were estimated to cost $3.8 million. A thorough fund disbursement analysis should have been conducted during project implementation. The PCR recognized the need for greater due diligence during project design and formulation, which included carrying out adequate cost estimations and setting aside enough resources to cover contingencies. The IEM also highlighted this due diligence as a lesson for future road projects.

Monitoring indicators should be carefully selected. Some indicators for impact and

outcome are not readily available nor closely linked to the project. As data are generally not readily available in Azerbaijan, sources for evaluation indicators in projects should be identified in more detail at appraisal. Documents such as those prepared at appraisal and project completion should include sufficient data for subsequent verification and follow-up. Specific omissions were traffic data, unit benefits, and unit costs that were necessary to fully comprehend economic analysis.

ADB-financed project sections should be carefully selected. At appraisal, two sections were intended to be financed by ADB. The Yevlakh-Ganja section involves much higher

Executive Summary xvii

traffic than the Gazakh–Georgian border section. Thus, the Yevlakh-Ganja section brought higher benefits and more significant socioeconomic impact to the project area. If ADB had financed the Yevlakh-Ganja section instead of the Gazakh–Georgian border section, the project might have been rated efficient as the recalculated EIRR would have been higher than the threshold of economic viability. Thus, the ADB-financed section of the project should have been more carefully selected. Recommendation

Institutional reform for road maintenance should be continued. The AMS was intended

to be introduced into the EA under the project in cooperation with the European Bank of Reconstruction and Development and the World Bank. Although the AMS intended to provide capacity development through training programs, it was not well accepted by the EA. Instead, many development donors turned to introducing performance-based contracts to increase the efficiency of maintenance. The AMS was too challenging for the EA. ADB needs to keep supporting the reform of the performance-based contracts, taking into account the lessons learned from the introduction of the AMS.

Overall Assessment

Overall, the project is rated successful. The project was implemented generally successfully despite a scope change due to cost overruns, and it contributed to socioeconomic development in the six regions of the east–west highway corridor in Azerbaijan and increased volume of international cargo transport. However, (i) the major objective of increasing traffic using invested infrastructure was not achieved and (ii) the recalculated EIRR was 5.9%, which was lower than the threshold of economic viability. A recent economic downturn in Azerbaijan unfortunately contributed to lower than

estimated traffic. In addition, average GDP growth of the Gazakh region (project area)

was lower than the national average.

CHAPTER 1

Introduction

Project Description and Objectives

The east–west highway corridor is a key road corridor linking Azerbaijan’s capital and the western part of the country. However, the poor condition of the highway corridor impedes major economic development in Azerbaijan due to poor transport services. The highway corridor also forms part of the international corridor linking Asia and Europe; therefore, improvement of the highway corridor was expected to improve the volume of international cargo transport. The East–West Highway Improvement Project financed by the Asian Development Bank (ADB) is part of the east–west highway corridor. Other highways in the corridor were financed by other donors.

The envisaged project impact was accelerated socioeconomic development in the six regions of west Azerbaijan (Agstafa, Ganja, Goranboy, Gazakh, Goygol, and Yevlakh) with total population of about 702,400; and enhanced regional cooperation. The outcome was increased traffic and improved quality and efficiency of transport on the east–west highway and selected local roads.

At appraisal, the project had four components: (i) improvement of 127 kilometers (km) of a two-lane paved road on sections of the Yevlakh–Ganja and Gazakh–Georgian borders of Azerbaijan’s primary east–west highway, (ii) improvement of 10 local roads in the project area totaling about 65 km, (iii) support for policy reform and improved governance in the road subsector through institutional strengthening, and (iv) facilitation of movement across Azerbaijan’s border with Georgia (at Red Bridge). At completion, only the 39.0 km Gazakh–Georgian border section was rebuilt under the project. The existing two-lane road of the section was upgraded to international standards, with a design speed of 120 km per hour (km/h) and the horizontal alignment generally following the existing road except for slight modifications in certain sections of curves with small radii. The institutional strengthening included four subcomponents: (i) road maintenance management, (ii) regulation on axle load control and enforcement of avoiding overloading, (iii) accident monitoring and an awareness campaign for road safety, and (iv) capacity development for project management and implementation. The DMF of the project is in Appendix 1.

Evaluation Purpose and Process

The purpose of this project performance evaluation report (PPER) is to assess the performance of the project and identify lessons for future projects. The project was evaluated six years after completion, which provides adequate time to assess progress in achieving the effectiveness, efficiency, sustainability, and impact of the project. The findings and lessons identified in the PPER will be useful for designing similar highway projects in Azerbaijan and other Central Asian countries.

2 Azerbaijan: East–West Highway Improvement Project

The independent evaluation mission (IEM) was fielded in May 2016 to prepare this PPER. The IEM had meetings with Azeravtoyol (former Road Service Agency), local governments, the World Bank, European Bank for Reconstruction and Development (EBRD), and stakeholder associations. The IEM was able to obtain data on performance indicators for the outcome, outputs, and activities. However, the IEM was not able to discuss in depth the implementation of the project, as the project management manager and key officials involved in the project have left the executing agency (EA). Hence, the PPER was largely dependent upon the project completion report (PCR) for its analysis of the implementation, including safeguards.

CHAPTER 2

Design and Implementation

Rationale

Road transport is one of the most important transport modes in Azerbaijan. In 2003, the share of road transport were 78% for passengers and 28% for freight in 2003. However, due to lack of resources for maintenance and overloading of trucks, about 75% of the road network, which had been developed primarily under the former Soviet Union was in poor condition at appraisal. The poor condition of the road network impedes major economic growth and private sector development by contributing to high transport costs and greater number of accidents in Azerbaijan.

Azerbaijan’s road network includes two major highways: (i) the east–west highway linking Baku, the county’s capital, to the Georgian border; and (ii) the north–south highway running from the Russian Federation border to the Iranian border, via Baku. The east–west highway forms part of the Central Asia Regional Economic Cooperation (CAREC) corridor and the “silk road” linking Asia to Europe. Although international traffic on the east–west highway was small at appraisal, the corridor was considered to have the potential to become an important transit route. Given the importance of the road corridor, it was estimated that 61% of the east–west and north–south highways required rehabilitation.

In recognition of the importance of roads in poverty reduction and economic growth, the government developed a transport development program (2005−2014), which includes (i) improving the east–west and north–south highways, (ii) strengthening the institutional capacity of the transport sector, (iii) improving secondary and rural roads, and (iv) improving road maintenance. As the east–west highway is one of the key corridors in Azerbaijan, the Government of Azerbaijan1 accords high priority to improving it.

Formulation

ADB provided project preparatory technical assistance for a feasibility study on the Yevlakh-Ganja road rehabilitation project that would be suitable for ADB financing.2 It covered (i) technical, economic, social, and environmental assessments of the project; and (ii) an analysis of Azerbaijan’s road sector to identify major issues to be addressed under the project. However, since the project scope was changed, the feasibility study was not used by ADB. ADB used a feasibility study on the Gazakh–Georgian border section prepared by the government in the same way assisted by an international consulting firm.

1 In this paper, the government means the Government of Azerbaijan unless otherwise stated. 2 ADB. 2004. Technical Assistance to Azerbaijan for Preparing the Yevlax-Ganja Road Rehabilitation Project.

Manila.


Costs and Financing

The total project cost, including physical and price contingencies, interest, and other charges during construction, was estimated at $93.2 million equivalent. During detailed design, the cost estimates for the civil works component were found to be much higher than the estimates at appraisal. Accordingly, at the request of the government, ADB approved the change in project scope on 20 November 2007, which excluded (i) the 55.0 km Yevlakh–Ganja section, (ii) the 41.15 km Ganja bypass road, (iii) local roads, and (iv) cross-border facilitation components. The Yevlakh–Ganja section was financed by the Islamic Development Bank (IDB), the Saudi Fund for Development (SFD), and the government. The Ganja bypass road and the local roads were financed by ADB under the Multitrache Financing Facility (MFF): Road Network Development Program.3 The cross-border facilitation facilities were financed by the government. A 445-meter road approaching the border was added to the Gazakh–Georgian border section using the loan savings. At completion, due to the change of the project scope, the total cost of the project was $53.4 million (excluding sections and components that were dropped from the project scope), about $39.8 million lower than the $93.2 million estimated at appraisal. However, the civil works cost for the Gazakh–Georgian border section was $45.0 million, about $28.2 million (168%) higher than the $16.8 million estimated at appraisal. There were also minor changes in the costs of consulting services, project management, and financial charges.

Under the financing plan envisaged at appraisal, the project was to be cofinanced by ADB (two ADB loans totaling $52.0 million—$49 million from ordinary capital resources (OCR) and $3.0 million equivalent from Special Funds resources), the IDB (a loan of $10.4 million), the SFD (a loan of $11.0 million), and the government of Azerbaijan (funding of $19.8 million). At completion, the project was financed by ADB (two loans totaling $40.4 million —$39.4 million from OCR and $1.0 million equivalent from Special Funds resources) and the government of Azerbaijan (funding of $13.0 million). The ADB OCR loan financed the civil works for the Gazakh–Georgian border section, detailed design for all road sections at appraisal, construction supervision for the Gazakh–Georgian border section, project management, and the financial charges for the ADB loans. The ADB loan from Special Funds resources financed the institutional strengthening component.

The ADB loan proceeds were 77.7% disbursed from January 2007 to June 2010, with $34.7 million used for civil works; $4.7 million for consulting services for detailed design, construction supervision, and the institutional-strengthening component; and $1 million for interest and commitment charges during construction. At the later stage of project implementation, there was a large amount of ADB loan savings, but it was too late to reallocate and use it before the loan closing. The loan closing date was 31 May 2010, but the cancellations of $9,652,199.59 under Loan 2205-AZE and SDR1,398,861.72 under Loan 2206-AZE from the loan account became effective on 15 June 2010.

Project Schedule and Implementation Arrangements

The project was originally envisaged to be implemented over 4 years from 2006 to 2009. The ADB loans took effect in October 2006. Detailed design started in

3 The Ganja bypass road was financed under Tranche 2 (Loan 2433-AZE) of the MFF: Road Network

Development Program (Loan 2354-AZE) and the local roads were financed under Tranche 1 (Loan 2355-AZE) of the MFF.

Design and Implementation 5

December 2006 and was completed in November 2007. All the initial financial bids for the civil works exceeded the estimates of the engineer and the tender process had to be repeated. Procurement for the civil works packages was delayed by repeated tendering. The contract for the Gazakh–Georgian border section was finally signed on 24 December 2007 (about 9 months later than scheduled). The civil works for that section started in February 2008. Both the civil works and construction supervision for the section were completed in March 2010.

At appraisal, the Road Transport Service Department of the Ministry of Transport (MOT) was the executing agency (EA). In 2007, the department was transformed into the Azer Road Service (ARS), a semiautonomous open joint-stock company reporting to MOT. The ARS was reorganized to manage the implementation of all road projects and also acted as the implementing agency for the project. A project implementation unit (PIU) headed by a project director, who served throughout the project and was assisted by a team of qualified technical, financial, and support staff, was established under ARS. The PIU managed all the project activities, including planning, financial management, consultant selection, civil works procurement, progress monitoring, and report preparation.

The MOT was dissolved in December 2015 and was not functioning at the time that the IEM was fielded. The ARS, which was under the MOT, became independent in December 2015 and was transformed into Azeravtoyol in March 2016. At that time, Azeravtoyol assumed the responsibility for road sector policy planning, implementation of road projects, and operation and maintenance of roads. In concurrence with the government organizational change, jurisdiction over axle load control enforcement was transferred from the MOT to the Traffic Police of the Ministry of Internal Affairs.

Consultants and Procurement

Consultants were recruited for (i) the detailed design of roads, (ii) supervision of construction, and (iii) institutional strengthening. An international consulting firm was recruited for the detailed design of all road components and construction supervision of the Gazakh–Georgian border section. A contract for a total of 797 person-months was signed with the selected consultant on 7 December 2006. This contract was later amended to cover an additional 52 person-months of services. In 2008, consultants were engaged for the institutional strengthening component for a total of 92 person-months.

According to the PCR, the overall performance of the consultants was satisfactory. The consultants for the detailed design and construction supervision carried out all activities required in the terms of reference. During implementation, the consultants performed their duties, working in close cooperation with the PIU. The consultants were empowered to make certain decisions on technical and contractual issues on behalf of the ARS. They helped the PIU prepare monthly and quarterly progress reports, implemented monitoring and evaluation programs, and held on-the-job training and seminars for the contractors and PIU staff. The consultants for the institutional-strengthening component worked closely with the MOT, the ARS, and other stakeholders to obtain Azerbaijani ownership of the project and its sustainable implementation. They prepared numerous general and task-specific reports, manuals, guidelines, and memos that were circulated for comments in draft forms, finalized on the basis of the comments received, and distributed to relevant recipients, after introductory training, where necessary. These reports and other deliverables have proven to be useful references for stakeholders related to the road subsector. A study tour to Finland for four ARS staff, arranged by the institutional-strengthening consultants to familiarize the staff with on-


road safety and axle load-control issues, was proven to be advantageous in subsequent works.

According to the PCR, the overall performance of the civil works contractors was satisfactory. The main contractor for the Gazakh–Georgian border section deployed a team of 10 expatriate staff, mostly from Italy. In the early stages of implementation, mobilization was slow and the contractors experienced difficulties finding suitable local subcontractors. The team leader was replaced several times, adversely affecting the work to a certain degree. However, the contractor completed the civil works as required within the contracted schedule. The quality of the civil works was well recognized by the supervision consultants and ARS.

Safeguards

The project required minor land acquisition and temporary land occupation. The MOT prepared a resettlement plan to ensure that fair compensation and assistance would be provided to affected people or businesses. According to the surveys and interviews, there were no complaints concerning land acquisition and resettlement. The resettlement plan was implemented satisfactorily and no outstanding land acquisition and resettlement issues were reported to ADB (PCR, para 44.).

The project implementation had no impact on indigenous peoples. Ethnic Azeri account for 98.6% of the population in the project area; the few other nationalities include Kurds, Russians, and Turks. The minorities are neither indigenous nor negatively affected by the project.

At appraisal, an initial environmental examination (IEE) of the project was undertaken in accordance with ADB’s Environmental Assessment Guidelines (2003). The IEE report concluded that the project would have no major adverse impact on the environment, as the road improvements would be primarily carried out on the existing alignments. No protected or environmentally sensitive areas, such as designated wetlands, forests, nature conservation areas, or places of archaeological importance, were within or near the project area. The project was classified as category B. However, the IEE report identified mitigation measures that civil works contractors were requested to take while being monitored by the supervision consultant, the implementing agency, and government environmental agencies.

An environmental protection report prepared by the consultants in September 2007, during detailed design, became the basis for environmental monitoring works during project implementation. 4 Although no environmental impact study or full environmental monitoring program was required, substantial considerations for environmental protection were incorporated in the detailed design, including using the original loan allocation for environmental management ($100,000). During implementation, the ARS and the contractors took all precautions to prevent any adverse environmental impact in design and construction. For example, adequate drainage measures (bridges, culverts, stabilizing structures, and drains) were incorporated in the project design and construction. Regarding temporary impact (e.g., vibration, noise, dust), the contractor was required to undertake appropriate measures, such as the use of new rollers that cause minimal vibration and noise, and the frequent spraying of water on the road sections under construction. The PIU, with the assistance of the consultants, prepared four biannual environmental reports, which recorded all the

4 Kocks Consult GmbH. 2007. Environmental Protection Report: Gazakh–Georgian Border Section. Koblenz.

Design and Implementation 7

environmental issues and indicated no significant negative environmental impact that occurred during implementation.5

An HIV/AIDS awareness program was carried out by the project. During the project, seven seminars on HIV/AIDS were held for the project management staff, contractors, and subcontractors. Manuals and booklets were distributed to workers and residents in the project area to intensify HIV/AIDS prophylaxis, promote a healthy lifestyle, and increase awareness of HIV/AIDS. During implementation and after project completion, the government undertook measures in accordance with applicable national laws and regulations to prevent the trafficking of illegal substances and humans at the border with Georgia.

Loan Covenants, Monitoring, and Reporting Arrangements

The project complied with most of the loan covenants. The government established an adequate organizational structure for project implementation, including designating the executing and implementing agencies, and setting up the PIU with sufficient staffing. The project implementation activities were well managed; they covered, among others, carrying out procurement, using the services of consultants, monitoring progress, controlling construction quality, preparing monthly and quarterly reports, and implementing financial audits. The quality of the completed project roads met the standards specified in the project documentation. The government provided the required counterpart funds totaling $13.0 million equivalent. The contractor and the subcontractors followed the labor laws and ADB’s requirement in hiring laborers, and encouraged women to participate in the project. The government likewise implemented the road safety program by developing road accident monitoring, reporting, and information systems, thereby enhancing MOT’s capacity to identify accident “black spots” on the road network and by providing staff training.

Although two weigh bridges of high quality were installed and another four are being installed in subsequent ADB loan projects, six weigh stations were not completed. The loan covenant for the monitoring and evaluation (Loan Agreement Schedule 6, para. 18) was partially complied with. It required data collection at inception, at project completion, and three years after project completion. However, the monitoring report after the project completion was not submitted. Thus, information on the socioeconomic impacts and benefits of the project was not obtained from the report. Details of the status of compliance with major loan covenants for the project are in Appendix 5.

5 Kocks Consult GmbH. 2010. Environmental Assessment Report: Gazakh–Georgian Border Section. Koblenz.

CHAPTER 3

Performance Assessment

Relevance

Government policy. The project is a priority in the government’s transport development program (2005−2014). The project aimed at facilitating socioeconomic development aligned with the government’s State Program for Poverty Reduction and Economic Development (2003–2005),6 by (i) improving the east–west highway and local roads linking the western part of Azerbaijan to Baku, the country’s capital and (ii) increasing local communities’ access to markets, job opportunities, and social services. The project also aimed to strengthen Azerbaijan’s transport links to Georgia and thus promote regional cooperation. As part of the Asian highway network and one of Azerbaijan’s main routes for external trade, the east–west highway carries traffic between the Caspian and Black seas and has the potential to become an important route for transit transport between Asia and Europe.

ADB strategy. ADB’s assistance strategy for Azerbaijan aims to reduce poverty and raise living standards in a sustainable way.7 Given Azerbaijan’s location and reliance on transport links with neighboring countries, the latest country strategy and program update identifies transport as a strategically important sector for ADB assistance.8 In line with the government’s priority, ADB’s assistance to roads focuses on improving main highways to enhance regional transport linkages, supporting policy reform, and strengthening institutional capacity. The project, which was ADB’s first road project in Azerbaijan and was included in its country assistance program for 2005, was consistent with its country strategy. The project also forms part of the CAREC Transport and Trade Facilitation Strategy 2020 supported by ADB. ADB set up the Sustainable Transport Initiative Operational Plan for 2008–2020, focusing on developing transport systems that are accessible, affordable, environment-friendly, and safe.9 The project has a road safety component and is relevant to ADB’s initiative on sustainable transport.

Scope change. During detailed design, the cost estimates for the civil works component were about double than the estimates at appraisal mainly due to sharp price increases for fuel, utilities, and major road construction materials. Gross domestic product recorded average annual growth of 29.8% in 2007, and inflation rates soared to 16.8% and 29.8% in 2007 and 2008, respectively. The cost escalation was remarkably high when the detailed cost was estimated in December 2006 and was to a large extent unforeseeable. This caused a substantial reduction in the project’s scope which resulted in most components being dropped and reallocated to be financed by other loans and government sources. These components include (i) the Yevlakh-Ganja section, (ii) the Ganja bypass, (iii) local roads, and (iv) cross-border facilitation. The cost overrun due to

6 The project was continued to be aligned with the State Program on Poverty Reduction and Sustainable

Development (SPPRSD) for 2008-2015, which has similar strategic goals. 7 ADB. 2000. Economic Report and Interim Operational Strategy for Azerbaijan. Manila. 8 ADB. 2004. Country Strategy and Program Update (2005–2006): Azerbaijan. Manila. 9 ADB. 2010. Sustainable Transport Initiative Operational Plan. Manila

Performance Assessment 9

the price escalation was to a large extent, unforeseeable, and the scope change was not counted as a major negative point. However, with the scope changed, the DMF should have been modified to make the changes clear. Despite the fact that the Yevlakh–Ganja section was no longer within the project scope, the PCR still followed up on the achievements in the section. When the IEM was fielded to Azerbaijan, many stakeholders still thought that ADB was involved in the Yevlakh-Ganja section and was expected to play a certain role in resolving the technical design weaknesses in the section.10 After the Yevlakh-Ganja section was dropped from the project scope, ADB was no longer responsible for the section, and recognition of ADB’s changed role should have been shared by stakeholders.

Due diligence for portfolio management. Reasons for the scope change may have been unforeseeable elements caused by unusual inflation of materials costs. However, if the large amount of loan savings during the project implementation had been used, the local roads component could have been implemented under the project. It was too late to reallocate and use the savings before loan closing. The IEM recognized the need for greater due diligence during project design and implementation.

Design monitoring framework. Of the seven impact indicators, only one indicator was available. Of the five relevant outcome indicators, only three indicators were available or relevant. Since indicators are important means of evaluating the performance of projects, their selection should have been given more attention with identification of the data sources.

Rating. The PCR rated the project as highly relevant as the project was highly relevant to the government's objectives and policies, as well as to ADB's country strategy. The IEM notes that the project was fully aligned to national policy and ADB strategies. However, because (i) due diligence on the Yevlakh-Ganja section after the scope change was not needed but still continued ambiguously, (ii) due diligence for portfolio management could have been better, and (iii) there was a weakness in selecting DMF indicators, the project is assessed as relevant. The project completion validation report (PVR) rated the project relevant. The PCR rated the project highly relevant, but the PPER downgraded it for the reasons given above.

Effectiveness

Outcome indicators. The outcome had the following five indicators: (i) increase of traffic, (ii) improvement of international roughness index (IRI), (iii) reduction of vehicle operating costs (VOCs), (iv) reduction of bus fares and freight charges, and (v) reduction of road accidents. Consideration of bus fares and freight charges was dropped from the analysis, because they are heavily regulated by the government and the relation between the project and this indicator is weak in nature. VOCs are a key indicator for efficiency, and should be assessed in the context of efficiency evaluation to avoid double counting.

Traffic. The target year of annual average daily traffic (AADT) was 2010, but the Gazakh–Georgian border section was completed in 2010. Thus, this evaluation changed the target year to 2011 (the first full year of full operation). The traffic forecast at appraisal on the Gazakh–Georgian border section was 3,800 vehicles per day in 2011 and the actual traffic was 3,565 (6% lower than the target). Thus, the traffic levels forecast at appraisal were not achieved.

10 Access to local villages was hindered by the lack of turning possibilities and limited facilities for pedestrians

to cross the highway safely.


Road surface roughness. Road surface roughness is measured according to the IRI, with typical values ranging from 2.0 for a road in good condition, to 4.0 for a road in fair condition, 8.0 for a road in poor condition, and 12.0 or more for a road in very poor condition. The target value for the project after completion of the works was 2.5. Azeravtoyol possesses a deflectometer to measure roughness and in 2010 found that all sections of the project road had an IRI of 1.3 or better. These data appears to be somewhat too optimistic. The PCR assumed an IRI of 3.0 immediately after reconstruction, increasing to 4.4 or more prior to periodic maintenance. Similar values have been assumed in EBRD studies. As availability of the deflectometer is very limited in Azerbaijan, it was difficult to obtain reliable IRI data. Nevertheless, because the IEM observed that the road surface condition was good, the target was considered to have been achieved.

Accidents. The target reduction in the number of road accidents nationwide was 10% by 2012 from 2,388 in 2004. The actual number of accidents increased to 2,892 in 2012. However, considering the length of the project in comparison to the total road network length, the appraisal target was somewhat irrelevant to the project. The number of accidents on major arterial roads (M-roads) decreased from 868 in 2004 to 425 in 2012, a reduction of 50%. Although no official accident data are available for the project, the police department of Gazakh explained to the IEM that the number of road accidents was reduced by 50% in the Gazakh–Georgian border section after project completion. The target is considered to have been achieved.

Outputs. The 39.0 km Gazakh–Georgian border section had been rebuilt. To prevent adverse environmental impact during implementation, the resettlement plan was implemented and mitigation measures set out in the IEE were undertaken. Seven seminars on HIV/AIDS were held for project management staff, contractors and subcontractors. The improvement of local roads was dropped from the project scope and financed under subsequent ADB projects. According to the PCR, the national annual budget for road maintenance increased from $55 million in 2006 to $80 million in 2009, compared with the target of $65 million in 2009. A road maintenance unit was reorganized and operational in the Gazakh region. A road safety audit had been undertaken and a road safety audit manual and “black spot” improvement guidelines prepared. Guidelines were also prepared on environmental and social impact assessments and on management for road projects. Of the 12 weigh stations that were to be established across the country by 2007, only two were operational with another four being installed under other ADB-financed projects. The Yevlakh–Ganja section was dropped from the project scope but completed with financing by the IDB, the SFD, and the government. Improved cross-border facilities at the Georgian border crossing of Red Bridge were dropped from the scope and implemented under a government-funded project. In summary, almost all the envisaged outputs had been achieved except components dropped by the change in scope.

Safeguards. The PCR observed that safeguards were well-managed during project implementation. The IEM could not meet with people who were involved in implementing safeguards due to the staff turnover. The safeguard were also applied to the Yevlakh-Ganja section before the section was dropped from the project scope.

Rating. Of the six outcome indicators, two were achieved, one was dropped in the scope change, one was irrelevant, and two were not achieved (but one of the two was assessed under efficiency to avoid double counting). Although the outcome target for traffic was not achieved, it was only 6% lower than the target, and other two outcome targets were achieved. All the envisaged outputs have been achieved except


some components that were dropped in the scope change. The project is rated effective, the same rating as given in the PCR and the PVR.

Efficiency

Methodology of economic analysis. The economic costs and benefits of the project were analyzed using a standard evaluation approach, comparing the ”without project” (WOP) and ”with project“ (WP) scenarios. The project costs comprised capital investment and maintenance costs, as used in the PCR.11 In the PCR, maintenance costs were based on standard national norms for the WP scenario but no maintenance costs were included in the WOP scenario. In the current evaluation, 50% of the maintenance costs in the WP scenario were included in the WOP scenario. This assumes to be sufficient to maintain an IRI of 8.0. All costs were economic costs (i.e. exclusive of taxes and duties) at 2011 prices. A residual value of the infrastructure was added as a negative cost in the final appraisal year. The main sources of economic benefits were savings in VOCs, value of passenger travel time savings (VoTs) and accident cost savings. The traffic forecasts were revised based on updated traffic data counting and GDP forecasts. The economic analysis is described in detail in Appendix 6.

Vehicle operating costs. VOCs were estimated at 2011 prices by a World Bank mission using the standard Highway Development Model (HDM).12 Assuming an IRI of 8.013 before reconstruction and 2.0 after, unit VOCs decreased by $0.02/km for cars, $0.16/km for buses, $0.06/km for small trucks, and $0.12/km for large trucks. Unit VOCs were reduced by 6.7% for cars, 18.5% for buses, and 10.7% for trucks. Except for buses, these savings per kilometer were much less than the savings that were estimated in the PCR. The appraisal target reduction in VOCs after project completion was 40%. This target was unrealistically challenging, and it was not achieved.

Economic internal rate of return. At appraisal, the overall project was found to be economically viable, with an economic internal rate of return (EIRR) of 17.9% for the project overall, 12.2% for the Gazakh–Georgian border section, and 20.4% for the Yevlakh-Ganja section. The PCR reevaluated the project's EIRR at 21.5% for the Gazakh–Georgian border section. The PPER reevaluated the EIRR at 5.9% for the same section. Although the appraisal documents did not present data in sufficient detail to allow detailed comparison of the benefits, the lower traffic volumes and lower speed increases on the Gazakh–Georgian border section were major reasons for the lower recalculated EIRR. Traffic forecast is linked to GDP growth and recent economic downturn in Azerbaijan, and expected lower GDP growth affected recent and future traffic volumes. The Gazakh region, which is the project area of the Gazakh–Georgian border section, has lower average GDP growth than the national average, which might have negatively affected the traffic volume in the Gazakh–Georgian border section.

Sensitivity analysis. The revised sensitivity analysis showed that an increase of 20% in benefits would lead to an EIRR still below the economic viability threshold of 12%. In all cases of the sensitivity analysis, the EIRR never reaches 12%.

11 Actual maintenance costs were requested from Azeravtoyol but were not received. 12 World Bank. 2011. Improving the Sustainability of Road Management and Financing in Azerbaijan.

Washington, D.C. 13 The DMF assumed an IRI of 6.0 in the WP scenario. Using an IRI of 8.0 was a conservative assumption that

increased the VOC savings.


Process efficiency. In terms of process efficiency, the project was originally envisaged to be implemented over 4 years from 2006 to 2009, but the civil works for the Gazakh–Georgian border section started in February 2008 and were completed in March 2010. Due to repeated tendering, the project implementation schedule and civil work contract were delayed by about 9 months. The section was completed nearly one year later than scheduled.

Rating. In summary, the project is rated less than efficient as the EIRR of the project is less than 12%; while the PCR and the PVR rated the project efficient on the basis of PCR’s higher recalculation of EIRR exceeding 12%.

Sustainability

Financial sustainability. Since the project is non-revenue generating, the financial internal rate of return (FIRR) was not calculated. The maintenance of national roads is financed through the road fund administered by the Ministry of Finance. Fund sources comprised of seven categories of road user charges that include vehicle registration tax, import duties, and inspection fees, but does not include fuel taxes, which are assigned to the central budget. The road maintenance budget has increased steadily from $114 million in 2007 to $276 million in 2015. Funds for maintenance are allocated according to a ”needs-based“ approach. Actual annual maintenance expenditure needs were requested from Azeravtoyol, at both the national and the project levels. If traffic increases, government revenues will increase through user charges and be a financial source for meeting increasing maintenance needs. Considering that the road fund revenues have increased, the financial sustainability is considered to be generally satisfactory.

Road asset management. Efficient planning of road maintenance requires the collection and analysis of data, including an inventory of assets, road surface condition data, and traffic counts. A new road AMS was introduced in Azeravtoyol in 2011 through a World Bank initiative and a database was established to support the system. Equipment was acquired for measuring surface roughness and counting traffic, and software for maintenance planning was introduced. Consultants conducted capacity-building training in maintaining and running the system. However, the database was not connected with local maintenance units, and the managers who originally were responsible for the system were no longer employed by Azeravtoyol at the time of the IEM. The IEM observed that the system was not used.

Institutional sustainability. At the highest level, there was recent disruption when the responsibility for the planning, construction, and maintenance of roads was removed from MOT and transferred to Azeravtoyol. Many senior staff who had been involved with the project were no longer employed by Azeravtoyol, leading to concerns on continuity and sustainability. At the local level, regional maintenance units are responsible for the routine maintenance of all main and local roads. Their activities include (i) cleaning roads, (ii) making minor repairs, (iii) identifying cracks and reporting them to the head office, and (iv) repairing cracks. Given that the road surface condition had not deteriorated for 5 years after the completion of the project, the IEM notes that there is certain level of institutional capability to maintain roads sustainably.


Efficiency of maintenance. The efficiency of in-house maintenance units is considered to be suboptimal14 as the regional maintenance units have very high staffing levels compared with modern road agencies in other countries.15 Performance-based contracts will improve the efficiency of the maintenance units of Azeravtoyol as the payment to the units shifts from being “input-based” to being “output-based.” Azeravtoyol is planning to introduce performance-based contracts into a few maintenance units that were recently established to maintain highways.

Overloading. Overloading of trucks was a serious problem at appraisal, leading to pavement degradation and increased maintenance costs. Before appraisal, although the government had developed regulations to limit vehicle axle loads to 13 tons, enforcement was weak, due to limited institutional capacity and the lack of weighing facilities. Several weigh stations were built along the highway including two under another ADB project.16 All trucks are now monitored by induction loops and cameras located before each weigh station. Trucks that are suspected of being overloaded are stopped and weighed accurately at the stations. If the trucks are found to be overloaded, penalties are issued to the registered operator of the truck and the excess loads are offloaded in designated areas. Responsibility for operation of the weigh stations had recently been transferred from the MOT to the traffic police. Azeravtoyol officials indicated that the incidence of truck overloading has decreased. In a meeting with the Azerbaijan International Road Carriers Association, they confirmed with the IEM that enforcement related to overloading has been strengthened in recent years. Reduction of overloaded trucks will be positive for the sustainable assessment.

Rating. The project is likely sustainable financially and institutionally, which the PVR concurred. However, (i) the efficiency of the institution needs to be improved by introducing performance-based contracts, and (ii) the avoidance of staff turnover should be considered. Reduction of overloaded trucks also contributes positively to the rating. In practice, the IEM observed that the road surface conditions have not deteriorated for 5 years. Overall, the PPER also rates the project as likely sustainable.

14 World Bank. 2011. Improving the Sustainability of Road Management and Financing in Azerbaijan.

Washington, D.C. 15 The World Bank calculated a staff ratio of 54.6 staff per 100 km, compared with 5–10 per 100 km in more

modern road agencies in transition economies. 16 ADB. 2007. Report and Recommendation of the President to the Board of Directors: Proposed Multitranche

Financing Facility for the Road Network Development Program in Azerbaijan. Manila. (Loan 2354-AZE Tranche 1).

CHAPTER 4

Other Assessments

Development Impact

Performance target. The performance targets found in the DMF were (i) poverty incidence in the six regions reduced from 37.2% in 2003 to below 20% in 2020, (ii) per capita income for the bottom quintile of the population increased by 5% annually, (iii) the dropout rate in secondary schools reduced by 10%, (iv) grain production increased by 4% per annum, (v) industrial production increased by 7% per annum, (vi) trade across the border of Red Bridge with Georgia increased by at least 100% in 2020, and (vii) daily traffic across the Red Bridge border increased by 10% per year. Data on poverty incidence, income for the bottom quintile of the population, reduced dropout rate in secondary schools, and grain production were not available. These performance targets were disregarded for evaluation.

Economic development. The project has likely contributed to socioeconomic development in the Gazakh region. However, the average GDP growth of the Gazakh region was 10.4% during project implementation (2005–2010) and 5.2% after project completion (2010−2014), figures that were lower than the national averages of 17.2% during project implementation and 6.5% after project completion. The average GDP growth for the six regions that were originally intended project areas was 19.2% during project implementation (2005–2010) and 13.1% after project completion (2010−2014). This growth was higher than national averages. If the project impact area was extended to the six regions—since the project is corridor development—the project might have significant contribution to socioeconomic development. Thus, the project might have a significant impact on regional economic development.

Poverty reduction. Poverty in the project area was reduced through (i) improved access to markets for agricultural and industrial products as a result of better road conditions, (ii) improved access to employment, (iii) improved access to social services and education, and (iv) lower transport costs. Better conditions of the highway and local roads to villages enable the transport of fresh agricultural products with less damage and shorter journey time. For example, agricultural products such as tomatoes, cucumbers, and dairy products can be exported more easily via the highway to Georgia, Turkey and Baku and via Ganja airport to the Russian Federation. Shortening the commuting time from villages to cities broadens the opportunities of the local population to work in the cities. For example, travel time from Kemerli to Gazakh was reduced from one hour to 20 minutes; travel time from Asan to Gazakh was reduced from 40 minutes to 15 minutes.17

Employment and income. Average annual employment in the Gazakh region increased by 8.1% during the project implementation (2005–2010) and by 0.7% after project completion (2010–2014). At the national level, average annual employment

17 The local roads to these villages were financed by Loan 2354-AZE Tranche 1 under a MFF for Road Network

Development Program.

Other Assessments 15

increased by 0.9% during project implementation and 1.3% after project completion. The rate of increase in the Gazakh region was better than the rate for the national average during project implementation, but lower than the national average after project completion. Employment in the construction sector in the six regions increased by 35.7% during project implementation and 16.2% after project completion (data for the Gazakh region are not available), although it is not clear to what extent the employment was attributable to the project. An additional 80 local people were directly employed in permanent positions in Gazakh at road maintenance units of Azeravtoyol. The average annual rate of salary increase in the Gazakh region was 185% during project implementation and 7% after project completion. As the Gazakh region is a rural area, where industrial development is still weak, the direct employment impact of the project civil works might have a significant impact in the region.

New business. New business activities create job opportunities and stimulate the local economy. Business enterprises for textiles, cement factories, furniture shops, petrol stations, and service areas were established along the project highway after project completion. As the number of international long distance travelers increased, the service areas have additional restaurants, motels or sleeping cabins, stores, refueling facilities, and transport maintenance services (garages and workshops). Local roads have also promoted entrepreneurship among the villagers. A milk-gathering factory and a cotton factory were constructed in the Gazakh region after project completion.

International freight. The east–west highway is part of the CAREC corridor and is an important route connecting Europe via the Black Sea, Georgia, and Azerbaijan with the countries of Central Asia such as Kazakhstan and Turkmenistan via the Caspian Sea. It is also an important route from Turkey through Georgia and Azerbaijan to the Russian Federation. Average daily cross-border traffic in 2015 was 1,150 cars, 100 buses, and 430 trucks, which exceeded the appraisal target of 500 vehicles in 2020. Increased traffic and trade have brought significant benefits to the local economy, though it was not possible to quantify these benefits.

Rating. As the project is likely to have substantial positive development impacts and no significant environmental negative effects were reported, it is assessed as significant, as in the PCR and the PVR. The project appears to have an impact on socioeconomic development of regions along the corridor, as well as on the national economy by developing the key national transport corridor.

Performance of the Asian Development Bank

The PCR rated the overall performance of ADB as satisfactory. The project was administered from ADB headquarters with assistance from the ADB Azerbaijan Resident Mission. ADB conducted eight review missions and was closely involved in identifying and resolving issues, including conduct of site visits. Approval of documents at both processing and implementation stages, including processing of claims for payment, was punctual and without unreasonable delays. The role of ADB missions in providing timely advice on technical and contract administration matters was recognized by the EA.

However, it was misleading that performance on the Yevlakh-Ganja section was followed up by ADB after the section was removed from the project scope. Some stakeholders’ concerns on the weak technical design of the extended four lanes on the Yevlakh-Ganja section were still being brought up to ADB. With the change in scope, ADB should have clearly explained that ADB no longer assumes any responsibility for the Yevlakh-Ganja section. Also, it appeared that none of the ADB missions followed up on


the issue of monitoring and evaluation (M&E) with the EA. M&E reports were limited and did not comply with the specific loan covenant on the project’s M&E requirement. An unusual percentage of the loan amount was not disbursed. If more stringent due diligence had been done, the project would have financed a part of the local roads that was originally intended to be financed under the project. Furthermore, the PCR did not report key data that were necessary to fully comprehend the economic analyses (e.g. VoT unit costs, basis for the calculation of VOCs). This PPER rates the overall performance of ADB as satisfactory, the same rating as in the PCR and the PVR.

Performance of the Borrower and the Executing Agency

The PCR and the PVR rated the performance of the borrower and the EA as satisfactory. The PCR noted that an adequate organizational framework was established for efficient and timely project management. The government provided the required counterpart funds and other support in a timely manner. The EA conducted regular monitoring of construction progress including quality control of physical works. The PIU, with assistance from the consultants, prepared the required progress reports. The project accounts and financial statements were audited by external chartered accountants acceptable to ADB, and the audit reports were submitted to ADB as required under the loan agreement. The EA facilitated and supported ADB’s review missions during implementation and at completion. The PCR observed, however, that the draft project completion report received from the implementing agency lacked data and/or information on the results of performance indicators.

During the preparation of the PPER, it was difficult to obtain some data, because of the recent restructuring of the MOT and the fact that many of the staff who had been involved in the implementation of the project were no longer employed by the EA. Some shortcomings of the EA might have been caused by unfamiliarity with ADB procurement rules and procedures and inadequate staff capacity. Considering that this is the first road project financed by ADB in Azerbaijan, the IEM rates the performance of the borrower and the executing agency as satisfactory.

CHAPTER 5

Overall Assessment, Issue, Lessons, and

Recommendation

Overall Assessment

The project is assessed relevant, effective, less than efficient, and likely sustainable. Although the PCR assessed it as highly relevant, the IEM assessed it as relevant because (i) due diligence on the Yevlakh-Ganja section after the change in scope was not needed but was ambiguously continuted, (ii) 21% of ADB-approved loan amount was not used, and (iii) there was weakness in selecting DMF indicators. The target traffic volume was not achieved, although the targets for road surface roughness and accidents reduction were achieved. The Gazakh–Georgian border section had a low EIRR at 5.9% which is below the threshold of 12%. Thus, the PPER assessed the project as less than efficient. As the project is considered to be financially and institutionally sustainably, and overloading of trucks is better controlled, the project is likely to be sustainable. Compared with the PVR rating, efficiency was downgraded but other ratings were the same. In summary, the project is rated successful, as it has an overall weighted rating of 1.7518 (see Table below).

Overall Project Rating

Criteria Weight

(%)

Rating Rating

Value

Weighted

Rating

Relevance 25 Relevant 2 0.50

Effectiveness 25 Effective 2 0.50

Efficiency 25 Less than efficient 1 0.25

Sustainability 25 Likely sustainable 2 0.50

Overall Rating Successful 1.75 Note: Highly su essful ≥ .5 , su essful .5>S≥ .75), less than successful (1.75>LS≥ .75 , unsu essful < .75 . The overall rating becomes automatically less than successful if one or more of the four subratings’ value is 0. Source: Asian Development Bank Independent Evaluation Department.

Issue

The EIRR largely depends on the economic situation at the time of recalculation. The PPER found that the recalculated EIRR of less than 12% is mainly due to the recent economic downturn and the lower expected future growth in GDP in Azerbaijan. In

18 Independent Evaluation Department. 2016. Guidelines for the Evaluation of Project Sector Operations.

Manila: ADB. For a project to be rated "successful", it should have an overall rating greater than or equal to 1.75 and less than 2.50.


contrast, the PCR recalculated the EIRR when the economy was booming in Azerbaijan and thus obtained a higher EIRR. Economic analysis of infrastructure projects is strongly affected by economic forecasts. If the Azerbaijan economy becomes better than forecasted, the efficiency rating of the project could be improved.

Lessons

A large amount of unused loan should be avoided. Althoug the actual cost for the civil works sections increased by about 50%, the project still ended with unused loan proceeds of $9.65 million (Loan 2205) and SDR1.40 million (Loan 2206). These were sufficient to finance the local roads, which were estimated to cost $3.8 million. A thorough fund disbursement analysis should have been conducted during project implementation. The PCR recognized the need for greater due diligence during project design and formulation, which includes carrying out adequate cost estimations and setting aside enough resources to cover contingencies. The IEM also highlighted this lesson for future road projects.

Monitoring indicators should be carefully selected. Some indicators for impact and outcome are not readily available nor closely linked to the project. Since data are generally not readily available in Azerbaijan, sources for evaluation indicators should be identified in more detail at appraisal. Documents such as those prepared at appraisal and project completion should include sufficient data for subsequent verification and follow-up. Specific omissions were traffic data, unit benefits, and unit costs that were needed to fully comprehend the economic analysis.

ADB-financed project sections should be carefully selected. At appraisal, two sections were intended to be financed by ADB. The Yevlakh-Ganja section involves much higher traffic than the Gazakh–Georgian border section. Thus, the Yevlakh-Ganja section brought greater benefits and greater socioeconomic impact to the project area. If ADB had financed the Yevlakh-Ganja section instead of the Gazakh–Georgian border section, the project might have been rated efficient as the recalculated EIRR would have been higher than the threshold of economic viability. Thus, selection of the ADB-financed project section should have been made more carefully.

Recommendation

Institutional reform for road maintenance should be continued. The AMS was intended to be introduced to the EA under the project in cooperation with the World Bank and the EBRD. Although the AMS was intended to provide capacity development through training programs, it was not well accepted by the EA sustainably because it was too challenging for the EA. Instead, many development donors turned to introducing performance-based contracts to increase the efficiency of maintenance. ADB needs to keep supporting the reform related to the performance-based contracts taking into account the lessons learned from the introduction of the AMS.

Appendixes

APPENDIX 1: PROJECT DESIGN AND MONITORING FRAMEWORK Design Summary Performance Indicators Project Achievements

(as in PCR) PPER Update Assessment

Impact Accelerated socio-economic development in six regions (districts) of west Azerbaijan, and enhanced cooperation between Azerbaijan and neighboring countries.

Poverty incidence in six regions reduced from 37.2% in 2003 to below 20% in 2020.

Likely to be achieved. Poverty shows a reducing trend and per capita income has increasing trend, due to (i) increased access to employment and income, (ii) improved access to markets, and (iii) reduced price of transport, food and other daily necessities.

Average GDP growth of the six regions was 19.2% during project implementation (2005–2010) and 13.1% after project completion (2010-2014). However, the average GDP growth of the Gazakh region was 10.4% during project implementation and 5.2% after project completion. The GDP growth of the six regions was higher than national averages of 17.2% during project implementation and 6.5% after the project completion, but that of the Gazakh region was lower than the national average.

As data on poverty incidence were not available, it was difficult to assess the achievement.

Per capita monthly income from employment for bottom quintile of population in six regions increased by 5% per year from AZN 22,380 in 2003 to AZN 51,200 in 2020.

The average annual salary increase rate in the six regions was 41% during project implementation and 7% after project completion. The average annual salary increase rate in the Gazakh region was 185% during project

Achieved.

Project Design and Monitoring Framework 21

Design Summary Performance Indicators Project Achievements (as in PCR)

PPER Update Assessment

implementation and 7% after project completion. Average annual employment in the six regions increased by 1.6% before the project (2000–2005) but decreased by 0.27% during project implementation (2005–2010), and increased by 0.7% after project completion (2010–2014).

Dropout rate for secondary schools in six regions reduced by 10% by 2020.

Grain production in six regions increased by 4% per annum (pa), from 149,400 tons in 2003 to 290,000 tons in 2020.

Travel time for villagers was reduced by half and frequency of travel to town centers increased from 5 to 6 to about 15 times per year.

Industrial production in six regions increased by 7% pa from AZN310 billion in 2003 to AZN820 billion in 2020.

A total of 56 new small and medium-size commercial and business enterprises was established.

Trade across border with Georgia at Red Bridge increased by at least 100% in 2020 compared with 2004.

The total foreign trade of the country increased by an average of 30% pa over 2005–09 and reached $27.5 billion in 2009.

Daily traffic across the border at Red Bridge increased by 10% pa,

Cross-border traffic in 2010 was about 20 large passenger buses and 1,000 trucks per day, plus

Average daily cross-border traffic in 2015 was 1,150 cars, 100 buses and 430 trucks.

Achieved

22 Appendix 1



from 10 vehicles in 2004 to 500 vehicles in 2020.

many local passenger and freight vehicles.

Outcome Improved quality and efficiency of transport on the east-west highway and selected local roads.

Daily traffic increased from 4,000 vehicles in 2004 to 6,300 in 2010 on the Yevlakh-Ganja section, and from 2,300 vehicles in 2004 to 3,600 in 2010 on the Gazakh-Georgian border section.

Traffic on the project roads increased by an average of 10.8% pa in 2006–2010, reaching 7,400 AADT on the Yevlakh-Ganja section and 4,125 AADT on the Gazakh–Georgian border section. Achieved.

Gazakh-Georgian border: updated target year 2011 (first full year of operation) with target traffic of 3,800 vehicles. Actual traffic in 2011 was 3,565 (6% lower than the target).

Not achieved

The international roughness index (IRI) for the project section of the east–west highway reduced from more than 6.0 in 2005 to 2.5 in 2009 (after project completion).

Average IRI of 3.0 on Yevlakh-Ganja and Gazakh-Georgian border sections. Partially achieved.

IRI on project sections measured as 1.3 or better in 2010 (after project completion).

Achieved

IRI for improved local roads reduced from more than 6.0 in 2005 to less than 5.0 in 2009 (after project completion).

The local roads were dropped from the project and were constructed under subsequent ADB-assisted projects.

Vehicle operating costs (VOCs)on project roads reduced by 40% after project completion.

Total VOCs were reduced by 20% on average. In 2011, total VOC saving were estimated at about $11 million. Considering the very high increase in traffic, the indicator is considered achieved.

Unit VOCs at 2011 prices were reduced by 6.7% for light vehicles, 18.5% for buses and 10.7% for trucks.

Not achieved, although the target of 40% for VOC reduction target was unrealistic. Since this is an indicator for efficiency, it is better to be assessed within the context of the economic internal rate of return (EIRR).

Bus fares and freight charges on the project sections reduced by 10% by 2012.

Bus fares and freight charges were dropped from the evaluation as they are regulated by the government.

The indicator was irrelevant.




Road accidents reduced by 10% in 2012, compared with 2,388 in 2004.

Accident rate (persons killed) per 10,000 was vehicles reduced from 14.6 in 2004 to 9.5 in 2010. Likely to be achieved.

Nationwide there were 2,892 road accidents in 2012, an increase of 21% from 2004, therefore the target was not achieved. However, highway accidents were reduced from 868 in 2004 to 425 in 2012, a decrease of 50%. On the project highway, accidents were reduced by about 50%.

Achieved. Nationwide accidents are not linked with the project. The PPER assessed accidents on the project highway.

Outputs 1. Improved two-lane paved road on the Yevlakh–Ganja section and Gazakh–Georgian border section of east–west highway.

About 40km completed by mid-2007 and 127km by end of 2008.

Achieved. Some 94km of roads, comprising 39km on the Gazakh–Georgian border section and; 65km on the Yevlakh–Ganja section were rebuilt. The Ganja bypass was dropped and implemented under separate project.

Safety audits implemented during design and construction.

Achieved. Safety audits were incorporated in the detailed design.

Resettlement plan implemented satisfactorily before start of civil works.

Achieved. The resettlement plan was implemented satisfactorily and no complaints were reported.

Mitigation measures in IEE fully implemented during project.

Achieved. Mitigation measures in IEE were undertaken to prevent adverse

24 Appendix 1



environmental impact during implementation.

Risks of sexually transmitted infections (including HIV/AIDS) and health risks for construction workers, service providers and local residents adequately controlled.

Achieved. Seven seminars on HIV/AIDS were held for project management staff, contractors and sub-contractors.

2. Improved local roads in project area.

About 65km of 10 local roads improved by 2008.

Local roads were dropped from the project scope and improved under subsequent ADB projects.

3. Strengthened institutional capacity in road sub-sector.

3.1 Road maintenance. Annual maintenance funding increased from $32 million in 2005 to at least $41 million in 2006, $48 million in 2007, $57 million in 2008 and $65 million in 2009.

Achieved. The budget for road maintenance was $55 million in 2006, $80 million in 2009 and $72 million in 2011.

Six regional maintenance units reorganized and fully operational by 2007.

Achieved. Road maintenance units were reorganized and operational in all six regions. The ARS was also reorganized to manage the implementation of all road projects.




3.2 Axle load control Twelve vehicle weigh stations established and fully operational along main roads by 2007

Partially achieved. Two weigh stations (at Agstafa and Alat) were established. Four weigh stations are being installed under other ADB financed projects.

3.3 Road safety New road safety program implemented by 2008 road accident monitoring, reporting, and information systems developed by 2008

Achieved. A road safety audit was undertaken on the Kurdamir bypass in March 2009. A road safety audit manual and “black spot” improvement were guidelines prepared and a system developed.

3.4 Project management and implementation

Project implementation unit in Road Transport Services Department (RTSD) reorganized to manage all road projects supported by ADB, the EBRD and the World Bank before project implementation.

Achieved. The ARS reorganized to manage the implementation of all road projects financed by external aid agencies.

Operational guidelines on project implementation units prepared and adopted by 2008.

Achieved. Operational guidelines on harmonized project implementation units were prepared and adopted.

Guidelines on environmental and social impact assessments and management for road projects developed an adopted by 2008.

Achieved. Guidelines on environmental and social impact assessments and

26 Appendix 1



management for road projects were prepared and adopted.

4. Improved facilities and capacity for border crossing at Red Bridge.

Completed by November 2008. Achieved. The cross-border facilitation component was excluded from the project and implemented with government funds.

APPENDIX 2: PROJECT COSTS AND FINANCING PLAN

Table A2.1: Project Costs ($ million)

Item Appraisal Actual

Total Cost

ADB Financing

Total Cost

ADB Financing

A. Base Cost 1. Road Improvement Component

a. Land Acquisition and Resettlement 0.5 b. Civil Works i. Yevlakh–Ganja Section 39.1 14.4 ii. Gazakh–Georgian Border Section 16.8 14.2 45.0 34.7 iii. Local Roads 3.8 3.0 c. Consulting Services for Detailed Design and Construction Supervision 4.2 3.0 6.0 3.7 d. Environmental Management 0.2 0.1

2 Road Subsector Institutional-Strengthening Component a. Consulting Services 1.2 1.2 1.0 1.0 b. Equipment 1.8 1.8

3 Cross-Border Facilitation Component a. Civil Works 2.9 2.5 b. Consulting Services 0.8 0.8 c. Equipment 1.2 1.2

4 Project Management 1.0 0.8 0.4 0.0 5 Local Taxes and Duties 7.8

Subtotal (A) 81.3 43.0 52.4 39.4 B. Contingencies

1 Physical Contingencies 5.3 3.0 2 Price Contingencies 2.4 1.8

Subtotal (B) 7.7 4.8 0.0 0.0 C. Financial Charges

Interest and Commitment Charges during Construction 4.2 4.2 1.0 1.0

Subtotal (C) 4.2 4.2 1.0 1.0

Total (A+B+C) 93.2 52.0 53.4 40.4 ADB = Asian Development Bank. Sources: ADB. 2011. Completion Report: East–West Highway Improvement Project in Azerbaijan. Manila; ADB Loan Financial Information System; and Azer Road Service Open Joint-Stock Company.

28 Appendix 2

Table A2.2: Financing Plan Appraisal Actual

Source Amount

($ million) Share (%)

Amount ($ million)

Share (%)

Asian Development Bank 52.0 55.8 40.4 75.7

Ordinary Capital Resources 49.0 52.6 39.3 73.6

Special Funds Resources 3.0 3.2 1.0 1.9

Azerbaijan Government 19.8 21.2 13.0 24.3

Total 93.2 100.0 53.4 100.0

Sources: ADB. 2011. Completion Report: East–West Highway Improvement Project in Azerbaijan. Manila; ADB Loan Financial Information System; and Azer Road Service Open Joint-Stock Company.

APPENDIX 3: CHRONOLOGY OF MAJOR EVENTS

Date Main Event 2005 24 January–4 February ADB fact-finding mission held 10 May Management Review Meeting held 23 September Staff Review Committee Meeting held 8 December ADB Board approval granted

2006

1 June Loan agreements signed 27 October ADB fax to the MOT declaring loan effectiveness sent 7 December Contract with consultants for detailed design and construction supervision signed

2007

3 September Invitation to bid covering the two civil works contracts (lot 1: Ganja Bypass, and lot 2: Gazakh–Georgian border) posted on ADB website 20 September Contract for construction supervision of the Yevlakh–Ganja section signed 4 October First periodic financing request ($200 million) approved by ADB 7 November Detailed design completed

20 November Major change in scope approved by Vice President (Operation 1) reducing the scope of the east–

west highway reconstruction component from the originally approved two contract packages covering two road sections to one contract covering the Gazakh–Georgian border section and consequently reallocating the loan proceeds from subcategory 1A to subcategory 1B

2009

8 January Contract for institutional strengthening consultant signed 2010

31 January Contract for institutional strengthening consultant completed 30 March Civil works for the Gazakh–Georgian border section completed

30 March Contract for construction supervision of the Gazakh–Georgian border section completed

8 April Takeover certificate issued for the Gazakh–Georgian border section 31 May Original ADB loan closing date 7 June Final disbursement made under the loan

15 June ADB fax advice sent to the MOT regarding undisbursed balance under the two

loans to be canceled effective 15 June 2010; loans closed 2011

12–22 September ADB project completion review mission held

2016 16-28 May ADB independent evaluation mission held

ADB = Asian Development Bank, MOT = Ministry of Transport, RFP = request for procession, VO = variation order. Source: ADB. 2011. Completion Report: East–West Highway Improvement Project in Azerbaijan. Manila.

APPENDIX 4: ORGANIZATIONAL STRUCTURE FOR PROJECT IMPLEMENTATION

ARS = Azer Road Service, GG = Gazakh–Georgian border, PIU = project implementation unit, YG = Yevlakh–Ganja. Note: The MOT was dissolved in December 2015. The ARS, which was under the MOT, became independent in December 2015 and was transformed to Azeravtoyol in March 2016. The Yevlakh–Ganja section became irrelevant to civil works under the project after the project scope was changed in 2007. Source: Azer Road Service Open Joint-Stock Company.

APPENDIX 5: STATUS OF COMPLIANCE WITH MAJOR LOAN COVENANTS

Item

Reference in Loan/Project Agreement Status of Compliance

1. Project Executing Agencies Partially complied with.

RTSD will be the project Executing Agency for components A1 and B (Loan 2205), and C (Loan 2206) of the project and SCC will be the project Executing Agency for the carrying out of component D of the project.

LA, Schedule 6, para. 1

The MOT acted as the executing agency for the road-related components (the cross-border-related component was excluded from the project at completion).

2. Project Implementation Unit Complied with.

A joint PIU will be reorganized to manage all road projects supported by ADB, the European Bank of Reconstruction and Development, and the World Bank, including ADB-funded road improvement activities under the project. The PIU will comprise RTSD staff and externally contracted staff to supplement limited human resources in RTSD. It will be headed by a project director who will be assisted by a team of qualified technical, financial and support staff. SCC will appoint a project coordinator to manage implementation of the components C (Loan 2205) and D (Loan 2206) of the project and liaise with RTSD and other relevant government agencies during implementation.


The PIU for the project was established under the ARS, which was headed by a project director and comprised a team of qualified technical, financial, and support staff.

3. Project Steering Committee Complied with.

A PSC will be established to ensure close interagency coordination and supervise implementation. The PSC will be chaired by the Director of RTSD and will comprise representatives of the Ministry of Economic Development, Ministry of Finance, Ministry of Transport, PIU project director and the project coordination from SCC, Representatives of relevant local communities will be invited to attend PSC meetings for local road improvement. The PSC will meet at least quarterly to review the progress of implementation and provide approvals and guidance on Project implementation, as necessary.


The Steering Committee for the project was established in 2007, which comprised representatives from the Ministry of Economic Development, Ministry of Finance, and Ministry of Transport as well as the PIU project director and the SCC project coordinator. The PSC met at least quarterly to review implementation progress and provide approvals and guidance on project implementation as necessary.

4. Counterpart Funding Complied with.

The Borrower shall provide, on a timely basis, all funds and resources necessary for reconstruction operations, maintenance and management of the project facilities. The Borrower shall cause the Ministry of Transport to ensure that RTSD can successfully implement Components A1 and B as identified in Schedule1 and operate and maintain them after completion. The Borrower shall ensure that SCC successfully implements Component D as identified in Schedule 1.


The government provided required counterpart funds and all necessary supports for the project in a timely manner, with a total of $13.0 million.

5. Construction Quality Complied with.

The Borrower through RTSD shall ensure that the project roads are rehabilitated according to technical specifications of the design. In addition, the Borrower through the Ministry of Transport and RTSD shall also ensure that construction supervision, quality control and project management are performed according to internationally accepted standards.


The quality control of construction was carried out by the contractor and the construction supervision consultants. The consultants inspected and assessed the works to ensure that the standards and specifications in the project document were met. No remarkable quality problem was reported in the defect liability period.

32 Appendix 5

Item


6. Road Maintenance Complied with.

The Borrower shall ensure that a road maintenance program covering the country’s road network and based on the recently introduced pavement management system will be implemented.


The MOT, with assistance from the consultants, implemented the road maintenance program, including (a) developing and updating the road network database annually with the maximum use of staff of regional road maintenance units, (b) preparing annual road maintenance plans, (c) implementing the annual road maintenance plans, (d) providing training to ARS staff for operating the pavement management system database.

7. Axle Loads Complied with.

The Borrower shall cause RTSD to fully establish vehicle weighing stations along the roads in Azerbaijan, including the two stations located in the project area to enforce the axle-road regulations prior to project completion.


Two weight bridges of high quality (one in Agstafa and one in Alat) were established. The other four are under installation under other ADB-assisted road projects.

8. Project Management Complied with.

In the interest of providing continuity for project management, the PIU project director shall serve in the position throughout the project implementation period, except in case of gross dereliction of duties or such other serious cause provided, that such determination is made through appropriate due process. The Borrower, through RTSD, shall provide the PIU with adequate office space and support services in a form satisfactory to ADB throughout the implementation period. The Borrower, through RTSD, shall ensure that upon project completion, PIU staff is integrated in RTSD to implement future projects.

LA, Schedule 6, para. 8 (Loan 2205); para. 4 (Loan 2206)

During implementation, an adequate institutional arrangement was well established to ensure efficient and timely management of the project. The MOT and the ARS provided close monitoring and coordination to the construction progress and quality control of the project. The PIU director served throughout the project and was assisted by a team of qualified technical, financial, and support staff. The PIU, with assistance from the consultants, prepared the periodic project progress reports.

9. Project Auditing Complied with.

The Borrower shall cause RTSD to (a) undertake timely audit of project accounts according to sound auditing standards by an external auditor acceptable to ADB and (b) submit to ADB audited project accounts and related financial statements in English within 6 months of the end of each fiscal year during implementation.


The financial accounts and statements for the project were audited by external chartered accountants acceptable to ADB, and the audit reports were submitted to ADB as required.

10. Safeguard Policies and Co financing Complied with.

The Borrower shall (a) inform the cofinancers in writing of its obligation to comply with ADB’s policies on Involuntary Resettlement and Environment and the steps that will take to comply with such policies; and (b) immediately inform ADB if any cofinancer objects to the government complying with ADB’s such policies. All project activities financed by the cofinancers will be implemented in accordance with ADB’s policies on Involuntary Resettlement and Environment.


According to the surveys and interviews, there were no serious complaints concerning land acquisition and resettlement. The project implementation had no impact on indigenous peoples. The Resettlement Plan was satisfactorily implemented, and no outstanding issues on the land acquisition and resettlement were reported to ADB.

Status of Compliance with Major Loan Covenants 33

Item


11. Land Acquisition and Resettlement Complied with.

The Borrower, through RTSD and concerned local authorities, will ensure that (a) all land and rights-of-way required by the project are made available in a timely manner; (b) the resettlement plan prepared by RTSD and acceptable to ADB is implemented according to its terms, all applicable domestic laws and regulations and ADB’s Policy on Involuntary Resettlement; (c) the plan is updated based on detailed design and the updated resettlement plan is submitted to ADB for approval and disclosed to affected persons prior to the commencement of civil works; (d) no further road reserve clearance is undertaken in the project area until a socioeconomic assessment of affected persons is completed and appropriate mitigation measures are included in the updated resettlement plan, (e) civil works contractors’ activities are in compliance with requirements of the resettlement plan and ADB’s Policy on involuntary Resettlement; and (f) an independent monitor acceptable to ADB is engaged to carry out monitoring and evaluation and report to ADB.


The project required only minor land acquisition and temporary land occupation. The MOT prepared a resettlement plan to ensure that fair compensation and assistance would be provided for affected people or businesses. For the temporary impacts (e.g., vibration, noise, dust, and so on), the contractor undertook appropriate measures, such as the use of new rollers that cause minimal vibration and noise and frequent spraying of the road section under construction with water.

12. Environment Complied with. The Borrower, through RTSD, will ensure that (a) the project design, improvement and operation comply with applicable domestic laws, regulations and standards; (b) all environmental mitigation measures, institutional requirements and monitoring plans specified in the initial environmental examination are complied with; (c) the mitigation measures are updated during the detailed design; and (d) all the above requirements are incorporated in the bidding documents and civil works contracts to ensure compliance.


An environmental protection report1 was prepared during the detailed design. During implementation, the ARS and the contractors took measures to prevent adverse environmental impact in design and construction. Adequate drainage measures were incorporated in the project design and constructed. The PIU prepared four biannual environmental reports that recorded all the environmental issues and indicated no significant negative environment impacts during implementation.

13. Governance and Anticorruption Complied with. The Borrower, through RTSD and SCC, shall follow ADB’s Anticorruption Policy. Borrower acknowledges that ADB, consistent with its commitment to good governance, accountability and transparency, reserves the right to undertake directly or through its agents, investigation of any alleged corrupt, fraudulent, collusive or coercive practices related to the project. The Borrower will cooperate with such investigation and extend all necessary assistance including access to all relevant books and records and engaging in depended experts who may be needed for satisfactory completion of such investigations. All costs related to such investigations will be borne by the project. The Borrower through RTSD and SCC will ensure that the following anticorruption measures during project implementation are taken: (a) anticorruption provisions acceptable to ADB will be included in all bidding documents and contracts, including provisions specifying ADB’s right to audit and examine the records and accounts of the executing agencies and all contracts, suppliers, consultants and other service providers as they relate to the project; (b) all decisions relating to procurement will be made by the Tendering Committee that comprises representatives of the Ministry of Economic Development, Ministry of Finance, Ministry of Transport, State Customs Committee and State Procurement Agency in accordance with ADB’s Guidelines for Procurement and the Procurement law of Azerbaijan; and (c) the supervision consultant shall verify the


The government followed ADB’s Anticorruption Policy during implementation, including carrying out all the procurement and recruitment in accordance with ADB’s guidelines; established project account, which were audited by chartered financial auditors acceptable to ADB.

34 Appendix 5

Item


contractors’ payment claims in accordance with contract specification.

14. Labor Laws Complied with.

The Borrower, through RTSD, shall ensure that civil works contractors comply with all applicable labor laws and regulations as well as international treaties, do not employ child or forced labor for road improvement, provide equal opportunities for women in road improvement activities and promptly pay equal value. RTSD and SCC will ensure that specific provisions to this effect are included in all bidding documents and civil works contracts and that compliance is monitored on an ongoing basis.


The civil works contractors were required to meet all labor law standards, in terms of conditions of employment, safety in the workplace, insurance, and pay conditions, according to the national laws in Azerbaijan, as well as providing equal job and pay opportunities for women.

15. Gender and Development Complied with.

The Borrower, RTSD shall take measures to encourage contractors to employ women during project implementation and promote women’s participation in activities to monitor project impacts on local communities. RTSD and SCC shall ensure that the project is implemented according to ADB’s Policy on Gender and Development.


During implementation, the project hired 13 women among the 406 local laborers, which was significant considering the nature of construction work and the cultural restrictions for women in Azerbaijan.

16. Health Standards Complied with.

The Borrower, through RTSD and SCC, shall ensure that all bidding documents and civil works contracts require contractors to provide the following to construction workers at camp sites: (a) information on the risk of sexually transmitted infections; and (b) benefits on accidental death and disability, safety and security, sanitation and appropriate working conditions. The Borrower, RTSD and SCC shall also disseminate similar information to transport operators using project facilities and local residents in the project area.


The government undertook appropriate measures in accordance with applicable national laws and regulations to prevent trafficking of illegal substances and humans at the border with Georgia during implementation and after project completion. The HIV awareness program was conducted monthly. Documentation of HIV/AIDS information dissemination is discussed in the attached Social Impact Assessment Report.

17. Drugs and Human Trafficking Complied with.

The Borrower shall ensure that appropriate measures are undertaken in accordance with applicable domestic laws and regulations of Azerbaijan to prevent trafficking of illegal substances and humans at the border of Red Bridge with Georgia during project implementation and after project completion.


A national program was developed to establish a rehabilitation center for trafficking victims, where they will be provided with medical and psychological assistance. Measures to combat trafficking were undertaken on the national level.

18. Monitoring and Evaluation Partially complied with.

The Borrower, RTSD with assistance of the PIU and the supervision consultant shall monitor and evaluate project impacts to ensure that the project facilities are managed effectively and the project benefits maximized. The Borrower, RTSD and SCC shall collect the data agreed with ADB at the inception, project completion and 3 years after project completion.


Two monitoring and evaluation activities were carried out during implementation in 2009 and 2010. Socioeconomic impacts of the project were monitored. A final report was submitted to ADB.

ADB = Asian Development Bank; AIDS = acquired immunodeficiency syndrome; ARS = Azer Road Service Open Joint-Stock Company; HIV = human immunodeficiency virus; MOT = Ministry of Transport; PIU = project implementation unit; PSC = project steering committee; RTSD = Road Transport Service Department; SCC = State Customs Committee. 1 Kocks Consult GmbH. 2007. Environmental Protection Report for the Gazakh–Georgian Border Road. Koblenz.

APPENDIX 6: ECONOMIC REEVALUATION

General 1. The independent evaluation mission (IEM) carried out a re-evaluation of the economic costs and benefits of the project using a standard evaluation approach, comparing the “without project” (WOP) and ”with project“ (WP) scenarios. The Highway Development Model 4 (HDM-4) was used as the basis for calculating vehicle operating costs (VOCs). The methodology used for this evaluation is similar to those used at appraisal and for the PCR. Input data from the PCR were used as much as possible in order to maintain consistency in approach. However, not all necessary input unit values were reported in the PCR and certain assumptions had to be made. Economic indicators were calculated for the Gazakh–Georgian border section. General assumptions relating to the project assessment included an appraisal period of 20 years from the first year of operation, a social discount rate of 12% in accordance with ADB guidelines and a base year for monetary values of 2011.

Traffic Analysis 1. Traffic survey 2. A traffic survey was carried out by the IEM during 8-11 June 2016. Manual classified counts were carried out at one location at the Gazakh–Georgian border section, for 24 hours on two separate days (one weekday and one weekend), as shown in Table A6.1. All counts began at 3am, which had been previously identified as the time of lowest traffic. 3. The data obtained in the IEM survey were adjusted to average annual daily traffic (AADT) by applying a daily factor to convert the survey day to an average day of the week, and a seasonal adjustment factor to convert the survey month to an average month. A previous study1 on the corridor suggests that there is relatively little variation from day to day, therefore the daily factor was set to 1.0 weighted by five weekdays and two weekend days. The study indicates that traffic in June is 12% above the average, therefore, a conversion factor of 0.88 was applied. The resulting traffic volumes are shown in the Table A6.1 below. On the two sections, cars and trucks constitute about 75% and 10% of the total traffic respectively.

Table A6.1 Traffic Volumes, Gazakh-Georgian Border Section 2016 Location Motorcycle Car Van/Pick-

up Minibus Bus Trucks Tractor Total

Excluding M/C Trac

Total including M/C Trac

Two-axle

Three-axle

Four-axle

Five-axle

AADT 7 3,994 58 384 82 135 128 104 236 8 5,122 5,137 Share (%)

0.1 77.8 1.1 4.5 1.6 2.6 2.5 2.0 4.6 0.2 100

AADT = average annual daily traffic.

Source: Asian Development Bank Independent Evaluation Department.

2. Existing traffic

4. Table A6.2 shows traffic growth in the Gazakh–Georgian border section from 2001 to 2016. The table includes the ADB survey counts from 2016 and the counts by Azeravtoyol from 2011. It should be noted that although the Azeravtoyol data are identified as AADT, no seasonal adjustment factors were applied. The data for 2011 were collected in December and thus, are likely to underestimate AADT by about 6%. The data for 2014 were collected in August and is likely to overestimate AADT by about 15%.

1 Kocks Consult. 2007. Draft Final Engineering Report, Gazakh–Georgian Border road section.

36 Appendix 6

In this table, the Azeravtoyol data have been adjusted for seasonal variation and weighted according to the length of the section on which each count was made. The ADB survey counts exclude motorcycles and tractors and have been averaged across the survey days and adjusted for seasonal variation. 5. On this section, traffic has increased threefold since 2001 despite a decrease in 2011. At appraisal, the traffic forecast for 2016 AADT was 5,377 and the actual AADT was 5,121, which was 4.8% lower than the forecast. This difference, which may be related to a decline in GDP observed in the country since 2014, contrasts with the PCR forecast traffic growth of 8.0% per year during 2011–2015 and 7.6% in 2016. National traffic trends between 2014 and 2016 are not available. However, various sources note a decline in freight transport volume during this period. For example, according to the State Customs Committee, the total freight transport volume by all modes fell by 8.7% in the first half of 2016 compared with 2015, which was 7.4% lower than in 2014.2 Another source notes that the volume of cargo carried in the first quarter of 2016 on the Transport Corridor Europe-Caucasus-Asia in Azerbaijan amounted to 11.84 million tons, which is 12.4% lower than the volume during the same period of 20153. Overall, between 2001 and 2016 passenger traffic increased in relation to GDP with an elasticity of 0.70 and freight traffic increased with an elasticity of 0.96. 6. The low traffic on the Gazakh–Georgian border section can be also explained by low GDP growth in the Gazakh region. The average GDP growth of the Gazakh region in which is the Gazakh–Georgian border section lies was 10.4% during the project and 5.2% after project completion. In the five regions where the Yevlakh-Ganja section lies, the average GDP growth was 20.4% during the project and 15.0% after project completion. The average GDP growth of the Gazakh region was lower than the national average; in contrast, the average GDP growth of the five regions was higher than the national average.

Table A6.2 Traffic Growth Gazakh–Georgian Border, 2001–2016 (AADT)

Year Car Minibus

Van Bus Small Truck

Medium Truck

Large Truck Total Data Source

2001 1,314 158 39 94 25 52 53 1,735 PCR 2002 1,416 175 40 91 34 60 69 1,886 Interpolated 2003 1,526 194 41 89 48 68 89 2,055 Interpolated 2004 1,645 215 42 87 66 78 115 2,248 Interpolated 2005 1,773 239 44 84 91 89 148 2,468 Interpolated 2006 1,911 265 45 82 125 102 192 2,722 PCR/Kocks 2007 1,957 305 62 102 169 118 251 2,964 Interpolated 2008 2,003 352 86 127 228 137 328 3,260 Interpolated 2009 2,051 406 118 158 307 159 428 3,627 Interpolated 2010 2,100 468 163 196 415 184 559 4,085 PCR 2011 2,687 100 130 23 53 156 416 3,565 AAY database adjusted 2012 3,064 121 114 12 61 130 363 3,866 Interpolated 2013 3,493 147 101 6 71 108 318 4,244 Interpolated 2014 3,982 178 89 3 82 90 278 4,702 AAY database adjusted 2015 3,988 261 72 17 105 108 307 4,858 Interpolated 2016 3,994 384 58 82 135 128 340 5,121 Survey AADT = average annual daily traffic. Source: PCR survey, Kocks Consult survey, Azeravtoyol annual count, and Asian Development Bank Independent Evaluation Department.

3. Forecast traffic

7. For the purposes of the revised economic evaluation, traffic was forecast to the year 2030 for each section for light vehicles, buses, and trucks. Traffic growth was related to forecast growth in GDP,

2 http://1news.az/economy/20160719113855659.html 3 http://ru.apa.az/ekonomika-azerbaydjana/infrastruktura/nazvan-obem-gruzoperevozok-cherez-azerbajdzhan-v-ramkakh-

traceca.html

http://1news.az/economy/20160719113855659.html

http://ru.apa.az/ekonomika-azerbaydjana/infrastruktura/nazvan-obem-gruzoperevozok-cherez-azerbajdzhan-v-ramkakh-traceca.html

http://ru.apa.az/ekonomika-azerbaydjana/infrastruktura/nazvan-obem-gruzoperevozok-cherez-azerbajdzhan-v-ramkakh-traceca.html

Economic Reevaluation 37

using a typical elasticity for the region of 1.2 for passenger vehicles and 1.0 for trucks. The GDP forecast was obtained from the International Monetary Fund (IMF) database4 for the years to 2020, as shown in Table A6.3. Average GDP growth in the Gazakh region was lower than the national average, but since GDP growth forecasts in the regions were not available, the IEM used the forecast growth at the national level.

Table A6.3 Forecast Growth in GDP, 2016-2020 (%)

2016 2017 2018 2019 2020

2.465 2.577 3.592 3.545 3.283 Source: IMF.

8. The resulting traffic forecasts are shown in Table A6.4. On the Gazakh–Georgian border section, the forecast overall traffic growth is 3.0% in 2017, 4.2% in 2018 and 2019, 3.9% in 2020 and 3.5% per year thereafter. This is significantly lower than the PCR forecast of 7.6% per year during the period 2016–2020 and 4.8% per year thereafter.

Table A6.4 Traffic Forecasts in Gazakh–Georgian Border, 2017–2030

Year Light

Vehicles Bus Truck Total

2017 4,118 480 678 5,276 2018 4,295 501 702 5,498 2019 4,478 522 727 5,727 2020 4,654 543 751 5,948 2021 4,822 563 774 6,159 2022 4,995 583 797 6,375 2023 5,175 604 821 6,600 2024 5,361 626 845 6,832 2025 5,554 648 871 7,073 2026 5,754 671 897 7,322 2027 5,961 696 924 7,581 2028 6,176 721 952 7,849 2029 6,398 747 980 8,125 2030 6,629 773 1,010 8,412

AADT = average annual daily traffic. Source: Asian Development Bank Independent Evaluation Department.

4. Comparison of PPER and former traffic estimates 9. Traffic in the Gazakh–Georgian border section. Traffic in 2011 was 9% lower than forecast at appraisal and 23% lower than forecast in the PCR. In 2016, traffic was 5% lower than forecast at appraisal and 21% lower than forecast in the PCR. The revised forecast in overall traffic growth was 3.0% in 2017, 4.2% in 2018 and 2019, 3.9% in 2020, and 3.5% per year thereafter. This is significantly lower than the PCR forecast of 7.6% per year during 2016–2020 and 4.8% per year thereafter, largely due to lower forecasts of GDP. Reduced GDP growth resulted in forecast traffic that is 15% lower in 2020 and 33% lower in 2028 compared with the forecasts at appraisal. The PPER forecast is 31% lower in 2020 and 37% lower in 2028 than the PCR forecast (Table A6.5).

4 http://www.imf.org/external/data.htm

38 Appendix 6

Table A6.5: Comparison of Traffic Forecasts, Gazakh–Georgian Border Section, Selected Years (AADT)

Gazakh–Georgian border Year PPTA RRP PCR PPER 2011 … 3,888 4,399 3,565 2016 … 5,377 6,452 5,121 2020 … 6,970 8,641 5,948 2028 … 11,710 12,531 7,848

… = not availa le, AADT = average annual daily traffic, PCR = project completion report, PPER = project performance evaluation report,

PPTA = project preparatory technical assistance, RRP = report and recommendation of the President.

Source: ADB. 2004. Technical Assistance to Azerbaijan for Preparing the Yevlax-Ganja Road Rehabilitation Project. Manila; ADB. 2006. Report and Recommendation of the President to the Board of Directors: Proposed Loan to Azerbaijan for the East–West Highway Improvement Project. Manila; ADB. 2011. Completion Report: East–West Highway Improvement Project in Azerbaijan. Manila; and Asian Development Bank Independent Evaluation Department.

C. Costs and Benefits

1. Costs 10. The project costs comprised capital investment costs as used in the PCR and maintenance costs. The capital investment costs were the actual annual investment costs incurred, totaling $48.8 million for the Gazakh–Georgian border section. In the PCR, maintenance costs for the WP scenario were based on standard national norms, including an annual $30,000 per kilometers for a category II road and overlay every 5 years at 20% of the capital cost. While the PCR did not include maintenance costs in the WOP scenario, the PPER counted 50% of maintenance cost of the WP scenario as the input in the WOP scenario, assuming that the 50% of maintenance cost is sufficient to maintain a road with an IRI of 8.0. Thus, the PPER counted the cost more optimistically than the PCR. All costs were economic costs at 2011 prices. A residual value of the infrastructure was added as a negative cost in the final appraisal year. 2. Benefits 11. The main sources of economic benefits were savings in VOCs, passenger value of travel time savings (VoTs) and accident cost savings. 12. Economic benefits. As the project involved the reconstruction of an existing highway, the benefit calculation considered only existing traffic. It was assumed that there would be no diversion of traffic from other routes or modes, and no newly generated traffic. 13. Vehicle operating cost savings. The VOC savings were calculated by applying unit VOCs to the total vehicle–kilometers on each section. VOCs vary according to type of terrain, surface roughness and type of vehicle. 14. Road surface roughness is measured according to an international roughness index (IRI), with typical values ranging from 2.0 for a road in good condition, to 4.0 for a road in fair condition, 8.0 for a road in poor condition, and 12.0 or more for a road in very poor condition. Surface roughness was measured by Azeravtoyol between 2010 and 2014 using a deflectometer and is shown in Table A6.6. The data shows that for each section the IRI was generally less than 2.0. However, these data appears to be somewhat optimistic—the PPTA assumed an IRI of 2.5 immediately after reconstruction, increasing to 4.4 or more prior to periodic maintenance. Similar values have been assumed in EBRD studies. In the current analysis, an IRI of 2.0 (good condition) was assumed in the WP scenario and 8.0 (poor condition) in the WOP scenario.


Table A6.6: Surface Roughness on the Gazakh–Georgian border section, 2010–2014 (Average IRI)

2010 2011 2012 2013 2014

1.2–1.3 1.2 n/a 1.3 1.3–1.5

n/a = not available, IRI = international roughness index. Source: Azeravtoyol.

15. VOCs for different vehicle types and IRIs were estimated by a previous World Bank mission5 at 2011 prices using the industry standard HDM model for vehicles travelling on roads in flat terrain with varying IRIs.6 These VOCs were appropriate for the current analysis at 2011 prices and are shown in Table A6.7. The VOCs used for trucks are weighted according to the mix of trucks of different sizes observed in the traffic survey. These unit costs indicate lower savings than those estimated in the PCR. The unit savings stated in the PCR could only be achieved if the WOP road had a constant IRI of 15 throughout the appraisal period and the WP road had a constant IRI of 2 throughout the appraisal period. Comparison with the unit VOCs at appraisal was not possible as the values used were not reported in the PPTA documentation or in the report and recommendation of the President (RRP).

Table A6.7: Vehicle Operating costs, 2011 ($/veh.km)

Item IRI Light vehicles

Bus Truck

WP 2.0 0.28 0.53 0.67 WOP 8.0 0.30 0.65 0.75 Saving 0.02 0.12 0.08 ADB PCR saving 0.06 0.15 0.18 ADB = Asian Development Bank, PCR = project completion report, IRI – international roughtness indicators, WP = with project, WOP = without project. Source: World Bank and ADB. 2011. Completion Report: East–West Highway Improvement Project in Azerbaijan. Manila.

16. Time savings. In the PPTA, traffic speed was not considered as it was assumed that the savings resulting from reduced journey times would be very small. Benefits due to time savings were introduced in the RRP but neither the unit costs nor the basis for their derivation were presented. The PCR estimated that after project implementation vehicle speeds would double from 30-40km/h in the WOP scenario to 60-80km/h in the WP scenario. During the IEM, actual speeds were measured using the "moving observer" method and compared with speeds before reconstruction as provided by the Traffic Police. The results are shown in Table A6.8. On the Gazakh–Georgian border section, the speed of light vehicles increased from 55km/h to 84km/hsection, and the speed of heavy vehicles increased from 50 km/h to 55 km/h.

Thus, in all cases the speed increase after project implementation was less than forecast in the PCR. 17. For the re-evaluation, VOTs were derived from the average gross monthly wage rate of the country—AZN364.20 in 2011,7 —assuming 20% of trips were made for work purposes and an average number of 2.4 passengers per car and 13.2 passengers per bus.8 Trips made for work purposes were valued at the gross monthly wage rate plus employers overheads (estimated at 50% of the gross wage), while other trips were valued at 30% of the net monthly wage rate. The resulting VoTs per vehicle per hour were calculated to be $2.15 for cars and $10.94 for buses. The unit VoTs were increased in line with forecast growth in GDP on an annual basis.

5 World Bank. 2011. Improving the Sustainability of Road Management and Financing in Azerbaijan. Washington, D.C. 6 The universal method for calculating VOCs is to use the World Bank HDM VOC model. However, the model requires significant

amounts of input data which is time consuming and not always straightforward to collect. Furthermore, it is generally accepted that VOCs should be consistent between projects in the same corridor.

7 State Statistical Committee of the Republic of Azerbaijan. 2015. Statistical Yearbook 2015. Baku. 8 Kocks Consult. 2007. Draft Final Engineering Report, Gazakh–Georgian Border road section. Koblenz.

40 Appendix 6

18. Accidents. Accidents were not included in the analysis at appraisal. In the PCR 5% was added to the benefits to reflect unquantified savings, including those related to accidents. The IEM requested the policy department of Azeravtoyol to provide official accident data on the project roads but these data were not available. Therefore, an estimate was made on the basis of published national data. At the national level, the overall number of road traffic accidents in Azerbaijan increased between 2001 and 2015, but the number of accidents on major arterial roads (M-roads, including the east–west highway) decreased, from 49% of the total to 12%. Overall, however, fatalities increased from about 0.3 per accident to 0.4 and injury rates declined slightly from about 1.1 per accident to 1.0 (Table A6.8 and Figure A1). 19. Between 2011 and 2015, accidents on M-roads decreased by 11.8% per annum (pa). The national accident rates were used to calculate pro-rata estimates of numbers of accidents on the Gazakh–Georgian border section (a two-lane M-road), based on road standards and the length of the respective sections. The resulting estimated numbers of accidents in 2011 were 9 on the Gazakh–Georgian border, with 3.3 fatalities and 9.7 serious injuries. Slight injuries and damage-only accidents were not recorded in the data. 20. The economic cost of accidents was estimated on the basis of the costs assigned to countries that had levels of GDP per capita similar to Azerbaijan in 2011.9 These costs include the value of safety calculated based on a willingness-to-pay approach, direct and indirect costs (mainly medical and rehabilitation costs, administrative costs of the legal system and production losses) and material damage. These costs were $285,714 per fatality and $35,714 per serious injury. A conservative cost of $2,500 was added for slight injuries and no injuries. The resulting costs of accidents per million vehicle–kilometres were calculated to be $25,285 on the Gazakh–Georgian border section. These costs were increased in line with forecast growth in GDP on an annual basis.

Table A6.8: Road Accidents in Azerbaijan, 2001–2015

All M road Share

on Rate per accident

Year Accidents Accidents M roads

(%) Fatalities Injuries Fatalities Injuries

2001 1,985 975 49.1 559 2,228 0.28 1.12 2002 2,196 741 33.7 642 2,486 0.29 1.13 2003 2,311 837 36.2 724 2,691 0.31 1.16 2004 2,388 868 36.3 811 2,766 0.34 1.16 2005 3,179 1,035 32.6 1,065 3,668 0.34 1.15 2006 3,197 712 22.3 1,003 3,606 0.31 1.13 2007 3,104 645 20.8 1,077 3,432 0.35 1.11 2008 2,970 644 21.7 1,052 3,232 0.35 1.09 2009 2,792 533 19.1 930 3,044 0.33 1.09 2010 2,721 495 18.2 925 2,871 0.34 1.06 2011 2,890 401 13.9 1,016 3,031 0.35 1.05 2012 2,892 425 14.7 1,168 2,997 0.40 1.04 2013 2,846 365 12.8 1,164 2,948 0.41 1.04 2014 2,635 342 13.0 1,124 2,676 0.43 1.02 2015 2,220 257 11.6 894 2,265 0.40 1.02

Source: Azerbaijan State Statistical Committee.

9 HEATCO. 2006. Developing Harmonised European Approaches for Transport Costing and Project Assessment.


D. Economic Re-evaluation and Sensitivity Analysis

1. Economic evaluation 21. Based on the above estimates of costs and benefits, the economic internal rate of return (EIRR) was recalculated at 5.9% for the Gazakh–Georgian border section, compared with 12.2% at appraisal and 21.5% in the PCR. The recalculated EIRRs indicate that the return on the reconstruction of the Gazakh–Georgian border section was far below the threshold of 12%. Table A6.9 shows streams of costs and benefits. 22. On the Gazakh–Georgian border section, the appraisal documents did not present data in sufficient detail to make a comparison. The difference between the annual VOC, VoT and accident benefits in the PCR and in the reevaluation are significant, due to the lower traffic volumes and lower speed increase. In the reevaluation, benefits due to time savings were 63% lower in 2011, 62% lower in 2016, and 6% lower in 2030, than the appraisal estimates. The time saving benefits in the reevaluation were 86% lower in 2011, 83% lower in 2016, and 88% lower in 2030 than the PCR estimates.

Table A6.9: Economic Re-evaluation for the Gazakh–Georgian Border Road Section

($ million)

Year Capital Maintenance Total VOC Time Accidents Total Net

Benefit Net Present Value

2006 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2007 2.4 0.0 2.4 0.0 0.0 0.0 0.0 (2.4) (3.8) 2008 13.7 0.0 13.7 0.0 0.0 0.0 0.0 (13.7) (19.2) 2009 20.2 0.0 20.2 0.0 0.0 0.0 0.0 (20.2) (25.3) 2010 12.5 0.0 12.5 0.0 0.0 0.0 0.0 (12.5) (14.0) 2011 0.0 0.6 0.6 1.8 0.6 0.1 2.6 2.1 2.1 2012 0.0 0.6 0.6 1.9 0.8 0.2 2.8 2.2 2.0 2013 0.0 0.6 0.6 2.0 0.9 0.2 3.1 2.5 2.0 2014 0.0 0.7 0.7 2.1 1.0 0.2 3.3 2.6 1.9 2015 0.0 5.6 5.6 2.4 1.2 0.2 3.8 (1.8) (1.1) 2016 0.0 0.7 0.7 2.7 1.4 0.2 4.3 3.7 2.1 2017 0.0 0.7 0.7 2.8 1.5 0.2 4.5 3.8 1.9 2018 0.0 0.7 0.7 2.9 1.6 0.3 4.8 4.1 1.8 2019 0.0 0.8 0.8 3.0 1.7 0.3 5.0 4.3 1.7 2020 0.0 5.7 5.7 3.1 1.8 0.3 5.2 (0.4) (0.1) 2021 0.0 0.8 0.8 3.2 2.0 0.3 5.5 4.8 1.6 2022 0.0 0.8 0.8 3.4 2.2 0.3 5.9 5.1 1.5 2023 0.0 0.8 0.8 3.5 2.4 0.4 6.2 5.4 1.4

0

500

1,000

1,500

2,000

2,500

3,000

3,500

Nu

mb

er

of

acc

iden

ts

All M road

Figure A1: Road Accidents in Azerbaijan, 2001–2015

Source: Azerbaijan State Statistical Committee.

42 Appendix 6

Year Capital Maintenance Total VOC Time Accidents Total Net

Benefit Net Present Value 2024 0.0 0.9 0.9 3.6 2.5 0.4 6.5 5.7 1.3 2025 0.0 5.8 5.8 3.7 2.7 0.4 6.8 1.1 0.2 2026 0.0 0.9 0.9 3.9 2.9 0.5 7.2 6.3 1.1 2027 0.0 0.9 0.9 4.0 3.0 0.5 7.5 6.6 1.1 2028 0.0 1.0 1.0 4.1 3.2 0.5 7.8 6.9 1.0 2029 0.0 1.0 1.0 4.2 3.4 0.5 8.1 7.1 0.9 2030 (39.0) 1.0 (38.0) 4.4 3.5 0.6 8.4 46.4 5.4

Net Present Value: –32.67 Economic Internal rate of Return (EIRR): 5.9%

Discount Rate: 12.0% Source: Asian Development Bank Independent Evaluation Department.

2. Sensitivity analysis

23. The EIRRs were subjected to sensitivity analysis to test different scenarios of costs and benefits. The results are presented in Table A6.10. They show that on the Gazakh–Georgian border section the EIRR never reaches 12%.

Table A6.10: Sensitivity Analysis, EIRR (%) Scenarios

EIRR

Base Case 5.9 Sensitivity Tests 1 Maintenance Cost 10% Higher 5.7 2 Maintenance Cost 20% Higher 5.4 3 Benefits 10% Lower 5.1 4 Benefits 20% Lower 4.2 5 Benefits 10% Higher 6.7 6 Benefits 20% Higher 7.4 7 Maintenance Cost 10% Higher, Benefits 10% Lower 4.8 8 Maintenance Cost 20% Higher, Benefits 20% Lower 3.7 9 GDP growth 10% Lower 5.7 10 GDP growth 20% Lower 5.5 11 GDP growth 10% Higher 6.1 12 GDP growth 20% Higher 6.4

EIRR = economic internal rate of return.

Source: Asian Development Bank Independent Evaluation Department.