brazil - enhancing public management for service delivery...

Document ofThe World Bank

Report No.: ICR00003751

IMPLEMENTATION COMPLETION AND RESULTS REPORT(IBRD-83070)

ON A

LOAN

IN THE AMOUNT OF US$500 MILLION EQUIVALENT

TO THE

STATE OF RIO DE JANEIRO, BRAZIL

WITH THE GUARANTEE OF THE FEDERATIVE REPUBLIC OF BRAZIL

FOR A

ENHANCING PUBLIC MANAGEMENT FOR SERVICE DELIVERY IN RIO DE JANEIRO DEVELOPMENT POLICY OPERATION

January 31, 2017

Transport and ICT Global Practice Latin America and the Caribbean Region

CURRENCY EQUIVALENTS(Exchange Rate Effective as of November 29, 2016)

Currency Unit = Brazilian Real (R$) US$1.00 = R$3.40

FISCAL YEARJanuary 1 – December 31

ABBREVIATIONS AND ACRONYMS

AGETRANSP

Regulatory Agency for Public Transport in tge State of Rio de Janeiro (Agência Reguladora de Serviços Públicos Concedidos de Transportes Aquaviários, Ferroviários e Metroviários e de Rodovias do Estado do Rio de Janeiro.)

BRT Bus Rapid Transit (Transporte Rápido por Ônibus)BUI Integrated Transport Card (Bilhete Único Integrado)Casa Civil Office of the Chief of Staff (Secretaria de Estado da Casa Civil)CPS Country Partnership Strategy (Estratégia de Parceria com o País)DETRO Road Transport Department (Departamento de Transportes Rodoviários)DIME Development Impact EvaluationDPO Development Policy OperationDPL Development Policy Loan (Empréstimo para Políticas de Desenvolvimento)DSA Debt Sustainability AnalysisICR Implementation Completion and Results ReportKPI Key Performance IndicatorLOA Annual Budget Law (Lei de Orçamento Anual)M&E Monitoring and EvaluationMRJ Municipality of Rio de Janeiro (Cidade de Rio de Janeiro)MTEF Medium-Term Expenditure FrameworkNCR Net Current Revenue (Receita Corrente Líquida)NMT Non-motorized Transport (Transporte não Motorizado)PAD Project Appraisal Document

PDTUMaster Plan on Urban Transport in the Rio de Janeiro Metropolitan Region (Plano Diretor de Transporte Urbano da Região Metropolitana do Rio de Janeiro)

PDO Project Development ObjectivePPP Public-Private Partnerships (Parcerias Público-Privadas)

RJMR Rio de Janeiro Metropolitan Region (Região Metropolitana do Rio de Janeiro)

SRJ State of Rio de Janeiro (Estado de Rio de Janeiro)

SEASDH State Secretariat of Social Welfare and Human Rights (Secretaria de Estado Assistência Social e Direitos Humanos)

SEFAZ State Secretariat of Finance (Secretaria de Estado da Fazenda)SETRANS State Secretariat of Transport (Secretaria de Estado de Transportes)SPM Subsecretary of Women Policy (Subsecretaria de Políticas para as

Mulheres)UNOPS United Nations Office for Project ServicesVLT Veículo Leve sobre Trilhos (Light Rail Vehicle)

Regional Vice President: Jorge FamiliarSenior Global Practice Director: Jose Luis Irigoyen

Country Director: Martin RaiserPractice Manager:Gylfi Palsson

Project Team Leader: Daniel PulidoICR Team Leader: Bianca Bianchi Alves

BRAZILEnhancing Public Management for Service Delivery in Rio de Janeiro

Development Policy Operation

CONTENTS

Data Sheet

A.Basic Information.....................................................................................................................iB. Key Dates.................................................................................................................................iC. Ratings Summary.....................................................................................................................iD. Sector and Theme Codes........................................................................................................iiE. Bank Staff..............................................................................................................................iiiF. Results Framework Analysis..................................................................................................iiiG. Ratings of Program Performance in ISRs...........................................................................viiiH. Restructuring (if any)..........................................................................................................viii

Introductory Comments about the Country Context

1. Program Context, Development Objectives and Design.............................................................12. Key Factors Affecting Implementation and Outcomes...............................................................73. Assessment of Outcomes...........................................................................................................134. Assessment of Risk to Development Outcome.........................................................................225. Assessment of Bank and Borrower Performance......................................................................236. Lessons Learned........................................................................................................................267. Comments on Issues Raised by Borrower/Implementing Agencies/Partners...........................27Annex 1. Bank Lending and Implementation Support/Supervision Processes.............................28Annex 2. Beneficiary Survey Results............................................................................................30Annex 3. Stakeholder Workshop Report and Results...................................................................31Annex 4. Summary of Borrower's ICR and/or Comments on Draft ICR......................................32Annex 5. Comments of Co-financiers and Other Partners/Stakeholders......................................33Annex 6. Review of the Fiscal and Debt Sustainability Analyses Provided During Preparation. 34Annex 7. Background on Results for Policy Objective 1.2 Regarding Revenue Forecasting Errors.......................................................................................................................................................36Annex 8. Public Transport System in Rio de Janeiro....................................................................39Annex 9. Supported Policy Areas..................................................................................................43Annex 10. Results Framework......................................................................................................46

A. Basic Information

Country: Brazil Program Name:

Enhancing Public Management for Service Delivery in Rio de Janeiro

Program ID: P147695 L/C/TF Number(s): IBRD-83070ICR Date: 01/31/2017 ICR Type: Core ICR

Lending Instrument:Development Policy Loan (DPL)

Borrower:STATE OF RIO DE JANEIRO

Original Total Commitment:

US$500.00 million Disbursed Amount: US$500.00 million

Revised Amount: US$500.00 millionImplementing Agencies: State Secretariat of Transportation (SETRANS) Casa Civil (Office of the Chief of Staff) Secretariat of Finance (SEFAZ) State Secretariat of Social Welfare and Human Rights (SEASDH)Co-financiers and Other External Partners:

B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 09/30/2013 Effectiveness: — 12/06/2013 Appraisal: 10/18/2013 Restructuring(s): — 01/30/2015 Approval: 11/21/2013 Mid-term Review: — 03/06/2015 Closing: 01/31/2015 01/31/2016

C. Ratings Summary

C.1 Performance Rating by ICR Outcomes: Unsatisfactory Risk to Development Outcome: High Bank Performance: Moderately Unsatisfactory Borrower Performance: Unsatisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR)Bank Ratings Borrower Ratings

Quality at Entry: Unsatisfactory Government: UnsatisfactoryQuality of Supervision: Satisfactory Implementing Unsatisfactory

i

Agency/Agencies:Overall Bank Performance:

Moderately Unsatisfactory

Overall Borrower Performance: Unsatisfactory

C.3 Quality at Entry and Implementation Performance IndicatorsImplementation

Performance Indicators QAG Assessments (if any) Rating:

Potential Problem Program at any time (Yes/No):

NoQuality at Entry (QEA):

None

Problem Program at any time (Yes/No):

YesQuality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Moderately Unsatisfactory

D. Sector and Theme Codes

Original ActualMajor Sector/Sector Public Administration Sub-National Government 36 36 (Historic)Health and other social services Other social services 14 14 Transportation Urban Transport 50 50

Major Theme/Theme/Sub Theme Environment and Natural Resource Management Climate change 28 28 Mitigation 28 28 Human Development and Gender Gender 14 14 Public Sector Management Public Administration 18 18 Transparency, Accountability and Good Governance 18 18 Public Finance Management 18 18 Public Expenditure Management 18 18 Social Development and Protection Social Inclusion 9 9 Indigenous People and Ethnic Minorities 3 3 Other Excluded Groups 3 3

Participation and Civic Engagement 3 3 Urban and Rural Development Urban Development 41 41 Urban Infrastructure and Service Delivery 41 41

E. Bank Staff

Positions At ICR At Approval Vice President: Jorge Familiar Hassan Tuly Country Director: Martin Raiser Deborah Wetzel Practice Manager/Manager: Gylfi Palsson Aurelio Menendez

Program Team Leader: Daniel PulidoShomik Raj Mehndiratta / Georges Darido

ICR Team Leader: Bianca Bianchi Alves

ICR Primary Author:Bianca Bianchi Alves/ Fatima Arroyo Arroyo

F. Results Framework Analysis

Program Development Objectives (from Project Appraisal Document)The objective of the proposed operation is to assist the Government of Rio de Janeiro in its efforts to enhance public management for the delivery of key public services for vulnerable populations by instituting new policies and regulations to improve the medium-term planning and monitoring of public expenditures (Policy Area 1); the accessibility, quality, and affordability of urban mobility services for the poor (Policy Area 2); and the availability of targeted social services aimed at reducing domestic and gender-based violence (Policy Area 3). This objective directly contributes to the poverty reduction and shared prosperity agendas, insofar as financially sustainable and targeted urban mobility and social support services benefiting the poor and other populations at risk, such as women, enable more equitable access to economic and social opportunities, allowing the benefits of growth to be shared more widely.

Revised Program Development Objectives (if any, as approved by original approving authority)Not applicable.

(a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target

Values

Actual Value Achieved at

Completion or Target Years

Indicator 1: Maintenance of the number of events in which the operational risk policy procedures were not complied with within already achieved satisfactory levels

Value (Quantitative or Qualitative)

Occurrences = 52; non-conformities = 12

Occurrence = 48; non-conformities = 11





approval documents)


Values



Date achieved 12/31/2012 12/31/2014 12/31/2015 12/31/2015Comments (including % achievement)

Target 900% and 400% achieved for 2015. Data for 2016 (until September 2016): Occurrence = 11; non-conformities = 10 (achieved)

Indicator 2: Reduction in the number of times in a year that the actual monthly cash balance has remained below the minimum threshold (set at 1.5 times the payroll amount)


3 out of 12 2 out of 12 2 out of 12 2 out of 12


Target marginally achieved for 2014 and 2015. Data from January to September 2016 shows that the indicator was not achieved in 7 out of 9 months.

Indicator 3: Reduction in the percentage of payments made after 30 daysValue (Quantitative or Qualitative)

3.6% 3.0% 3.0% 9.2%


Target not achieved for 2015, since there was an increase instead of reduction. Data from January to September 2016 reflects worsening to 12.6%.

Indicator 4: Reduction in the gap between actual and estimated current revenues, measured as a share of the estimated revenues


6.6% 5% 5% 5.5%


Target not achieved for 2015. Calculating the indicator with public data generates an even larger gap (Over 216% deviation in relation to client reported numbers - see Annex 7).

Indicator 5: Publication in the Internet of revenue estimates documents for the 2014 budgetValue (Quantitative or Qualitative)

No Yes Yes Yes


Target achieved.

Indicator 6: Annual Budget directive issued for the 2015 budget cycle Value (Quantitative or Qualitative)

No Yes Yes Yes

Date achieved 12/31/2012 12/31/2014 12/31/2015 12/31/2015Comments Target achieved. However, the budget directive was issued internally, not published



approval documents)


Values



(including % achievement)

publicly, which was the recommendation of the Bank, although not explicit in the target value.

Indicator 7: A comprehensive report including current and prospective long-term commitments and liabilities associated to PPPs is published as scheduled


No Yes Yes Yes


Target achieved. However, no public-private partnerships (Parcerias Público-Privadas, PPPs) have been executed to date, and only a few are under consideration.

Indicator 8: All PPP-related bidding documents, contracts, and supporting analysis are published on a dedicated website in a timely manner


No Yes Yes Yes


Target achieved. However, no PPPs have been implemented to date and only a few are under consideration.

Indicator 9: Permanent Committee for the implementation of PDTU, involving relevant government and civil society actors, meets 4 times in its first year of existence and presents strategy for the implementation of priority project under PDTU.


0 4 4 4


Target achieved, despite the delays in completing the PDTU, financed by the related World Bank Investment Policy Financing Operation (P111996).Line 2 – Estacio-Carioca link was defined as priority and an implementation strategy was defined.

Indicator 10: No transport project outside of the PDTU (that is, not endorsed by the PDTU) will be undertaken by the SRJ


No Yes Yes Yes


Target achieved. The Bank recommended further dissemination of the PDTU and a commitment by other financiers to only fund projects in the priority list, since several agencies might undertake transport projects and interests may differ.

Indicator 11: At least one (1) multimodal integration project, connecting state and municipal transit systems, is completed


0 1 1 2



approval documents)


Values




Target 200% achieved. Both Maracanã and Madureira stations were completed.

Indicator 12: State prepares guidelines for the development of bicycle-public transit integration projects to be implemented by metropolitan municipalities and at least one (1) of these projects is implemented with the State’s technical assistance


0 1 1 2


Target 200% achieved. The state prepared the Guidelines of Non-motorized Transport (Transporte não Motorizado, NMT) and implemented projects with the state’s technical assistance component.

Indicator 13: The concession bidding documents will identify the bus lines to be sectioned or eliminated in order to prioritize integration with mass transit modes (name, current code, and in the cases of sectioning, new extension, new fleet and estimated demand)


0 19 19 90

Date achieved 12/31/2013 12/31/20 14 12/31/2015 12/31/2015Comments (including % achievement)

Target partially met. 90 lines were identified to be potentially eliminated, but the elimination will only occur after the award of the concessions and in a gradual manner.

Indicator 14: Inter-municipal bus lines in the RJMR (previously provided under permission regime) are in operation under new concession agreements incorporating performance monitoring


0

50% of the bus lines under the new route plan are awarded to competitively selected concessionaires

50% of the bus lines under the new route plan are awarded to competitively selected concessionaires

0

Date achieved 12/31/2013 12/31/2014 12/31/2015 12/31/2015

Comments (including % achievement)

Target not met. Nevertheless, the SRJ completed all public hearings required for launching the bidding process and has received comments from the State Auditor. The concession documents are being reviewed and are expected to be issued in March 2017.

Indicator 15: Service reliability, fleet modernization, and vehicle occupation requirements are properly monitored and enforced on inter-municipal bus concessions in operation


n.a. 80% of bus concessions either comply with requirements or

80% of bus concessions either comply with requirements or

82%



approval documents)


Values



are penalized are penalized Date achieved 12/31/2013 12/31/2014 12/31/2015 12/31/2015

Comments (including % achievement)

Target partially met. DETRO concluded the procurement process for the additional software licenses for real-time performance monitoring of bus services; 9,000 bus lines from the 11,000 are being monitored despite the new concessions not being in place.

Indicator 16: New operations contract awarded for the cable car system incorporates performance monitoring, availability-based remuneration, and incentives for maximizing ancillary revenue; and results in a reduction in the operational subsidies paid by the state


System operated under temporary arrangement with a subsidy of R$14.00 per passenger per trip

New operational contract results in a 5% reduction in the subsidy per passenger per trip and is monitored and remunerated using performance indicators

5% 14%


Target not met. The Government did enter into a new contractual scheme with a new cable car service operator that reduced subsidies by 14%, but the service has been suspended by the contractor because of fiscal constraints.

Indicator 17:

Number of women benefiting from increased access to information and to social and legal services as per the provisions of the “Lei Maria da Penha” within the Suburban Rail Operator in the RJMR (Operador do sistema de trens urbanos), the SuperVia and Teleférico systems.


0 50,000 50,000 53,732


Target partially met. 53,732 accessed the information kiosks, but the actual number of women accessing the social and legal services due to the program is unknown as the databases are not integrated.

Indicator 18: Increase in the number of women benefiting from the integration of the SuperVia and Teleférico systems to the ‘Casa da Mulher Brasileira.’


0 1,000 1,000 0


Target not met. Following the confirmation that the ‘Casa da Mulher Brasileira’ would not be built by the Federal Government, the World Bank agreed to replace with one of the state-run centers, but this was also not achieved.

G. Ratings of Program Performance in ISRs

No. Date ISR Archived DO IP Actual Disbursements

(US$, millions) 1 03/07/2014 Satisfactory Satisfactory 500.00 2 07/12/2014 Moderately Unsatisfactory Unsatisfactory 500.00 3 01/02/2015 Unsatisfactory Moderately Unsatisfactory 500.00 4 06/09/2015 Moderately Unsatisfactory Moderately Satisfactory 500.00 5 10/26/2015 Moderately Unsatisfactory Moderately Satisfactory 500.00 6 01/29/2016 Moderately Unsatisfactory Moderately Satisfactory 500.00

H. Restructuring (if any)

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in (US$, millions)

Reason for Restructuring & Key Changes MadeDO IP

01/30/2015 U MU 500.00

The project closing date was extended by one year from January 31, 2015 to January 31, 2016. The operation suffered technical implementation delays as a result of unforeseen legal constraints and institutional changes during the 2014 electoral period. This extension of the loan was expected to allow the client to fully achieve the program’s targets within the new terms and, therefore, achieve the proposed PDO.

Introductory Comments about the Country Context

1. Given the extraordinary and unprecedented challenges that Brazil has had during the 2013-17 period, the evaluation of the implementation of federal and subnational investment project within that period require some level of understanding of the overall macroeconomic, political and operational context that prevailed in the country. The following paragraphs are intended to provide a summary background on that period.

2. Following years of relatively rapid growth during the commodity boom from 2004-2010, Brazil entered a period of declining growth from 2011-2014. Growth averaged 4.5% between 2004 and 2010, 2.4 percent between 2011 and 2014, culminating in near stagnation in 2014. During the period of declining growth from 2012-2014 a number of States requested support to maintain services and, where possible, to accommodate some fiscal adjustment. At the time the slowdown was seen by all analysts as a temporary phenomenon, likely to result it zero growth in 2015 but a gradual recovery thereafter.

3. The slowdown of 2011-2014 was followed by the deepest recession in more than a century in 2015 and 2016. Growth in the previous decade was largely based on favorable external conditions, consumption, and an expanding labor force. As these drivers were increasingly exhausted, structural constraints on potential growth became binding.1 Moreover a serious structural fiscal problem became apparent. During the boom rising real revenues were able to accommodate increasing (and downwardly rigid) expenditures, particularly a generous pension system and other expenditure commitments enshrined in the constitution. As growth declined so did public revenues. Assuming that the slowdown was temporary, the policy response was to try and boost the economy through subsidized credit, tax breaks and other subsidies, all of which contributed to a deterioration of the fiscal position. The recession in 2015 was further deepened by the impact of the Lava Jato corruption investigation, which contributed to paralyzing investment, reducing the ability of Government to act, and driving market confidence to record low levels. The result was that in 2015 GDP fell by 3.8 percent, and is estimated to have fallen by a similar amount in 2016.

4. Falling revenues and rigid public spending drove the fiscal deficit to over 10 percent of GDP in 2015. The primary balance deteriorated from a surplus of 2.2 percent of GDP in 2012, to a deficit of 1.9 percent in 2015, and despite fiscal adjustment measures is likely to reach a deficit of 3.1 percent in 2016. Since at least the beginning of the twentieth century the Brazilian economy has never experienced such a fall in GDP over two years. During 2015 and early 2016 the government attempted to pursue fiscal consolidation but was ultimately unable to reverse the trend of rising deficits, owing to the broad political crisis, including an institutionally paralyzing presidential impeachment process which impacted decision making and investment implementation at the Federal, State and Municipal government levels, and the ever widening Lava-Jato investigation, which implicated many politicians across all major political parties. This

1 These include infrastructure bottlenecks, high labor costs and low skill levels, a high tax burden and onerous tax system, excessive administrative burdens, shallow credit markets, and barriers to competition and international trade.

undermined market credibility and kept interest rates high, resulting in an even deeper economic contraction.

5. The effects of the crisis on subnational finances has also been devastating. The boom of revenues until 2011 benefited the subnational level as states receive the bulk of their income from complex rule based tax sharing mechanisms, and, in the case of a few states such as Rio de Janeiro, from oil royalties. The need and opportunity to advance important structural reforms during that boom period was missed by all stakeholders and, when the economy started tanking, legislative, policy, structural and political rigidities made fiscal adjustment almost impossible to pursue. Heavily indebted states such as Rio de Janeiro, Rio Grande do Sul and Minas Gerais, cut the little remaining investment and very soon became unable to meet payments of suppliers or wages on time. The current Administration is attempting to implement some key needed reforms (expenditure ceiling, pension reform, public employment and salaries, etc.) but still needs to overcome important legal and political obstacles.

6. Within that overall context, both Federal Agencies and sub-national governments (State and to a lesser degree Municipalities) saw their operational implementation capacity significantly eroded as public budgets shrank as a result of reduced revenues (taxation and royalties) and inter-governmental transfers. The heightened economic uncertainty, a short-term vision resulting from the two year election cycle, and the bureaucratic caution -- resulting from the presidential impeachment based on the violation of the Fiscal Responsibility Law and on the Lava Jato investigation -- have caused government counterparts to be extremely cautious and many investment and project implementation decisions and restructuring approvals to be significantly delayed. This affected the WBG’s Portfolio quality and required substantial project restructuring as the implementation context changed radically from what was assumed at Appraisal.

7. This ICR presents the detailed implementation completion review of one of the Bank’s projects most affected by the overall crisis in Brazil, and in particular in the State of Rio de Janeiro, which is the state most affected in the country to date

1. Program Context, Development Objectives and Design

1.1. Context at Appraisal

1. From 2011 to the time of appraisal (October 2013), macroeconomic indicators pointed to an economic slowdown in the State of Rio de Janeiro (Estado de Rio de Janeiro, SRJ). Industrial production was fairly stagnant since 2010, growing at an annualized rate of less than one percent between end-2010 and July 2013, which is well below the national performance of 3.5 percent. The low growth of industrial production over this period was driven by a sharp drop in extractive industries (-7.7 percent) which was balanced by growth in manufacturing of 2.9 percent, still below the national level of 3.7 percent (all average annual growth rates since end-2010). In addition, retail sales—a traditional source of resilience—had slowed, from 6.8 percent in 2011 to 4.1 percent in 2012 and 4.4 percent in 2013. Fiscal trends were already challenging as in 2012, for the first time in many years, the primary balance of the SRJ became negative, approaching R$3 billion. Although the SRJ remained in compliance with the Fiscal Responsibility Law (Lei de Responsabilidade Fiscal),2 a declining trend in public finances was apparent and accelerated strongly after appraisal.

2. At the same time, investment expenditure and payroll pressure were increasing rapidly and turning the state’s fiscal situation unsustainable. Rio de Janeiro committed to host two main international events—the World Cup in 2014 and the Olympics in 2016. At appraisal, Rio’s infrastructure investments increased to respond to the preparation for these events. Urban transport and urban development, which were required to fill the infrastructure gap in the state, were accelerated. While investment had been the fastest growing expenditure from 2008 to appraisal, payroll expenditures almost doubled (in nominal terms) in the same period, and there was no evidence of a significant effort to contain this growth. Pressures also stemmed from ongoing commitments in security and health, which required permanent spending on both personnel and maintenance.

3. Despite considerable improvements in the public service delivery in the state and in the Rio de Janeiro Metropolitan Region (Região Metropolitana do Rio de Janeiro, RJMR), the quality, availability, and affordability of services had not kept up with the growing expectations of the population. Citizen dissatisfaction with public transport service and high tariffs were catalysts for the protests that paralyzed many Brazilian metropolitan areas, including the RJMR, in June 2013. The protests reflected a broader perception of inadequate public services, concerns with overall government investments and management, and frustration with prosperity not shared. These protests increased the pressure on governments across Brazil to implement significant reforms to improve public service delivery and enhance the quality of public sector management.

2 Net Consolidated Debt to Net Current Revenues (Receita Corrente Líquida, NCR) remained under the 200 percent threshold, reaching 155.4 percent in April 2013. The limit on the value of guarantees given, which was 22 percent of the NCR, has also been abided with. The value of the outstanding guarantees has remained under 6 percent of the NCR since 2004. Lastly, consolidated personnel expenditures amounted to R$15.8 billion in 2012, which corresponds to 39 percent of the NCR, well below the 60 percent limit. However, reported personnel spending was lowered significantly by unusual accounting measures used by the state.

1

4. The RJMR had not carried out the complementary reforms necessary to maximize the impact of urban transport investments. The state was making important investments in the modernization of the suburban rail operator (Operador do sistema de trens urbanos) SuperVia in the RJMR and ferry systems (Barcas), as well as in the expansion of the metro system (Line 4). The Municipality of Rio had considerably built up a Bus Rapid Transit (Transporte Rápido por Ônibus, BRT) network. However, these important investments had not been accompanied by complementary reforms to promote more efficient use of the urban transport system, including the reorganization of bus services to enhance integration with mass transit modes.

5. In this critical context, the SRJ sought funds to ensure the continuation of service delivery. In this account, the State Secretariat of Finance (Secretaria de Estado da Fazenda, SEFAZ) put increased emphasis on the efficiency of financial management (Program Policy 1) and service delivery with a focus on two key services: (a) urban mobility services in the RJMR (Program Policy 2); and (b) social services to address domestic and gender-based violence (Program Policy 3).

1.2. Rationale for World Bank Involvement

6. The operation had its origins in the fiscal problems the state government was facing and the opportunities for adjustment under these circumstances. Given the fiscal difficulties of the state and the upcoming major events, with the imperative need for maintenance of service delivery, the Federal Government and the state government requested the World Bank’s assistance for a continuation of a long-term partnership with the state over the previous five years. The funds provided by the operation were seen as a temporary measure, enabling the state to survive the crisis until fiscal reforms were possible, as immediate fiscal adjustments were difficult given the upcoming World Cup (2014) and Olympics (2016). A discussion of the team’s assessment on the adequacy of the macroeconomic framework is included in annex 6.

7. The operation was part of a long-term engagement with the SRJ and followed four previous Development Policy Loans (Empréstimo para Políticas de Desenvolvimento, DPLs) (P117244, P111665, P122391, and P126465) approved over the previous three years (two in 2010 and one each in 2011 and 2012) covering areas of fiscal sustainability, education, health, competitiveness, housing, and urban development. The operation also had important synergies with the Strengthening Public Management and Integrated Territorial Development Project (P126735). From an urban mobility perspective, the proposed operation was closely linked to the ongoing Upgrading and Greening the Rio de Janeiro Urban Rail System Project (P111996), a project that is financing critical investments in the SuperVia suburban rail system and providing technical assistance aimed at improving urban transport policy and planning. On the fiscal side, the rationale was that the operation would focus on issues of public financial management (operational risks management, budget planning, and transparency).

8. The program was consistent with the Country Partnership Strategy (CPS) 2012–15,3 which stressed the need for a programmatic approach in World Bank interventions and for stronger policy integration of transport and urban development. This new DPL was an important component to support CPS outcomes across all four key pillars: (a) strengthen public

3 World Bank. 2011. Brazil - Country Partnership Strategy (CPS) for the period FY2012-2015. Report Number 63731. Washington, DC: World Bank.

2

and private investment; (b) improve service delivery to the poor; (c) strengthen regional and territorial development; and (d) support the effective management of natural resources and the environment.

1.3. Original Program Development Objectives (PDO) and Key Indicators

9. The objective of the proposed operation is to assist the Government of Rio de Janeiro in its efforts to enhance public management for the delivery of key public services for vulnerable populations by instituting new policies and regulations to improve the medium-term planning and monitoring of public expenditures (Policy Area 1); the accessibility, quality, and affordability of urban mobility services for the poor (Policy Area 2); and the availability of targeted social services aimed at reducing domestic and gender-based violence (Policy Area 3).4 This objective directly contributes to the poverty reduction and shared prosperity agendas, insofar as financially sustainable and targeted urban mobility and social support services benefiting the poor and other populations at risk, such as women, and enables more equitable access to economic and social opportunities, allowing the benefits of growth to be shared more widely.

10. The key performance indicators (KPIs) are presented in table 1 under each policy area.

Table 1. Operation’s Project Development Objectives (Policy Areas) and Key Performance Indicators

PDO Policy Area 1: Instituting policies and regulations to improve medium-term planning and monitoring of public expendituresPolicy Objective 1.1: Institutionalize sound financial and debt management policies and practices

KPI 1 Maintenance of the number of events (occurrences and non-conformities) in which the operational risk policy procedures were not complied with within already achieved satisfactory levels.

KPI 2 Reduction in the number of times in a year that the actual monthly cash balance has remained below the minimum threshold (set at 1.5 times the payroll amount)

KPI 3 Reduction in the percentage of payments made after 30 daysPolicy Objective 1.2: Strengthen the definition of the budget resource envelope, reinforcing its hard constraint, under a Medium-Term Expenditure Framework (MTEF)

KPI 4 Reduction in the gap between actual and estimated current revenues, measured as a share of the estimated revenues.

KPI 5 Publication in the internet of revenue estimates documents for the 2014 budget.KPI 6 Annual Budget directive issued for the 2015 budget cycle.Policy Objective 1.3: Implement a framework to deal with fiscal commitments originated from Public-Private Partnerships (PPPs)

KPI 7 A comprehensive report including current and prospective long-term commitments and liabilities associated to PPPs is published as scheduled.

KPI 8 All PPP-related bidding documents, contracts and supporting analysis are published on a dedicated website in a timely manner.

PDO Policy Area 2: Instituting policies and regulations to improve the accessibility, quality and affordability of urban mobility services for the poorPolicy Objective 2.1: Foster Modal and regional Integration in urban transport systems

KPI 9Permanent Committee for the implementation of the PDTU, involving relevant government and civil society actors, meets at least four (4) times in its first year of existence and presents strategy for the implementation of at least one (1) priority project under the PDTU.

KPI No transport project outside of the PDTU (i.e. not endorsed by PDTU) will be undertaken by the SRJ.4 The Loan Agreement does not include a definition of the operation’s PDO, but focuses on the prior actions of the program.

3

10KPI 11

At least one (1) multimodal integration project, connecting state and municipal transit systems, is completed.

KPI 12

State prepares guidelines for the development of bicycle-public transit integration projects to be implemented by metropolitan municipalities and at least one (1) of these projects is implemented with the State’s technical assistance.

Policy Objective 2.2: Enhance Performance and Transparency of Inter-Municipal Bus ServicesKPI 13

The concession bidding documents will identify the bus lines to be sectioned or eliminated in order to prioritize integration with mass transit modes.

KPI 14

Inter-municipal bus lines in the RJMR (previously provided under permission regime) are in operation under new concession agreements incorporating performance monitoring.

KPI 15

Service reliability, fleet modernization, and vehicle occupation requirements are properly monitored and enforced on inter-municipal bus concessions in operation

Policy Objective 2.3: Ensure the long-term sustainability and effectiveness of the state’s urban transport subsidy/affordability programs

KPI 16

New operations contract awarded for the cable car system incorporates performance monitoring, availability-based remuneration, and incentives for maximizing ancillary revenue; and results in a reduction in the operational subsidies paid by the state.

PDO Policy Area 3: Instituting policies and regulations to improve the availability of targeted social services aimed at reducing domestic and gender-based violencePolicy Objective 3.1: Operationalize the Maria da Penha Law by leveraging transport infrastructure for improved delivery and targeting of servicesKPI 17

Number of women benefiting from increased access to information and to social and legal services as per the provisions of the “Lei Maria da Penha” within the SuperVia and Teleférico (cable car) systems

KPI 18

Increase in the number of women benefiting from the integration of the SuperVia and Teleférico systems to the ‘Casa da Mulher Brasileira’

11. For the purposes of the Implementation Completion and Results Report (ICR), the operation is evaluated against the PDOs of each Policy Area 1, 2, and 3 and the corresponding KPIs defined for the individual policy objectives5 within each policy area. Together, the three policy-level PDOs provide an equivalent but more detailed operational definition of the operation’s high-level PDO, as described in table 2.

Table 2. Operation’s Overall and Policy-level PDOs

Operation’s Overall PDO Correspondence with Policy Areas PDOsInstituting new policies and regulations to enhance public management

Instituting policies and regulations to improve medium-term planning and monitoring of public expenditures (Policy Area 1)

Instituting new policies and regulations to deliver key public services for vulnerable populations

Instituting policies and regulations to improve the accessibility, quality, and affordability of urban mobility services for the poor (Policy Area 2)Instituting policies and regulations to improve the availability of targeted social services aimed at reducing domestic and gender based violence (Policy Area 3)

1.4. Revised PDO and Key Indicators, and Reasons/Justification

12. The PDO was not revised. KPI 18—Increase in the number of women benefiting from the integration of the SuperVia and Teleférico systems to the ‘Casa da Mulher Brasileira’— was

5 These policy objectives are also referred through the text as ‘policy actions’, so as to more clearly distinguish from the PDOs.

4

revised by the midterm review, following the confirmation that the center ‘Casa da Mulher Brasileira’ would not be built by the Federal Government. The World Bank agreed that the SuperVia and Teleferico Systems would be integrated to one of the state-run centers, which would be built as part of the Via Lilac program.

1.5. Original Policy Areas Supported by the Program (as approved)

13. The operation supported essential policy reforms and institutional strengthening actions in three policy areas: public expenditure, urban mobility service delivery, and gender-based violence. Annex 9 includes a full list of areas and objectives, prior actions, and expected results.

Policy Area 1: Instituting policies and regulations to improve medium-term planning and monitoring of public expenditures

14. Under Policy Area 1, the operation supported a program aimed at institutionalizing a series of sound public fiscal management practices to help consolidate fiscal space and enhance efficiency in Government expenditures—a necessary precondition to maintaining financial sustainability, while continuing to improve service delivery.

15. Policy Area 1 defined three policy objectives: (1.1) Institutionalize sound financial and debt management policies and practices; (1.2) Strengthen the definition of the budget resource envelope, reinforcing its hard constraint, under an MTEF; and (1.3) Implement a framework to deal with fiscal commitments originated from PPPs.

16. Out of the areas supported by this policy, no actions were specifically designed to address the fiscal challenges the state was facing, as they aimed mainly at reinforcing good practices for financial management.

Policy Area 2: Instituting policies and regulations to improve the accessibility, quality, and affordability of urban mobility services for the poor

17. Under Policy Area 2, the operation supported reforms that complemented the Government’s investments in the transport sector to improve the delivery of mobility services. Policy Area 2 defined three policy objectives: (2.1) Foster modal and regional integration in urban transport systems; (2.2) Enhance the performance and transparency of inter-municipal bus services; and (2.3) Ensure the long-term sustainability and effectiveness of the state’s urban transport subsidy/affordability programs.

18. Specifically, the policy actions supported by the operation were aimed at (a) developing an integrated multimodal metropolitan transport system under the umbrella of a master plan that would eventually become the planning tool of a metropolitan transit authority and the basis for civil society involvement; (b) improving the quality and productivity of inter-municipal bus services through modern concessions awarded to competitively selected private operators that are remunerated based on performance indicators measured through real-time data also available to users; and (c) consolidating the affordability/social policies which supported access to transport services for the most vulnerable populations by ensuring that these policies are financially sustainable. These reforms would directly and significantly help the poorest residents

5

of the RJMR by improving the quality of service, reducing travel times, and ultimately improving affordability of public transport.

19. Out of the actions mentioned under this policy area, the program placed great emphasis on improving inter-municipal bus services, which transports the majority of people in the metropolitan region and operates under an outdated licensing framework characterized by substandard monitoring, lack of transparency in operations, lack of performance incentives, and poor integration with other mass transit systems. This framework induces operational inefficiency, with unnecessary competition between modes, and results in higher carbon emissions. Most importantly, it negatively affects the quality of service for users. To respond to these issues, the program supported the implementation of a new concession scheme that included real-time monitoring of performance indicators, replaced open-ended licenses with performance-based contracts, and mandated reorganization of bus lines with a view toward enhancing integration with more efficient mass transit modes.

Policy Area 3: Instituting policies and regulations to improve the availability of targeted social services aimed at reducing domestic and gender-based violence

20. Under Policy Area 3, the operation backed ongoing efforts in the RJMR to improve the delivery of gender-focused services and reduce domestic and gender-based violence by supporting the implementation of the Maria da Penha Law. The idea was to use transport infrastructure as a platform for delivering information and social support services that had been previously constrained by limited resources for deployment. This reform expected to stimulate the mainstreaming of initiatives that maximize the role of the region’s transport infrastructure in addressing a broader social agenda and taking advantage of the high concentration of people using the public transport system for long periods, which provided a valuable space for social marketing. Policy Area 3 defined the policy objective (action) to operationalize the Maria da Penha Law by leveraging transport infrastructure for improved delivery and targeting of gender-specific services.

1.6. Revised Policy Areas

21. Not applicable.

1.7. Other significant changes

22. Recognizing the implementation delays caused by the political climate surrounding the election of the State Governor in December 2014, the SRJ formally asked the World Bank for an extension of the program closing date to continue receiving World Bank support in the critical areas of inter-municipal bus reform and gender-based violence prevention. This allowed the World Bank team to continue to provide technical assistance and to work with the Government on pushing the reform agenda forward. The closing date for the program was extended one year, from January 31, 2015 to January 31, 2016, to allow for continuous support to some of the most critical program elements.

2. Key Factors Affecting Implementation and Outcomes

2.1. Program Performance

6

23. This was a single-tranche operation, which disbursed upon effectiveness. The SRJ met all policy actions required by the Loan Agreement before negotiations. The Board of Directors approved the loan on November 21, 2013. Effectiveness was declared on December 6, 2013.

Table 3. Operations’ Prior Actions

Single Tranche DPL

Amount Expected Release Date Actual Release Date Release

US$500 million December 6, 2013 December 9, 2013 U$500 millionCondition StatusPrior Action #1: The state has adopted the legal framework to implement its Treasury’s operational risk policy, which policy provides for, among others, mandatory disclosure of related documents on the Internet and the state’s submission to yearly external audits of said policy.

Met, as evidenced by the State Decree n. 44.428 issued on October 11, 2013

Prior Action #2: The state has established the legal basis for the use of sound financial and debt management tools, such as budget and financial quotas, minimum and average cash balance, and procedures for making payments to its suppliers.

Met, as evidenced by the State Decree n. 44.429 issued on October 11, 2013

Prior Action #3: The state has adopted the following budget preparation practices to improve its accountability and fiscal discipline: (A) preparation, publication, and regular updating of detailed revenue estimates including subsequent years’ estimates; and (B) issuance of an annual budget directive setting appropriate sectoral budget ceilings which shall be consistent with the medium-term revenue forecasts and targets for public indebtedness.

Met, as evidenced by State Decree n. 44.431 issued on October 11, 2013

Prior Action #4: The state has adopted the following institutional practices to improve its transparency and accountability: (A) regular publication of a comprehensive estimate of fiscal commitments, including explicit and contingent payments from effective and prospective PPP contracts, which estimates shall be updated at least annually together with the Annual Budget Law; and (B) publication of all documents and analysis relating to PPP projects through a specialized website.


Prior Action #5: The state has (A) adopted the Urban Transport Master Plan (PDTU) as the policy framework for the planning and development of an integrated transport system in the RJMR; and (B) established a formal coordination mechanism to enable the state, the academia, civil society institutions, and the RJMR municipal governments to participate in the implementation of the PDTU.


Prior Action #6: The state has taken a first step to initiate the implementation of a policy to physically integrate municipal and state-managed mass transit systems in the RJMR.

Met, as evidenced by the Memorandum of Understanding entered into between the state and SuperVia (rail operator) for the implementation of multimodal integration projects, dated August 22, 2013.

Prior Action #7: The state has adopted a new policy to promote non-motorized transport (NMT) through which it will (A) ensure that secured bicycle parking will be available at all major transit stations in the RJMR; and (B) provide technical assistance to RJMR municipalities in developing high-quality NMT access to major public transport

Met, as evidenced by State Secretariat of Transport (Secretaria de Estado de Transportes, SETRANS) Resolution n. 1.114 issued on

7

stations. October 10, 2013.Condition StatusPrior Action #8: The state has adopted a policy to (A) replace existing permission-based route licenses for inter-municipal bus routes with competitively awarded concessions; (B) institute a performance-based system for managing said concessions; and (C) increase transparency in bus service performance for the public.

Met, as evidenced by the State Decree promulgated on October 11, 2013.

Prior Action #9: The state has adopted a technical evaluation platform for periodic monitoring of the financial viability and cost-effectiveness of all transport programs requiring ongoing subsidies in an effort to ensure their financial sustainability.

Met, as evidenced by Resolution n. 1.113 issued by SETRANS on October 10, 2013.

Prior Action #10: The state has established the Sub-secretariat of Policies for Women (SPM-RJ/SEASDH).

Met, as evidenced by State Decree 44.076 issued on February 20, 2013

Prior Action #11: The state has (A) adopted a policy to promote gender equality in the transport sector specifically focused on the women users of the transport system operated by SuperVia; and (B) committed to implement the ‘Programa SuperVia e Teleférico Lilás.’

Met, as evidenced by the (A) Joint Resolution SEFAZ/SETRANS/SEASDH issued on October 14, 2013; and (B) the allocation of US$15 million (equivalent) to the Subsecretary of Women Policy (Subsecretaria de Políticas para as Mulheres, SPM)-RJ/State Secretariat of Social Welfare and Human Rights (Secretaria de Estado Assistência Social e Direitos Humanos, SEASDH) in the borrower’s 2014 Budget Law Proposal published in the Official Gazette on October 1, 2013

2.2. Major Factors Affecting Implementation

24. The state elections in 2014 and resulting political implications reduced the appetite for reforms and delayed their implementation. The operation was approved under a Governor who resigned his post a few months after program effectiveness in April 2014. Following his resignation, the Vice-Governor assumed and soon started his own campaign for office, winning the elections of October 2014. These changes seem to have considerably affected program implementation. First, the SRJ refrained from moving forward with difficult reforms, such as the new bus concession scheme, because of the political implications associated with its implementation. In addition, following the elections, the new administration appointed new officials in key positions, leading to loss of institutional memory and generating implementation delays. The supervision team tried to manage this situation by immediately engaging with the new administration and insisting on the importance of the reforms. However, new officials, who had not been engaged in the preparation of the program, did not have much ownership over the policy actions.

25. Although the operation benefited from strong analytical underpinnings built as part of the World Bank’s long-term engagement, the urgency of the operation and limited

8

preparation period constrained the depth of the prior actions. Program identification, preparation, and appraisal were carried out over a very short period. Although the chosen policy actions adequately reflected the sectors priorities, the short preparation time did not allow complete implementation of the reforms before approval. On the other hand, the pressing urgency from the Government created an opportunity to include substantive urban transport reforms, in the form of better monitored and more efficient and integrated services, and to address violence against women, a critical issue in Rio de Janeiro.

26. The risks identified for this operation were relevant, in particular, the macroeconomic and fiscal risks. It was recognized that there were considerable risks to the operation because of the state’s fiscal difficulties. The World Bank acknowledged that the state government’s ability to improve its own fiscal situation was limited by federal legislation— particularly legislation governing tax rates and personnel expenditures (including pensions). It was also identified at appraisal that the state’s main sources of revenue, the tax revenue on the Circulation of Goods and Transportation and Communication Services (Imposto sobre Circulação de Mercadorias e Serviços - ICMS) and oil royalties, remained susceptible to the macroeconomic fluctuations in Brazil or abroad. In addition, expenditures were increasing rapidly, both on personnel (because of several ongoing commitments in security and health) and investments (largely linked to the World Cup and the Olympics). Nonetheless, despite the recognition of the abovementioned risks, the extent and severity of the economic and political shocks that plunged Brazil and the SRJ into a deep recession and severely affected the state’s fiscal situation were not expected at appraisal. The risks foreseen on oil royalties from rules changes were removed when a new rule on royalties was passed, creating a sense of stability. However, after the operation became effective, oil royalties did collapse, because of low oil prices and the corruption crisis surrounding Petrobras. The deterioration of the state’s fiscal situation became increasingly apparent soon after the operation became effective (end of 2013), given the rapid expenditure growth that year, and it was deepened significantly in the following years because of the collapse in revenues. This deterioration affected the achievement of some indicators in the fiscal policy area and the overall achievement of the PDO. A review of the assumptions from the fiscal analysis is provided in annex 6.

27. The main political and governance-related risk identified at appraisal was related to public demonstrations and unrest over the quality and price of public services, particularly, the changes in inter-municipal bus services. The mitigation measure defined at appraisal and implemented by the SRJ consisted of extensive consultations in different municipalities. This mitigation measure, despite being necessary, was one of the reasons for the delay in the issuance of the new bus concessions and the achievement of policy action to “Enhance the performance and transparency of inter-municipal bus services” and the PDO of “instituting policies and regulations to improve the accessibility, quality, and affordability of urban mobility services for the poor.”

28. Implementation capacity and institutional commitment risks materialized. For instance, although by appraisal the responsibility for implementing the inter-municipal bus concession scheme was with the Office of the Chief of Staff (Secretaria de Estado da Casa Civil, Casa Civil), this responsibility was later delegated to the bus regulator, Road Transport Department (Departamento de Transportes Rodoviários, DETRO), which demonstrated little capacity and accountability in pursuing the reform. Despite continuous calls by the World Bank

9

team and the intervention of upper management, the Governor’s office did not provide sufficient political support to put pressure on DETRO to deliver. The election cycle, the revision process by the state auditors, and the Olympics further delayed implementation of the concession process. The mitigation measures concerning quality of concession documents, such as specialized technical reviews ensuring that key principles are included, were adequately identified and implemented.

29. Regarding the gender-related policies, a coordination risk and a funding risk were identified at appraisal. On the first one, it was considered that a high level of inclusiveness of the multiple stakeholders (SEASDH, SETRANS, SuperVia, cable car companies, a number of other sectoral state and municipal agencies, the private sector, and the nongovernmental organization community) could hinder the efficiency in implementation, which in fact materialized. However, the project was also affected by low capacity of the implementing agency and the very high turnover of public officials. The low capacity was partially identified at appraisal as lack of adequate funding by the state for the effective implementation. However, the mitigation measure to allocate US$15 million to SPM-RJ-SEASDH in the 2014 State Budget did not materialize completely. To overcome some of these problems, SEASDH proposed an outsourcing of some components of the program to United Nations Office for Project Services (UNOPS), a United Nations agency appointed as program manager for implementing some of the actions. Nonetheless, program execution continued to face challenges because some of the activities, such as the construction of childcare and service centers for women, required institutional coordination to find available lots and secure funding for operations and maintenance. Institutional arrangements to operate these facilities took longer than expected, and relative progress was seen only after UNOPS was appointed. Until now, only a budget of approximately US$4 million was so far allocated to UNOPS.

30. The design of the operation under the three policy areas reflected different strategies. The policy objectives in the fiscal area were mostly focused on operational improvement, not on fiscal adjustment, which was the crux of the fiscal difficulties facing Rio. However, given the political cycle and the upcoming World Cup and Olympics, there was no political space for the Government to introduce fiscal adjustment measures and the policy area PDO was realistic and designed in a way that the operation could be ultimately held accountable, responding to the borrower’s needs. The three policy actions were modest and substantially linked to the project objective for this policy area, but the indicators failed to reflect the policy actions, which affected the program outcomes. The project objectives and policy actions for the transport policy area encompassed the relevant strategic reforms and measures to address financial bottlenecks that would positively improve service delivery for the poor, and the indicators reflected the PDO for this policy area. However, the elections cycle and the World Cup and Olympics affected the commitment toward the implementation of these reforms. Despite being ambitious, this PDO was realistic given the state of the World Bank’s engagement with the SRJ. The project objective and policy actions for the gender policy area were strategic and aligned with a bottleneck of operationalizing the Maria da Penha Law that the SRJ was failing to comply with, but the novelty in approach, low capacity of the implementation agency that was finally appointed to implement the program, and lack of commitment from the SRJ negatively affected the results. The adequacy on the design and the relevance of the policy actions and indicators to achieve each policy area PDO are assessed in detail in annex 10.

10

31. The number of agencies involved without a leading office slowed down the implementation of the program. The institutional arrangement design defined four agencies to implement the program—Casa Civil, SETRANS, DETRO, SEFAZ, and SEASDH—which generated complexities in managing the implementation. Starting in May 2014, the Casa Civil created a task force to supervise the implementation of the program and help coordinate execution. This task force was headed by the Sub-Secretary of International Relations, in charge of financing programs with multilaterals. This decision had positive impacts and implementation showed localized signs of recuperation. This support, however, was not sufficient to achieve the necessary reforms.

2.3. Monitoring and Evaluation (M&E) Design, Implementation and Utilization

32. M&E design. The M&E framework had 18 indicators to measure the achievement of seven policy objectives in three policy areas, which were defined as the PDOs. The relevance of the M&E design is evaluated against the policy area PDOs, and more details are presented in annex 10.

33. Indicators of the fiscal pillar, in general, presented low relevance to measure the achievement and progress of the PDO. For example, the indicators under the policy action “Institutionalize sound financial and debt management policies and practices” were not a good measure of the improvement in the medium-term planning and monitoring of public expenditures, even within the framework of operational risk instituted by the PDO. The indicators under the policy action “Strengthen the definition of the budget resource envelope, reinforcing its hard constraint, under a Medium-Term Expenditure Framework (MTEF)” were procedural and did not allow for actually improving the medium-term planning and monitoring of public expenditures, as the hard constraints were not enforced. The indicators under the “Implement a framework to deal with fiscal commitments originated from PPPs,” although worthwhile for its efforts to improve transparency, did not actually work toward improving the framework, and thus, had low relevance in improving the medium-term planning and monitoring of public expenditures. The indicators chosen were also very timid, presenting at best, marginal improvements relative to the baseline; for example, “maintain non-conformities at already achieved satisfactory level.” Moreover, the informational asymmetry between the World Bank and the client posed difficulties for the World Bank to assess the achievement of the indicators as many of them were poorly defined and the definitions used by the client diverged from those in the program document.6

34. For the transport pillar, indicators were in general highly connected to the PDO. The indicators for policy actions of fostering modal and regional integration and enhancing performance of inter-municipal transport systems were directly related to the policy objective to improve accessibility, quality, and affordability of urban mobility services for the poor, despite sometimes needing to be measured for a longer time frame than the operation to be relevant,

6 For example, in the case of “reduction in the number of times in a year that the actual monthly cash balance has remained below the minimum threshold (set at 1.5 times the payroll amount).” Neither ‘cash balance’ nor ‘payroll amount’ are further defined. Further, the Government assessed its performance against their internally used threshold of ‘one-time payroll’ rather than the stated “1.5 times the payroll amount”. Additionally, the indicator on issuance of the Annual Budget directive did not define if it should be disseminated internally or externally – while the Bank assessed that the dissemination should be externally, the borrower issued only internally.

11

such as “no transport project outside the PDTU.” The indicator of “new operations contract awarded for the cable car system,” under the policy action of “ensuring the long-term sustainability and effectiveness of the state’s urban transport subsidy/affordability programs,” was insufficient to reflect the improvement of the accessibility, quality, and affordability of urban mobility services for the poor, but was adequate based on the short time frame of the operation. Considering the relative importance of the Integrated Transport Card (Bilhete Único Intermunicipal, BUI) program compared to the cable car service, it would have been sensible to include an indicator related to the BUI to measure progress in this area.

35. Finally, indicators for the gender pillar were highly relevant to improve the availability of targeted social services aimed at reducing domestic and gender-based violence. However, the indicator on “number of women benefiting from increased access to information and to social and legal services” was difficult to measure because of the lack of integration between the services databases and the indicator “increase in the number of women benefiting from the integration of the SuperVia and Teleférico systems to the Casa da Mulher Brasileira” depended on the implementation of service centers that were not concluded.

36. M&E implementation. Despite deficiencies in the results framework, throughout implementation, the Casa Civil has kept track of the indicators for the policy actions supported by the program. The team has also conducted periodic assessment of the state of compliance with the targets and maintained a document that was frequently updated to report on changes in indicators’ values. For the fiscal area and transport policy areas, the indicators were easy to measure as they reflected ongoing work that the agencies were developing. For the gender policy area, on the other hand, during implementation, it became clear that measuring the actual access to social and legal services as a result of project activities (kiosks) was not easy to obtain, given the current lack of integration on the databases. Therefore, the indicator on “higher access to social and legal services” was measured instead with the “number of women accessing the kiosks with Lei Maria da Penha”, which likely presents a correlation. The indicator “increase in the number of women benefiting from the integration of the SuperVia and Teleférico systems to the Casa da Mulher Brasileira” was replaced with access to any state-run center and then, finally implemented as the access to the itinerant buses, a mobile service center known as the ‘Lilac Bus.’

37. M&E utilization. The indicators were used to inform decision making and resource allocation for the World Bank supervision; in particular, the team leveraged resources from both Development Impact Evaluation (DIME) and The Umbrella Facility for Gender Equality (UFGE) to provide guidance to some of the components of the project. The indicators for measuring results in the gender policy area that were feasible to calculate mostly concerned outputs (number of women who access an information) and did not allow to measure the actual impact of the policy reform on accessing social services or in improving the lives of women. The discussion that the World Bank team promoted around how to measure the gender indicators has brought awareness to the SEASDH team on the need to improve its monitoring framework to measure outcome results from policy actions, despite the methodological difficulties.

12

3. Assessment of Outcomes

3.1. Relevance of Objectives, Design and Implementation

38. Relevance of objectives rating: Modest. The development objective of Policy Area 1, of “instituting policies and regulations to improve medium-term planning and monitoring of public expenditures,” failed to address the series of fiscal challenges the state was already facing at appraisal. This objective did not sufficiently support fiscal discipline measures to increase the fiscal space for public investment, although it worked to improve predictability and transparency of budgetary resources, both objectives of the CPS 2012–15. The development objective of Policy Area 2, of “instituting policies and regulations to improve the accessibility, quality, and affordability of urban mobility services for the poor,” was and remains relevant to (a) improve efficiency of private investments; (b) improve the services for low-income households; (c) improve environmental sustainability of investments; and (d) promote economic development according to the CPS 2012–15. The development objective of Policy Area 3, of “instituting policies and regulations to improve the availability of targeted social services aimed at reducing domestic and gender-based violence” reflected the borrower’s priority and the CPS 2012–15 to improve the services for low-income households with increasing attention to supporting efforts to promote gender inclusion.

39. Relevance of design rating: Modest. The policy actions of Policy Area 1 were only modestly consistent with the PDO for that policy area, with the exception of the policy action “implement a framework to deal with fiscal commitments originated from PPPs,” which was consistent with the PDO despite the low relevance of indicators. The policy actions for Policy Area 2 were highly consistent with the PDO for that policy area, despite some of the indicators not being ambitious enough to reflect the full extent of the policy actions. As an example, the cable car contract was not a sufficient representation of “ensuring the long-term sustainability and effectiveness of the state urban transport/affordability programs.” The advances in the BUI, on the other hand, were monitored and represented the policy action more adequately. The policy action and activities for Policy Area 3 did not allow for supporting the “institution of policies and regulations,” but were directly linked to the “improvement of the availability of targeted social services aimed at reducing domestic and gender-based violence.” Therefore, the policy action was substantially relevant and supported the achievement of a significant part of the policy area objective.

40. Relevance of implementation rating: High. Implementation assistance was responsive to changes in counterpart needs; the project was restructured once to adapt to delays in the implementation and the related gender trust fund, DIME, and the Central Management Unit actively supported the implementation of project components. The extension was deemed adequate because, despite inefficiencies, the Government was showing growing commitment toward the implementation of some of the activities, progressively achieving the steps required to reach the results, and longer support from the World Bank team was expected to improve the achievement of the PDO.

13

3.2 Achievement of Program Development Objectives

41. The SRJ has not achieved the PDOs and the challenges persist. For Policy Area 1, the improvement of medium-term planning and monitoring of public expenditures can be considered negligible, given the context of fiscal emergency and the state’s default. The fact that some fiscal indicators were met reinforces the deficiencies of the results framework. For Policy Area 2, despite the stronger results framework, the objective to improve the accessibility, quality, and affordability of urban mobility services for the poor was only modestly achieved because some of the most important actions, such as the new bus concessions and the long-term sustainability and effectiveness of the state’s urban transport subsidy/affordability program, were not achieved. For Policy Area 3, the improvement of the availability of targeted social services aimed at reducing domestic and gender-based violence was modest, given the deficiencies of the results framework and the fact that only one indicator was partially achieved. More detail is presented in the following paragraphs, analyzing the achievement of each indicator toward the PDOs under each policy action.


Policy Objective 1.1: Institutionalize sound financial and debt management policies and practices

42. Although the indicators under Policy Action 1.1 were partially met, the SRJ is effectively bankrupt as a result of overspending and the severity of the current economic crisis. Improvements to internal controls could not, by their nature, offset the fall in revenues that was driven by the downturn of the economy and, in particular, the oil sector. In the first part of the program, there was a downward trend in both the number of noncompliance occurrences and the number of non-conformities. These positive outcomes were a result of more effective monitoring procedures and standard requirements. The cash balance remained above the threshold for most of 2014 but fell below it in the last two months of the year. In 2015, the cash balance experienced a similar pattern but ended the year below the state’s minimum requirement, forcing delayed salary payments.7 Data from 2016 exhibits a cash balance below the threshold in seven out of nine months (data from January to September 2016), which evidences an even more deteriorated situation after the closing date of the project. Additionally, the reduction in the percentage of payments made after 30 days did not meet the expected target, demonstrating how the severe economic crisis in Brazil and in the SRJ in particular has negatively affected the state finances, leaving it liquidity-constrained and unable to maintain its payment schedule.

43. In 2016, the SRJ has defaulted on its debt obligations and has insufficient resources to meet personnel and pension commitments. Even though the SRJ has received debt relief from the Federal Government and emergency financing before the 2016 Olympics, the state has entered selective default after missing loan payments to the Inter-American Development Bank

7 The cash balance has a strong cyclical tendency, with the balance at the beginning of the year typically more than double that at the year end. Hence, the critical months are the ones late in the year, when cash balances dropped below the threshold. However, since the target allows for two months below the threshold, the target can be considered narrowly achieved.

14

(Banco Interamericano de Desenvolvimento).8 The state is several months behind schedule in salary payments, pensions, and payments to vendors. The Judiciary has confiscated resources in the state accounts (including those where the World Bank loans are disbursed) to honor the state’s obligations. The state has surpassed the limit of the Fiscal Responsibility Law of 60 percent of net revenues in 2015 spent on personnel and the debt limit under the Fiscal Responsibility Law (200 percent of revenues). Rising personnel costs are at the core of the fiscal crisis as the SRJ has been the state with the highest growth of personnel expenses from 2009 to 2014, with the additional burden of more than half of this growth being for inactive personnel.9

44. A package of adjustment measures to address the fiscal crisis has been sent to the Legislative Assembly but was refused. The SRJ government has recently proposed an ambitious fiscal adjustment package which includes tax increases (estimated to generate US$1.4 billion a year), personnel cuts and additional pension contributions (US$3.4 billion), as well as spending cuts on other programs (US$180 million). However, these measures require legislative approval and have encountered severe resistance from civil society and the affected stakeholders and were not approved. Even if some of these measures are eventually approved, given the ongoing recession and structural pension deficit, the fiscal adjustment needed in the SRJ would take a number of years. Further financial assistance from the Federal Government or federal reforms to states’ spending obligations (especially pension reform) may be needed to resolve the SRJ’s insolvency.

Policy Objective 1.2: Strengthen the definition of the budget resource envelope, reinforcing its hard constraint, under a Medium-Term Expenditure Framework (MTEF).

45. Three results indicators were associated under this policy action and two were achieved, but the positive results have been overwhelmed by the fiscal crisis. The authorities have confirmed their commitment to strengthen multiyear planning and budgeting. Improved budget preparation practices introduced under the program (preparation, publication, regular updating of detailed medium-term revenue estimates, and the issuance of an annual budget directive setting appropriate sectorial budget ceilings) have been maintained. Revenue forecasts have been inaccurate in recent years (target not achieved), which is unsurprising, given the stronger than expected contraction in revenues in the current crisis.10 In the bigger picture, these policies, which are mostly operational, did not address the binding fiscal constraints the state faces and any progress achieved will be moot until an effective fiscal reform package is in place and the state is able to reconcile its revenues and spending commitments and meet obligations when due.

46. The SEFAZ, through an agreement signed with the State University of Rio de Janeiro, developed and implemented its own statistical models for the improvement of the State Treasury revenue projections. According to the reports, the first indicator, “reduction in the gap between actual and estimated current revenues, measured as a share of the estimated

8 See Standard & Poor’s ratings report of September 19, 2016.9 https://www.tesouro.fazenda.gov.br/documents/10180/337275/Boletim+entes+2016/e4428e40-5256-40f9-91f6-ee8dcb377a25.10 The Government reported a different indicator, comparing the estimated revenue to an adjusted measure of revenue. However, measuring against actual revenue (in accordance with the indicator), revenues for 2015 were overestimated by 5.5 percent (according to government measurement) and 11.9 percent according to World Bank measurement (see annex 7).

15

https://www.tesouro.fazenda.gov.br/documents/10180/337275/Boletim+entes+2016/e4428e40-5256-40f9-91f6-ee8dcb377a25

https://www.tesouro.fazenda.gov.br/documents/10180/337275/Boletim+entes+2016/e4428e40-5256-40f9-91f6-ee8dcb377a25

revenues,” was not achieved, reaching 2.5 percent by the end of 2014 and 5.5 percent by the end of 2015. Recalculating public available data, the gaps are even more pronounced, as shown in annex 7 (6.1 percent in 2014 and 11.9 percent in 2015). However, this indicator does not reflect what was agreed in the Project Appraisal Document (PAD); the team recalculated the indicator sent by SEFAZ based on the data provided and found that it was in fact not achieved. Additional information is presented in annex 7.

47. The second objective indicator of “publication in the Internet of revenue estimates documents for the 2014 budget” was achieved. The state published the 2014 revenue estimates in 2013. In 2014, the state prepared and published four technical notes on the review of the revenue estimates. Additionally, the state published the revenue estimates for 2015 and five technical notes on the review of the 2015 revenue estimates.11 The web publication boosted the transparency in the definition of the budget estimates. Despite the authorities’ commitment to adopt and release an MTEF, they have argued that the publication of medium-term estimates, as established in the program, created undesired rigidity for what should be only a reference point. Therefore, SEFAZ continues to use an MTEF but has made adjustments to incorporate ranges for the expenditure commitments that can be adjusted depending on economic factors. This information is publicly available.

48. The third objective indicator of “Annual Budget directive issued for the 2015 budget cycle” was also achieved. In 2014 and 2015, SEFAZ prepared the Control and Tax Model Management reports, and disseminated it within the Planning Department. To prepare these reports, SEFAZ used the medium-term fiscal model. The model was developed in partnership with IBRD over the years 2012 and 2013.

Policy Objective 1.3: Implement a framework to deal with fiscal commitments originated from Public-Private Partnerships

49. The two indicators under this policy action were achieved. As a result of the framework, the public administration promoted greater transparency and disclosure of their actions. SEFAZ established a methodology to identify the fiscal commitments as a result of PPPs, which includes the analysis, quantification, and qualification of the impacts of the explicit financial obligations, namely those defined in the PPP contract. A comprehensive report including current and prospective long-term commitments and liabilities associated to PPPs and all PPP-related bidding documents, contracts, and supporting analyses is published on the SEFAZ website. However, the weak link of the indicators to the policy area objective and the fact that very few projects are now under consideration, demonstrates that the PPP agenda would benefit from a larger government engagement to move forward.


Policy Objective 2.1: Foster modal and regional integration in urban transport systems

11 The reports are published on the web of transparency of SEFAZ: http://www.transparencia.rj.gov.br/transparencia/faces/sitios-transparencia-navigation/menu_sitios_analiseContas/Analise-Notas?_adf.ctrl-state=12irvpwc7e_4&&_afrLoop=493433715103000

16

50. Three indicators under this policy action were achieved. This policy action represented a key opportunity to implement essential structural changes for a more efficient mobility: reinforcing and institutionalizing the emerging culture of coordination between municipal and state governments, solidifying the practice of conducting investment planning informed by data and analysis contained in the Master Plan on Urban Transport in the RJMR (Plano Diretor de Transporte Urbano da Região Metropolitana do Rio de Janeiro, PDTU) and setting the stage for enhanced regional integration of transport systems, including non-motorized modes. These are pre-requisites for improving accessibility, quality, and affordability of the mobility services for the poor.

51. The first indicator was achieved. Despite the delays in completing the PDTU, financed by the related World Bank operation (P111996) and delivered in 2015, the Permanent Committee for the implementation of the PDTU, involving relevant government and civil society actors,12 met four times in 2015 and presented a strategy for the implementation of priority projects in the area of urban transport integration under the PDTU. In addition to presenting and discussing the PDTU in the Permanent Committee, the state has also presented this plan at the Legislative Assembly and at the commercial association of Rio. The PDTU serves as a basis for the municipal mobility plans, which are required by law13 and are now being prepared with financing from another World Bank loan. The second indicator was also achieved as the STR accomplished its commitment to not undertake transport projects outside of the PDTU. The STR has communicated to the World Bank the need for an agreement so that all multilateral banks follow this commitment. Although these indicators might seem trivial to achieve, the meetings precluded the development (update) of the strategic plan, an important tool for interinstitutional governance. Despite this achievement, there is a need to strengthen this mechanism of coordination with an institutional reform that implements a strong metropolitan agency.

52. As an evidence of the coordination between the municipal and state governments, the third indicator, of completing at least one multimodal integration project, connecting the state and municipal transit systems, was surpassed as two projects were delivered—both Maracanã and Madureira stations were completed. Maracanã Station, inaugurated in February 2014, integrates SuperVia urban train and subway systems. The main beneficiaries of this integration are the passengers of poor areas of the periphery of Rio (Baixada Fluminense, northern, southern and western areas). The lines integrated in Maracanã are SuperVia branches Deodoro, Santa Cruz, Japeri, Belford Roxo, and Saracuruna and Metro Line 2. Additionally, the works included the improvement of the access to the State University of Rio de Janeiro, the community of Mangueira, and the Maracanã Stadium. In addition to the Maracanã station, in the Madureira Station, the state (together with the MRJ) improved the integration between SuperVia and the TransCarioca BRT system. Madureira station is a major railway station of the Rio suburb, integrated with the Jacarepagua region of Barra da Tijuca and the south zone.

53. Finally, the state prepared the Guidelines of Non-motorized Transport (Transporte não Motorizado, NMT),14 the fourth indicator, which sets up the best practices for the 12 Meetings for the PDTU included SETRANS, DETRO, AGETRANSP, Municipal Secretariat of Transport, Metropolitan Transport Agency, civil society groups, sector representatives, and universities.13 Mobility Law # 12.587, from January 3, 2012, institutes the national policy instruments for urban development with the objective of integrating transport modes and improving accessibility and mobility of individuals and goods. 14 Manual de Referências de Projetos de Transporte Não Motorizado, December 2013.

17

development of integration projects of bicycle and public transport, to be implemented by metropolitan municipalities. Some of these projects were implemented with the state’s technical assistance component, such as the 40 bicycle racks integrated with high capacity transport in Cocota Station (Ferries). Other actions that the NMT guidelines influenced are (a) “Pedalando na Escola (Pedaling to School),” a project in partnership with the State Department of Education that aims to educate teachers and students about the benefits of cycling, associated with actions for a safer traffic; and (b) an agreement with MetrôRio and SuperVia to authorize bicycle boarding in their vehicles. In addition, in a related action outside the original scope of the program, with the support of the ongoing investment operation (P111996), the state prepared a new business plan to enhance bicycle-rail integration, part of the NMT Plan for the state. This business plan will guide the future operations of the bicycle parking facilities in the SuperVia system.

Policy Objective 2.2: Enhance performance and transparency of inter-municipal bus services

54. The full expected results of this policy reform were not achieved because of the delays in the development of the bidding of inter-municipal bus services and the non-issuance of the new concessions. The main expected result for this policy action was that the mobility conditions of people using inter-municipal buses would improve because of the introduction of modern services incorporating performance-monitoring and enhanced integration with mass transit under new open and transparent concessions with private operators. These would in turn improve accessibility, quality, and affordability of the mobility services for the poor.15

55. The SRJ, through SETRANS and DETRO, developed a model for a new inter-municipal bus concession scheme based on the reorganization of bus lines under different service areas. This proposed model fundamentally changed the previous paradigm by introducing competition for the market and not in the market (bidders compete for service areas and not for passengers in overlapping lines). The World Bank provided technical comments to improve the modeling of the concessions and the terms of the proposed Concession Agreement, seeking to align them with international best practices. The SRJ adopted some of the World Bank recommendations.

56. In terms of integration with the mass transit modes, the state identified 90 bus lines to be rationalized, resulting in a partial achievement of the first target indicator. However, instead of including the elimination of these bus lines in the new concessions documents, the state has opted to make gradual adjustments. The concession documents foresee a mechanism that allows the Government to eliminate or restructure certain lines as the capacity of mass transit modes increase. According to the SRJ, with this mechanism, reform will be gradual and rationalization of bus routes will occur in a second phase, after evaluation of their operational plans from the real-time data obtained with enhanced monitoring systems. This decision was taken to avoid gaps in operations and to maintain the economic attractiveness of the concessions, following a similar model used for the municipal concessions. However, the World Bank strongly suggested that the new concessions already included some of these rationalizations from contract signing, arguing this post adjustment will be difficult to implement as the operators would have little incentive to lose the fare collected from these passengers.15 Some of the main characteristics of the mobility patterns and transport infrastructure of the metropolitan area of SRJ are presented in annex 8.

18

57. The second indicator “inter-municipal bus lines in the RJMR (previously provided under permission regime) are in operation under new concession agreements incorporating performance monitoring” was not achieved. The reasons given by the state for the delays were the need for more extensive public consultation, administrative changes, and political considerations. After delays relative to the elections, the SRJ manifested that it did not want to provide significant changes in the transport service, given the proximity of Olympics and Paralympic games in the summer of 2016. The new bidding documents are under revision, after the state auditor has issued comments and suggestions. Public hearings also took longer than expected but are completed. The bid launch is now expected for March 2017. Despite the ready to implement concession documents, the STR might continue to face political challenges to issue the concession bid.

58. Despite not still being in place, the concession contracts include performance-based monitoring of buses that will increase government capacity to guarantee service quality and reliability and to identify opportunities for network operational adjustments, ultimately improving contract management. DETRO has also finalized the procurement process of software licenses for a real-time monitoring system that allows for monitoring service reliability, fleet modernization, and vehicle occupation requirements, and DETRO has just started to receive this monitoring data. Therefore, the third indicator was partially achieved, but the World Bank team has argued that it is important to use the monitoring system to ensure service delivery, not only enforcing current penalties but also assessing the system performance to allow for better rationalization and integration with other trunk systems.

59. DETRO has developed a smartphone application to allow better monitoring of service quality from the passengers’ perspective; passengers are able to report any non-conformity and send service feedback. The application response rate was, however, so far very low, but according to DETRO, comments and complaints through messaging system are being used to improve operations. DETRO has also implemented new face recognition technology in 1,200 buses to reduce BUI fraud.16 The performance-based contract and real-time monitoring/face recognition technology will allow for better enforcing the service (reliability, comfort, and crowdedness), while the application and messaging services might provide valuable data on service quality from the user perspective.

Policy Objective 2.3: Ensure the long-term sustainability and effectiveness of the state’s urban transport subsidy/affordability program

60. The state made some progress under Policy Action 2.3 but the main indicator was not achieved. Two projects were developed to ensure that subsidized programs remain fiscally affordable and sustainable: Cable car (cable car service) and BUI (Integrated Transport Card).

61. First, regarding the cable car service, a new operation contract awarded for the cable car system in March 2016 incorporated performance monitoring, availability-based remuneration, and incentives for maximizing ancillary revenue. The new contract value resulted in a reduction in the subsidy by 14 percent,17 above the target set in the results indicator (5 percent). Nonetheless, despite the reduction in the subsidy achieved by the new contractual

16 Fraud in the BUI system occurs mainly by sharing the fare card among users. The subsidy policy includes paying the amount above the tariff of R$6.50. This value has been updated to R$8.0 in a recent voting at the state assembly.

19

scheme, the sustainability of this result is in question. In October 2016, the private contractor managing the system announced the cancellation of the contract. Because of the fiscal crisis, the state had not paid the contractor and the operation was suspended, and the indicator was not achieved.

62. Second, SETRANS showed a strong commitment toward ensuring the financial sustainability of the BUI system and presented several actions aimed at optimizing this subsidy policy, reducing the fiscal burden to the state without affecting the poor. The SRJ contracted a technical audit of the Integrated Fare Policy, as mandated by the Prior Action Decree and it later strengthened this technical audit by increasing its scope and mandating for the analyses to be based on primary data, rather than on the treated summarized data provided by bus operators, increasing data reliability. Other actions included (a) the nomination of a high-level manager to oversee the program and fight against fraud; (b) better access to the data provided by the program administrator; (c) measures to reduce fraud, including better enforcement and introduction of face recognition cameras in 1,200 buses; and (d) the elimination of certain subsidy programs such as the elimination of a subsidy that existed on top of the Bilhete Unico.18 The implemented actions are consistent with the program objectives but they were only implemented when it became evident that the subsidy program was not financially sustainable. The World Bank had long argued for an optimization of the subsidy system, targeting the populations that needed subsidies the most beyond the policy of the ‘vale-transporte,’19, and incentivizing a more efficient use of the transportation system by providing a greater subsidy for trips that make use of mass transit modes, for which the SRJ makes great investments. In addition, the World Bank had also offered support to SETRANS, through DIME, in an effort to design an evaluation of the BUI policy to improve its outcomes and financial sustainability. However, the World Bank has not been granted required access to the data for this activity.


Policy Objective 3.1: Operationalize the Maria da Penha Law by leveraging transport infrastructure for improved delivery and targeting of services

63. The SRJ implemented an innovative strategy utilizing the existing SuperVia transport infrastructure and network as a means to improve the delivery of gender-focused services, at reduced fixed and operating costs. The objective was to increase access to information and to social and legal services according to the provisions of the Maria da Penha Law within the SuperVia and Teleférico systems, and to promote the integration of the SuperVia and Teleférico systems to the ‘Casa da Mulher Brasileira’. Insufficient government capacity, changes in management, and cuts in secretariat budget have delayed the implementation of the Via Lilac 17 The operation contract was awarded for R$2.7 million per month for a period of 36 months. If the number of passengers does not change (7,500 passengers per day), the subsidy per passenger per travel would be US$12. The original subsidy was R$14, therefore, the subsidy reduction is 14 percent. 18 By the ICR mission in October 2016, the Secretary of Transport has also communicated that a monthly cap on the BUI will be enforced. This maximum cap minimizes bus users’ fraud, limiting the ability of multiple users to use the same card.19 The vale-transporte is a targeted subsidy, imposed by the federal legislation, by which every employerwith more than nine employees must pay the difference between home-to-work transport costs of employees and 6 percent of their salaries. The employer can then deduct these costs from its income tax.

20

Program. The prior measure to allocate US$15 million to SPM-RJ-SEASDH in the 2014 State Budget did not materialize completely (only a budget of approximately R$4 million was so far allocated to UNOPS). In general, budget cuts have prevented the allocation of appropriate resources to implement the program and might have intensified the delays. Despite these delays, the indicators under this policy action were partially achieved.

64. First, regarding the access to information and to social and legal services according to the provisions of the Maria da Penha Law, within the SuperVia and Teleférico systems, a total of 40 information kiosks, out of 93, have been installed in SuperVia and ferry stations. These information kiosks (referred to as ‘Totems’) are interactive computer terminals installed inside the stations, which provide information about gender-based violence, women’s rights, and support services. Thirty new kiosks were expected to be implemented in the upcoming months. By January 2016, 53,732 had accessed the kiosks, allowing the state to partially meet the target of 50,000 for the first Indicator as there was an increase in access to information. It is unclear if the increase in information generated an actual increase in access to services as the databases for these centers are not integrated, making it impossible to identify what prompted the women to look for support. The World Bank has provided technical assistance to build up the capacity of SEASDH to find the best locations for the kiosks and provided guidance on how kiosk content could be improved to achieve better results. SEASDH has manifested its commitment to adjust the content of the kiosks to reflect the suggestions made by the World Bank team before the remaining 30 kiosks are installed.

65. Second, concerning the construction/renovation of the three gender-based violence prevention service centers (Casa Lilas) and four municipal day care centers attached to railway stations, the works had significant delays and by May 2015, it became clear that they would not be ready by the program closing date. SEASDH decided then to establish technical cooperation with UNOPS,20 which became responsible for implementing those centers. The agreement was signed in December 2015 and the funds (approximately US$3.2 million) were passed to UNOPS in April 2016. UNOPS has worked with SEASDH to reassess the preselected locations for the facilities, given the low commitment from the municipalities which generated great delays. Following this phase, UNOPS worked to improve project design and issue bidding documents. At this point, civil works for the construction of the three Casa Lilas’ and four day care centers have still not been initiated. Construction for some of the facilities was estimated to start in November 2016. There is a risk that the implementation of all seven facilities is affected by budget cuts as SEASDH has been identified as a cost reduction target.

66. Finally, the second project indicator “Increase in the number of women benefiting from the integration of the SuperVia and Teleférico systems to the ‘Casa da Mulher Brasileira’” was not achieved. However, SEASDH provided access to services to women with the ‘Lilac Bus’. This itinerant bus reached the poorest neighborhoods of 20 municipalities in the metropolitan area, serving more than 3,000 women with general services and information on violence prevention and women’s rights. SEASDH intends to continue this initiative and expand operations to other municipalities with difficult access for women to services. This initiative was a creative and effective way of reaching the desired audiences and the World Bank team encouraged this initiative to continue.20 UNOPS provides support to the Brazilian Government and United Nations agencies in the form of sustainable project management, procurement, and infrastructure services. https://www.unops.org.

21

3.3. Justification of Overall Outcome RatingRating: Unsatisfactory

67. The operation was rated Unsatisfactory despite some of the indicators being achieved, because there are major shortcomings in the operation’s achievement of its objectives. The relevance of the objectives is Modest, the relevance of the design is Modest, and the relevance of the implementation is High.

4. Assessment of Risk to Development Outcome Rating: High

68. Policy Area 1. The demonstrated progress on financial and debt management improves the SRJ’s capacity to foresee the impact of the financial uncertainties, but does not ensure the necessary fiscal adjustments and will be insufficient to put the SRJ on a path to fiscal sustainability. The continued fiscal crisis and related political tension and unrest present a paramount risk to development outcomes. The implementation of a framework to deal with fiscal commitments originated from PPPs in the broad sense, and together with a strengthened management system, increased likelihood of service sustainability and increased transparency. However, publication of fiscal commitments does not prevent the Government from failing to make payments, as demonstrated by the recent suspension of the cable car contract. While there are important fiscal reform measures under consideration (both at the state and federal level), their approval and effectiveness in resolving the crisis are far from ensured. If fiscal sustainability was achieved, the framework supported by this project could provide the state with the necessary tools to inform decision making for the medium-term monitoring of fiscal expenditures.

69. Policy Area 2. A change in the political support together with the strong influence of stakeholders may put in risk the sustainability of this policy area. The efficiency of the urban transport system (reducing travel times, saving energy and cost) is closely related to the success of the future integration of inter-municipal bus and mass transportation. There is a need to rationalize inter-urban bus routes and prioritize the use of more efficient trunk services such as SuperVia and Metro. By the time of this ICR, the state had yet to implement the new bus concession scheme, including route reorganization. On the other hand, the fiscal constraints in the SRJ, pressed for changes to subsidies’ policies of the BUI and a cap to the monthly BUI could minimize fraud and generate savings. If not carefully implemented, these changes might affect the users’ affordability and mobility, in particular, for those who use interurban buses, which are heavily subsidized.

70. Policy Area 3. The fiscal crisis in Brazil and the SRJ is generating budget cuts, creating a risk that the Via Lilac Program might be discontinued. Additionally, because the operations and maintenance of the service centers and childcare facilities will be under the responsibility of the municipalities where they are located, and these municipalities are also facing fiscal pressures, there is a risk that those are also discontinued. The World Bank team advised the state to prioritize service delivery and monitoring of the system for a few selected facilities instead of initiating construction of several new centers. With the support of the Umbrella Facility for Gender Equality trust fund (TA-P157959) and the DIME team, the World Bank suggested ways

22

to integrate databases from the different institutions that receive data about violence against women and to establish a trackable results chain to assess the results of these policies.

71. The fiscal difficulties the SRJ is facing offers an opportunity for the necessary changes, despite the projections that the situation should remain critical for a long period. The necessary reforms might also be delayed by the institutional reorganization occurring as a result of the fiscal crisis, such as the incorporation of SEASDH into the Secretary of Health and the incorporation of Secretary of Works, Secretary of Economic Development, Energy, Industry, and Services, and Secretary of Agriculture and Livestock according to the State Decree 45.809 from November 2016.

5. Assessment of Bank and Borrower Performance

5.1. Bank Performance

(a) Bank Performance in Ensuring Quality at Entry Rating: Unsatisfactory

72. The proposed operation for this policy was a result of a long engagement, and the World Bank undertook a risky operation to propose some critical reforms. However, the World Bank did not recognize the severity of the imminent fiscal crisis and the difficulties in approving these reforms under the time frame of the operation. In Policy Area 1, components could have been targeted toward measures of fiscal discipline instead of financial management, but the short period for preparation was a challenge for requiring meaningful prior actions. Therefore, actions and results’ indicators lacked ambition and relevance to the fiscal challenges the state was facing. In Policy Area 2, some of the program objectives were ambitious, and necessary to achieve the bottlenecks of the transport sector. However, the World Bank was not able to propose mitigation measures to face the set of barriers surrounding the bus reforms and the change in commitment from the Government and had little leverage after disbursement. In Policy Area 3, the World Bank established a relevant results framework, but has failed to recognize the risks of low commitment to mitigate the low capacity of agency and changes in personnel.

(b) Quality of SupervisionRating: Satisfactory

73. The World Bank worked closely with the Government and the implementing agencies to implement the program. The World Bank supervised the program diligently with the required expertise, with intense support from the Central Management Unit. Six Implementation Status and Results Reports were prepared over a period of two years, and the ratings were candid. The World Bank team conducted quarterly missions taking advantage of the implementation missions for the Investment Policy Financing Upgrading and Greening the Rio de Janeiro Urban Rail System (P111996), plus several missions from high-level management. The World Bank leveraged other sources of financing to improve and strengthen this operation (as in the case of Policy Areas 2 and 3), integrating the relevant activities to avoid duplication and provide systemic responses to the implementation challenges. The World Bank implemented actions to improve product implementation such as (a) insisting on the creation of a task force headed by the Casa Civil, (b) program actions such as implementing mobile Lilac houses by using buses to

23

serve poor neighborhoods in the periphery of Rio de Janeiro so as to achieve a higher number of women benefiting from information related to the domestic violence law, (c) by working on technical advice to reformulate the contents of kiosks and (d) by working actively with the SEASDH to improve their monitoring capacity. Moreover, the World Bank engaged with high-level international expertise to support the drafting of concession documents for the inter-municipal buses and had an active role in guaranteeing that state-of-the-art performance-based clauses were included in the document. Even though extensions of the closing date of Development Policy Operations (DPOs) are not common practice, the World Bank team supported the request of the SRJ, in an effort to continue the engagement to allow for the issuance of some of the important reforms.

(c) Justification of Rating for Overall Bank Performance Rating: Moderately Unsatisfactory

74. Because the sub-ratings for bank performance in ensuring quality at entry and the quality of supervision are Unsatisfactory and Satisfactory, the overall rating is determined by the outcome rating.

5.2. Borrower Performance

(a) Government PerformanceRating: Unsatisfactory

75. The Government’s commitment to the reforms and achievement of the development objectives was weak and changes to government counterparts and lack of ownership from some of the agencies delayed the implementation. The commitment to enhance resources for managing the program in Policy Area 3 was not achieved, which affected the program’s performance. The integration among transport agencies was deficient and despite considerable achievements by SETRANS, DETRO has not been able to push for the necessary restructuring on the inter-municipal bus lines and the issuance of the concessions, which affected Policy Area 2.

(b) Implementing Agency or Agencies Performance Rating: Unsatisfactory

76. Four secretariats (Casa Civil, SETRANS, DETRO, SEFAZ, SEASDH) were the implementing agencies of the project. Changes in the key staff of these secretariats (particularly SEASDH and DETRO) slowed down the implementation of the project and contributed to institutional memory loss.

77. The commitment of Casa Civil for project preparation was initially strong and efficient. However, changes in personnel right after program approval negatively affected implementation, which required strong support from the Governor’s office to achieve the critical reforms. Starting in May 2014, Casa Civil created a task force to supervise the implementation of the program and help coordinate execution. This task force was valuable for coordinating actions and monitoring progress in the implementation of the program. Nevertheless, the lack of political support diminished the effectiveness of this effort.

24

78. The performance of SEFAZ ensured the delivery of project indicators during implementation, complying with covenants and agreements. However, during preparation, it would have been essential to include actions toward fiscal adjustment, as, by project preparation, there were already visible signs of the rapid economic deterioration.

79. SETRANS was the implementing agency of Policy Area 2. Important achievements were obtained for the activities that were more exclusively under SETRANS’ jurisdiction, such as the development and dissemination of the PDTU, the advances in the BUI policies, and the implementation of integration stations Maracana and Madureira. SETRANS did not have the de facto authority to coordinate the implementation of the full extent of this policy area, despite being, at this point, the most appropriate agency to develop projects that required coordination among agencies and that implement metropolitan-wide projects in the SRJ. The fragmentation of responsibilities in the transport institutional setting and the inexistence of a strong metropolitan agency reinforced this situation.

80. DETRO. The agency regulates the inter-municipal bus transport and took the lead in discussion in parts of Policy Area 2. It was only after the strong supervision of the World Bank that DETRO started to include some of the necessary adjustments to concession documents, which have been completed, but not rolled out. Moreover, the necessary integration of inter-municipal bus lines with SuperVia, a key piece for the sustainability of the transport system, was not implemented, a result that can also be attributed to the inexistence of a metropolitan transport authority. Nevertheless, the concession documents developed by DETRO promote higher quality standards than the current licensing system, and, if implemented, will constitute a better platform for promoting the provision of quality bus services.

81. SEASDH. Although the implementation of the kiosks and construction of service facilities was funded, SEASDH lacked the institutional capacity to implement these interventions, and the changes of key staff caused loss of institutional memory. The decision to outsource some components of the program to UNOPS and the World Bank technical support in the implementation were key to the achievement of some prior measures. The implementation of the kiosks in the SuperVia stations was challenging. Nevertheless, the agency showed important resourcefulness in implementing the Lilac Bus, which allowed for expanding the reach of the Maria da Penha Law.

(c) Justification of Rating for Overall Borrower PerformanceRating: Unsatisfactory

82. Given the performance of the Government and agencies, the overall borrower performance was Unsatisfactory.

6. Lessons Learned

83. The operation included some necessary strategic reforms, but the instrument offered limited support for achieving these reforms. Given the critical fiscal situation the state was facing and the need for fast disbursement, the operation was prepared to cover the funding gap, taking advantage to implement strategic reforms for improving service delivery in both transport and gender policy areas. While the operation included these strategic objectives, the

25

short period for implementation of prior actions and the one-time disbursement left the World Bank with little leverage during implementation to push for these reforms, which required strong commitment from the Government. The political scenario ended up playing a stronger role and there was no direct incentive for the Government to maintain commitment, despite the intense supervision efforts. A recommendation for these cases, when the full extent of reforms cannot be achieved upfront, is to include in the operation sequential disbursements based on the gradual achievement of pre-conditions, which could provide continuous incentives to maintain government commitment.

84. There was a misalignment between measures supported by the program under the fiscal policy area and the fiscal challenges that the state was facing. During preparation, there was a recognition that fiscal adjustments in the period pre-events would be challenging for the state. Therefore, the fiscal actions supported by the program were not designed to address the issues that resulted in the current fiscal crisis, as the actions were related to financial management as opposed to fiscal adjustment measures. It is worth noting that it is unlikely that any operation could have avoided the deep fiscal crisis that the state is currently facing, especially given the limited authority of the state government to push for the pension reforms, which are critical for fiscal relief. Nevertheless, an overall recommendation is to seize the opportunity that comes with the request for urgent funding, by including in the operation activities for the development of the framework to address the necessary fiscal adjustment. This would give increased opportunity for the World Bank teams to support in the design of the solutions for adjustment and of the mitigating measures for the crisis.

85. The absence of a strong and committed leading agency has negatively affected the results of this multisector operation. This innovative operation was proposed using an integrated approach, which is key to improve efficiency and quality of public services. For example, in the transport sector, different transport modes have to be planned systemically, minimizing inefficiencies and externalities such as time loss, congestion and emissions. This means looking at transport modes as complementary rather than substitutes. In the gender sector, the new regulation (Maria da Penha Law) requires that the support centers are accessible, which can be provided by the transport system. Assessing the policy effectiveness requires integration of processes and databases (for example, hotlines, health centers and police department need integrated protocols to deliver a service that can be evaluated). The integrated approach of this operation had to face the complexity of institutional arrangements, marked by the fragmentation of responsibilities among government agencies. Moreover, there were inherent difficulties in this operation, which pursued structural reforms that affect multiple stakeholders. The efforts of Casa Civil to create task force (by May 2014) improved the pace of implementation, but were not sufficient to achieve the reforms. In this fragmented scenario, a determinant factor to improve the delivery of operations is to select a strong leading agency with sufficient authority over the implementing agencies, and to provide incentives for each agency to maintain commitment throughout project implementation.

86. Despite the overall unsatisfactory results, the design of the operation nonetheless offers valuable lessons. The implementation of an operation joining four different government secretariats, with multiple agencies in the case of the transport sector, and multiple counterpart cities (municipalities in the RJMR) in the case of gender policy, increased the complexity for supervision and coordination, for both the SRJ and the World Bank. Nevertheless, this cross-

26

sector operation allowed for increased awareness that service delivery should be integrated and multisectoral, and was pioneered a solution that integrates gender considerations into the transport agenda. It offered the human rights sector an opportunity to expand its playing field, using transport infrastructure to deliver information on gender violence and on the advances of the legal environment. The provision of these services in the train stations also benefits the transport sector, as the train operator can be seen as a community ally, where women’s safety is important. The improved service image enhances the attractiveness of the mass-transit system, likely increasing its market-share. The complexity of the design required additional technical support, which was mobilized through trust funds. In the end, the nature of the DPL instrument did not allow to fully leverage the Bank’s implementation support capacity, but valuable lessons how to design integrated solutions nonetheless remain.

87.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners

88. The SRJ considered the program to be satisfactory because many indicators were achieved or about to be achieved, allowing for more efficient service delivery, supported by the partnership with the World Bank. There is a disconnect between the ratings provided by the borrower and the ratings by the World Bank. This disconnect can be explained by different expectations from both sides. In most cases, the borrower assessment was based on the achievement of the indicators or on the achievement of the preparatory steps toward the achievement of the indicators. The World Bank’s ICR presents a more systemic view of the operation as a whole, including an evaluation of the design, and presents more stringent requirements for accepting the achievement of the indicators, which were often vague and, in some cases, allowed for different interpretations.

27

Annex 1. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team Members

Names Title Unit Responsibility/Specialty

LendingShomik Raj Mehndiratta Lead Urban Transport Specialist LCSTR Task team leaderGeorges B. Darido Urban Transport Specialist LCSTR Co-task team leaderDaniel Pulido Infrastructure Specialist LCSTR Team memberFabio Hirschhorn Consultant LCSTR Team memberHanayo Taguchi Program Assistant LCSTR Team memberBoris E. Utria Country Operations Advisor LCC5 Team memberRafael Chelles Barroso Senior Economist LCC5 Team memberFabio Sola Bittar Research Analyst LCC5 Team memberJoseph Kizito Lead Financial Management Specialist GGO22 Financial managementAlberto Coelho Gomes Costa Senior Social Development Specialist GSU04 SafeguardsMarcio Batitucci Social Environmental Specialist GEN04 SafeguardsMiriam Muller Social Scientist GPV04 SafeguardsAstrid Hegeland Social Specialist GPSOS SafeguardsChristine M. Richaud Lead Economist LCC5 EconomistCatalina I. Portelo Senior Counsel LEGLE LawyerEtel Patricia Bereslawski Lead Procurement Specialist LCC5 ProcurementPatricia Hoyes Senior Financial Management Specialist LCC5 Financial managementSupervisionDaniel Pulido Senior Infrastructure Specialist GTIDR Task team leaderShomik Raj Mehndiratta Lead Transport Specialist GTIDR Team memberGeorges B. Darido Urban Transport Specialist GTIDR Team memberBianca Bianchi Alves Urban Transport Specialist GTIDR Team memberFabio Hirschhorn Consultant LCSTR Team memberHanayo Taguchi Program Assistant LCSTR Team memberBoris E. Utria Country Operations Advisor LCC5 Team memberRafael Chelles Barroso Senior Economist LCC5 Team memberFabio Sola Bittar Research Analyst LCC5 Team memberJoseph Kizito Lead Financial Management Specialist GGO22 Financial managementAlberto Coelho Gomes Costa Senior Social Development Specialist GSU04 SafeguardsMarcio Batitucci Social Environmental Specialist GEN04 SafeguardsMiriam Muller Social Scientist GPV04 SafeguardsAstrid Hegeland Social Specialist GPSOS SafeguardsChristine M. Richaud Lead Economist LCC5 EconomistCatalina I. Portelo Senior Counsel LEGLE LawyerEtel Patricia Bereslawski Lead Procurement Specialist LCC5 ProcurementPatricia Hoyes Senior Financial Management Specialist LCC5 Financial managementFatima Arroyo Arroyo Consultant GTIDR Team member

(b) Staff Time and Cost

28

StageStaff Time and Cost (Bank Budget Only)

No. of Staff Weeks US$, thousands (including travel and consultant costs)

LendingFY14 42.55 123.76

Total: 42.55 123.76Supervision/ICRFY14 6.61 29.18FY15 17.24 38.56FY16 12.94 21.85FY17 3.00 10.12

Total: 39.79 99.71

29

Annex 2. Beneficiary Survey Results

Not applicable.

30

Annex 3. Stakeholder Workshop Report and Results

Not applicable.

31

Annex 4. Summary of Borrower's ICR and/or Comments on Draft ICR

1. The borrower’s ICR included a description of the program and concluded that significant progress was achieved in all three policy areas, especially considering the current political and economic crisis that the country is facing.

2. The borrower recognized that some of the targets for December 2014 were not achieved, given the changes in state secretaries and the electoral period, when the public administration is legally and politically prevented from implementing some of the actions. As a result of that, the state requested a program extension until January 31, 2016.

3. The report also informed that

With regard to Policy Area 1, all indicators were achieved in 2014, and six of the eight indicators were achieved in 2015 despite the economic-fiscal crisis;

With regard to Policy Area 2, five of the eight indicators were achieved, and the launch of the new concessions is subject to the approval of the State Audit Office; and

Indicators for Policy Area 3 were not only achieved but the state is also expanding the project with international funding.

4. No comments were provided to the draft ICR.

32

Annex 5. Comments of Co-financiers and Other Partners/Stakeholders

Not applicable.

33

Annex 6. Review of the Fiscal and Debt Sustainability Analyses Provided During Preparation

1. The preparation team had to cope with a rapidly changing country and state fiscal environment. In hindsight, the country context provided in the PAD was overly optimistic, though it did point out important risks and weaknesses. It was also broadly in line with what others (markets, International Monetary Fund) were thinking about Brazil at that time. The SRJ fiscal position was already problematic at the time of preparation, because the urgent need for the operation was based on liquidity constraints the state was confronting for the end of 2013. Nevertheless, the fiscal situation deteriorated dramatically after the operation was approved. Some of the factors driving the deterioration could not have been anticipated (oil price collapse, fallout from Petrobras corruption investigations, and prolonged nationwide recession).

2. The fiscal analysis recognized important vulnerabilities but was quick in assuming that they could be contained. The state’s dependency on oil royalties presented a fiscal risk that was acknowledged (mostly as risks coming from royalties’ reform, not from the oil price as materialized). The higher investment spending was justified as temporary in nature; however, the doubling of personal expenditures in five years (2007–2012) was a warning sign. The negative primary balance in a context of already large debt service obligations (155 percent debt of NCR, projected to reach 194 percent of NCR) was also a reason for concern. The fiscal vulnerability was recognized in the debt sustainability analysis (DSA) where the baseline scenario was projected to bring debt to 194 percent of revenues, close to the legal limit (200 percent) and most of the shocks being studied would lead to debt exceeding the limit. However, the fiscal team assumed that in the case of adverse shocks the state would contain expenditures, even though it should have been clear that containing expenditures (other than investment) would be difficult given the high rigidity of spending. Fiscal projections were very optimistic, assuming restraint in expenditures when recent history clearly pointed to real expansion and spending pressures were well known (World Cup, Olympics, protests, and so on).

3. The assumptions made for the DSA were too optimistic. Expenditures were projected to grow at very moderate rates, well below the historical experience of personnel and other current expenditures growing much faster than nominal gross domestic product. Comparing the DSA projections with the fiscal outcomes in recent years (Table 6.1) shows that current expenditures were underestimated in 2013 and capital spending was underestimated over the entire forecasting period. More importantly, revenue was overestimated; however, this can be explained by the dramatic fall in revenues beginning in 2014 because of the economic crisis.

34

Table 6.1. Comparing the DSA Projections with Fiscal Outcomes

DSA PROJECTIONS Outcomes* 2012 2013 2014 2015 2012 2013 2014 2015I. Revenue 53.1 57.4 62.8 65.1 55.0 57.9 57.2 58.8Taxes 35.1 38.5 42.5 43.1 35.1 35.7 37.4 36.9

ICMS 29.2 32 35.4 35.4 26.7 26.1 26.8 24.4IPVA 1.7 1.9 2.1 2.3 1.7 1.7 1.8 1.2OUTRAS 4.2 4.5 5 5.5 6.7 7.9 8.7 11.4

Social Contributions 1.3 1.4 1.5 1.7 1.3 1.4 1.9 1.9Transfers 3.8 4.3 4.6 4.9 6.3 5.9 5.9 5.8

Current Transfers 3.3 3.6 3.9 4.2 5.7 5.4 5.6 5.4Capital Transfers 0.5 0.7 0.7 0.7 0.5 0.5 0.3 0.4

Other Current Revenues 12.9 13.2 14.1 15.4 12.3 14.9 11.9 14.2

NCR 40.6 43.2 47.3 49.2 40.6 47.1 46.0 51.2

II. Expenses 52.6 56.2 61.3 65.1 52.8 58.1 59.7 61.8Compensation of Employees 26.7 28.3 31.3 34.2 14.4 18.6 18.2 19.1Interest Payments 2.6 2.9 3.1 3.2 2.6 2.9 3.2 3.8Other Current Expenditures 21 22.6 24.4 25 35.8 36.5 38.3 38.8

Transfers to Municipalities 8.8 9.6 10.6 10.8 8.4 9.6 10.1 10.0Goods and Services 12.2 13 13.8 14.2 27.3 27.0 28.2 28.8

III. Gross Operating Balance (I-II) 0.5 1.1 1.4 0 2.2 -0.2 -2.5 -3.0% of NCR 1.2 2.6 3 0 5.5 -0.3 -5.5 -5.8

IV. Transactions in Non-Financial Assets 5.5 6.6 6.8 5.3 5.5 6.9 7.7 6.6

Investment in Non-Financial Assets 5.3 6.4 6.6 5.1 5.3 6.7 7.6 6.6

Total Expenditure 58.1 62.8 68.1 70.4 58.2 64.9 67.4 68.4 V. Net Lending/ Borrowing (III-IV) -5 -5.4 -5.4 -5.3 -3.2 -7.1 -10.2 -9.6 VI. Primary balance -2.9 -3.1 -2.9 -2.8 -0.9 -4.7 -7.3 -3.6

% of NCR -7.2 -7.2 -6.2 -5.6 -2.2 -10.0 -15.9 -7.1 VII. Transactions in Financial Assets 2.8 3.8 4.7 1.8 2.7 2.4 4.4 1.9

New Loans 4.8 6.3 7.2 4.7 4.8 5.0 7.6 5.0Amortizations, net -2 -2.6 -2.5 -2.9 -2.0 -2.6 -3.2 -3.1

VIII. Gross Financing Needs 4.1 4.9 5 5.5 5.6 10.2 13.7 10.6

% of NCR 10.1 11.3 10.6 11.2 13.7 21.7 29.8 20.7 VIII. Overall Balance (VI + VII) -0.2 0.7 1.7 -0.9 -0.8 -5.2 -6.1 -5.6

35

*Figures in DSA and outcomes are not strictly comparable, since outcomes are based on public data (RREO and DCA) while DSA was built based on data submitted directly by the government. Additionally, accounting changes to tax revenue transferred to municipalities means that these transfers had to be estimated for the updated table.

36

Annex 7. Background on Results for Policy Objective 1.2 Regarding Revenue Forecasting Errors

1. The first results indicator under Policy Objective 1.2. deals with the accuracy of revenue forecasting. In the approved PAD it reads: “Reduction in the gap between actual and estimated current revenues, measured as a share of the estimated revenues. Baseline = 6.6% (average 2010–2012); Target = 5% (2014).” This is a quantifiable results indicator and a relevant measure of the state’s ability to plan its budget based on realistic revenue expectation.

2. The Government’s Conclusion Report changes the definition of the indicator. The Government’s Project Conclusion Report (Relatório de Conclusão do Projeto) reports both the projected revenue (in the Annual Budget Law [Lei de Orçamento Anual, LOA]) and the actual revenue collected in the year. However, it then calculates a so called ‘updated revenue’, using historical revenue data and economic variables (column G in table 7.1). It ultimately reports, as the indicator, the divergence between projected revenues and this ‘updated revenue’, not the actual revenue as stated in the indicator. This leads to underreporting of the projection error, which defines whether the target is met. In 2014, the divergence between the two errors was small (using the data from the Project Conclusion Report).

Table 7.1. Reported Results for 2014 (table 5 in the Project Conclusion Report)

3. For 2015, the divergence between the original and the adjusted methodology is significant. While the government calculations estimate revenue under projection of 4.7 percent, using actual revenues (as reported in the Project Conclusion Report) results in a revenue over projection of 5.5 percent, meaning that the target was not achieved (see Table 7.2, column I). Given the negative economic shocks in 2015, revenue falling short of the projection is, however, not surprising.

37

Table 7.2. Reported Results for 2015 (table 6 in the Project Conclusion Report)

4. In addition, the data reported in the Project Conclusion Report differ from that available in public sources. The numbers in the report diverge for both the revenue projection (compared to those in the published LOA documents) and the revenue outcomes (reported in the annual accounts). The calculation of actual current revenues is further complicated by the treatment of revenue deductions (mostly transfers to municipal governments, see Table 7.3). Using the publicly available data, the forecasting errors are larger than the ones reported, at 6.1 percent in 2014 and 11.9 percent in 2015 (up to 216% difference). This would mean that the target was missed in both years.

Table 7.3. Contrasting Government Reported Results with Public Data

Year Receitas Correntes (DCA) (-) Deduções (=) Receitas Correntes (RREO)2014 64,478,113,222 6,769,189,659 57,708,923,564 2015 68,867,486,689.42 17,468,898,975.23 51,398,587,714

Data: Siconfi

Ano Receita Bruta (I) Receitas de capital (II) I - II Publicação LOA2014 82,899,620,142 14,477,147,112 68,422,473,030 Janeiro 20142015 90,311,430,806 13,232,824,926 77,078,605,880 Janeiro 2015

Data: LOA

Ano Projetada Realizada (Rec. Proj. LOA) - (Receitas DCA) (%)2014 68,422,473,030 64,478,113,222 3,944,359,808 6.122015 77,078,605,880 68,867,486,689.42 8,211,119,190.58 11.92

Ano Receitas Correntes (A) Rec. Proj. LOA (B) B - A (%)2014 63,254,631,785 64,860,210,129 1,605,578,344 2.52015 68,831,319,802 72,616,406,022 3,785,086,220 5.5

Dados - Governo do Rio de Janeiro

Receita realizada (dados publicados)

Receita Projetada (LOA publicada)

Error de projecao

38

5. Significant forecasting errors are not necessarily a sign of poor performance of the tool employed by the state. Given the inherent volatility of the state’s revenue base and the special economic circumstances since 2014 (national economic crisis, changes in federal fiscal policy, inflationary spikes, oil price collapse, and Petrobras crisis paralyzing the Brazilian oil sector), a significant revenue forecasting error was probably inevitable. A fair evaluation of the forecasting methodology would require more in-depth analysis over a longer period of time.

39

Annex 8. Public Transport System in Rio de Janeiro

1. Introduction

1. The public transportation network of the RJMR integrates the following main systems: SuperVia, metro, water transport, cable car, inter-municipal buses, municipal buses, BRT, and VLT (Figure 8.1). The public transport network experienced significant development in the last decade as a result of RJMR engagement to provide better public transport service for their citizens as well as the catalytic effect of major international events hosted in Rio, the World Cup and the Olympics.

2. The trip distribution in the RJMR21 shows that public transport has the higher share with 49 percent of trips, private motorized transport 18 percent, and non-motorized modes 33 percent. Within the public transport modes, the number of trips per mode out of the number of trips by public transport is municipal buses 62.7 percent, inter-municipal buses 16.7 percent, metro 6.2 percent, SuperVia 5.3 percent, BRT 4.9 percent, water transport 1.0 percent, and VLT 0.5 percent. See table 8.1 for more details.

Table 8.1. Number of Trips in the RJMR and percentage of Trips Per Mode

Mode of Transport Trips/Day (2012)

% of Trips Out of Total Trips

% of Trips Out of Trips of Its Category

Total public motorized transport 10,662,500 49 100.0Municipal buses 6,671,000 31 62.6Inter-municipal buses 1,781,000 8 16.7Metro 665,000 3 6.2SuperVia 568,000 3 5.3BRT 520,000a 2 4.9Water transport 105,000 0 1.0VLTb 50,000 0 0.5Taxi and mototaxi 295,000 1 2.8Cable car 7,500 0.03 0.07Total private motorized transport 3,935,000 18 100.0Private cars 3,765,000 17 95.7Motorcycle 170,000 1 4.3Total private NMT 7,180,000 33 100.0Walking 6,634,000 30 92.4Cycling 546,000 3 7.6Total 21,777,500 100 —

Source: PDTU 2012.Note: a. Estimated from data from BRT Rio.b. Estimated from data from VLT Rio

2. Public Transport Modes

3. SuperVia. It is the suburban rail that serves the RJMR. The private concessionaire SuperVia is in charge of the management, operation, and maintenance of the system. The railroad network comprises 100 stations, 270 km in 12 cities: Rio de Janeiro, Duque de Caxias,

21 PDTU, 2012.

40

Guapimirim, Nova Iguaçu, Nilópolis, Mesquita, Queimados, São João de Meriti, Belford Roxo, Japeri, Paracambi, and Magé.

4. Metro. The underground railway network serves the city of Rio de Janeiro. The system covers a total of 58 km, serving 41 stations, divided into three lines: Line 1 (16 km), Line 2 (30.2 km), and Line 4 (16 km). MetrôRio is the concessionaire, which manages, operates, and maintains the three metro lines.

5. Water transport. It consists of ferries (Barcas) that operate in four lines that connect Rio de Janeiro with Niteroi and the islands of the Guanabana Bay. A private concessionaire operates the water transport.

6. Cable car. The 3.5 km-cable car, Alemao, serves a slum area in the north of Rio de Janeiro. The system consists of 152 gondolas, each of which can carry 10 passengers. The system integrates with the SuperVia at Bonsucesso Station. The system is operated by a private concessionaire, although currently the service is suspended because of delays in payment from the state.

7. Local and inter-municipal buses. A total of 130 bus companies provide services in the RJMR, with 1,200 bus lines and over 16,000 buses.22 The Federação das Empresas de Transportes de Passageiros do Estado do Rio de Janeiro is one of the main stakeholders in public transportation in the RJMR, it aggregates 10unions of bus companies accounting for 81 percent of all regular public transportation in the SRJ. In 2010, the MRJ granted concessions for its municipal bus lines. The state is currently preparing the tendering to concession the inter-municipal bus lines.

8. BRT. The BRT system consists of three lines: Transoeste, Transcarioca, and Transolimpica. The BRT lines are operated by a private concessionaire. The MRJ is in charge of the road maintenance of Transoeste and Transcarioca. In the case of Transolímpica, a private concessionaire provides road maintenance, manages the toll system, and offers emergency response in the corridor.

9. VLT. VLT Carioca is the 15-km light rail that serves the east area of Rio de Janeiro. It operates with 32 trams and it provides access to the local airport. The light rail is operated by a private concessionaire.

3. Bilhete Unico Integrado and Affordability of Public Transportation

10. In 2010, the SRJ launched the integrated electronic card called BUI. The BUI integrates fare collection of public transport modes and it allows residents of the 20 municipalities in the RJMR to transfer between modes within two and a half hours. The BUI allows people to pay a single fare and the SRJ subsidies the costs above that fare. The fare integration in the RJMR increased affordability of public transportation, mainly for those who live in the periphery and need to transfer modes to access to jobs. Currently, the BUI benefits more than 4.9 million people.

22 Kiepsch, Matthias. 2012. “Sustainable Urban Transport Approaches for Brazilian Megacities—the Examples of Rio de Janeiro and Curitiba.”

41

11. Nevertheless, the BUI subsidy proved to be unsustainable. The annual subsidy of the BUI increased 215 percent from 2010 to 2015, passing from R$190 million to R$600 million. The abrupt increase of subsidy motivated Rio to implement readjustment of fares in 2015 to reduce subsidy and adjust demand to more efficient modes. In December 2016, the unsustainable subsidy together with the critical fiscal situation in the RJMR catalyzed additional changes.

4. Policies and Institutions in Urban Transport

12. In 2016, the SRJ presented the updated PDTU. The World Bank supported the preparation of this new plan within the Upgrading and Greening the Rio de Janeiro Urban Rail System Project (P125630). The PDTU assesses the changes in the transport system and provides sustainable transport policies for the Metropolitan Region. For the first time in years, the development of this plan provided the MRJ and the SRJ the opportunity to work together in developing their investment programs and subsidy policies.

13. Responsibilities of urban transport in the RJMR are divided between the municipalities and the state:

(a) Municipal level.23 Urban transport responsibilities within a municipality fall under the jurisdiction of the municipality’s prefeitura (mayor’s office). Urban transport in the MRJ falls under the jurisdiction of the municipal Secretary of Transport, who mainly oversees buses (all of which are private), BRT, VLT, and traffic management. It is the municipalities’ responsibility to grant concessions to local bus companies and taxi services.

(b) State level.24 Metropolitan transport (in most cases, urban transport involving more than one municipality) is under the jurisdiction of the state. The SETRANS is responsible for policy setting, regulation, and supervision of all public and freight transport which cross municipal borders. SETRANS oversees SuperVia (suburban rail), metro (subway), intermunicipal buses, cable cars, and Barcas (ferries). SuperVia, metro, ferries, and cable cars operate under the concession of the state government. It is the state’s responsibility to grant concessions to inter-municipal bus companies.

14. DETRO is the agency in charge of regulation of the inter-municipal bus system in the state. There are two types of services regulated by DETRO: (a) the urban services that use vehicles with two doors, common seats, turnstiles, and can circulate with standing passengers; and (b) the intercity services whose buses have reclining seats and which cannot circulate with standing passengers.25

23 Rebelo, Jorge. “Reforming the Urban Transport Sector in the Rio de Janeiro Metropolitan Region: A Case Study on Concessions.”24 Rebelo, Jorge. “Reforming the Urban Transport Sector in the Rio de Janeiro Metropolitan Region: A Case Study on Concessions.”25 Kiepsch, Matthias. 2012. “Sustainable Urban Transport Approaches for Brazilian Megacities – the Examples of Rio de Janeiro and Curitiba.”

42

15. AGETRANSP is the regulatory agency of Barcas (water transport), SuperVia (railways), metro and highway concessions of the SRJ. It has the purpose of regulating, monitoring, and supervising the concessions.

43

Figure 8.1. Public Transport System in the RJMR

Source: Prefeitura de Rio de Janeiro.

44

Annex 9. Supported Policy Areas

Area / Objectives Prior Actions Expected ResultsPolicy Area 1: Instituting policies and regulations to improve medium-term planning and monitoring of public expenditures

Policy Objective 1.1: Institutionalize sound financial and debt management policies and practices

The state has adopted the legal framework to implement its Treasury’s operational risk policy, which policy provides for, among others, mandatory disclosure of related documents on the Internet and the state’s submission to yearly external audits of said policy, as evidenced by the State Decree n. 44.428 issued on October 11, 2013 (Prior Action #1)

The state has established the legal basis for the use of sound financial and debt management tools, such as budget and financial quotas, minimum and average cash balance, and procedures for making payments to its suppliers, as evidenced by the State Decree n. 44.429 issued on October 11, 2013 (Prior Action #2)

Maintenance of the number of events (occurrences and non-conformities) in which the operational risk policy procedures were not complied with within already achieved satisfactory levels. Baseline = [occurrences = 52; non-conformities = 12] (2012); Target = [occurrence = 48; non-conformities = 11] (2014)

Reduction in the number of times in a year that the actual monthly cash balance has remained below the minimum threshold (set at 1.5 times the payroll amount). Baseline = 3 out of 12 (2012); Target = 2 out of 12 (2014)

Reduction in the percentage of payments made after 30 days. Baseline = 3.6 percent (2012); Target = 3.0 percent (2014)

Policy Objective 1.2: Strengthen the definition of the budget resource envelope, reinforcing its hard constraint, under a Medium-Term Expenditure Framework (MTEF)

The state has adopted the following budget preparation practices to improve its accountability and fiscal discipline: (A) preparation, publication, and regular updating of detailed revenue estimates including subsequent years’ estimates; and (B) issuance of an annual budget directive setting appropriate sectoral budget ceilings which shall be consistent with the medium-term revenue forecasts and targets for public indebtedness, as evidenced by the State Decree n. 44.431 issued on October 11, 2013 (Prior Action #3)

Reduction in the gap between actual and estimated current revenues, measured as a share of the estimated revenues. Baseline = 6.6 percent (average 2010–12); Target = 5 percent (2014)

Publication in the Internet of revenue estimates documents for the 2014 budget. Baseline = No (2012); Target = Yes (2014)

Annual Budget directive issued for the 2015 budget cycle. Baseline = No (2012); Target = Yes (2014)

Policy Objective 1.3: Implement a framework to deal with fiscal commitments originated from Public-Private Partnerships (PPPs)

The state has adopted the following institutional practices to improve its transparency and accountability: (A) regular publication of a comprehensive estimate of fiscal commitments, including explicit and contingent payments from effective and prospective PPP contracts which estimates shall be updated at least annually together with the LOA; and (B) publication of all documents and analysis relating to PPP projects through a specialized website, as evidenced by the State Decree n. 44.430 issued on October 11, 2013 (Prior Action #4)

A comprehensive report including current and prospective long-term commitments and liabilities associated to PPPs is published as scheduled. Baseline = No (2012); Target = Yes (2014)

All PPP-related bidding documents, contracts, and supporting analysis are published on a dedicated website in a timely manner. Baseline = No (2012); Target = Yes (2014)


45

Area / Objectives Prior Actions Expected Results

Policy Objective 2.1: Foster modal and regional integration in urban transport systems

The state has (A) adopted the Urban Transport Master Plan (PDTU) as the policy framework for the planning and development of an integrated transport system in the RJMR; and (B) established a formal coordination mechanism to enable the state, the academia, civil society institutions, and the RJMR municipal governments to participate in the implementation of the PDTU, as evidenced by the State Decree n. 44.433 issued on October 10, 2013 (Prior Action #5)

The state has taken a first step to initiate the implementation of a policy to physically integrate municipal and state-managed mass transit systems in the RJMR, as evidenced by the Memorandum of Understanding entered into between the state and SuperVia (rail operator) for the implementation of multimodal integration projects, dated August 22, 2013 (Prior Action #6)

The state has adopted a new policy to promote non-motorized transport (NMT) through which it will (A) ensure that secured bicycle parking will be available at all major transit stations in the RJMR; and (B) provide technical assistance to RJMR municipalities in developing high quality NMT access to major public transport stations, as evidenced by SETRANS Resolution n. 1.114 issued on October 10, 2013 (Prior Action #7)

Permanent Committee for the implementation of the PDTU, involving relevant government and civil society actors, meets at least four (4) times in its first year of existence and presents strategy for the implementation of at least one (1) priority project under the PDTU. Baseline = 0 (December 2013); Target = 4 meetings and 1 PDTU project discussed (December 2014)

No transport project outside of the PDTU (that is, not endorsed by the PDTU) will be undertaken by the SRJ

At least one (1) multimodal integration project, connecting state and municipal transit systems, is completed. Baseline = 0 (December 2013); Target = 1 (December 2014)

State prepares guidelines for the development of bicycle-public transit integration projects to be implemented by metropolitan municipalities and at least one (1) of these projects is implemented with the state’s technical assistance. Baseline = 0 projects (December 2013); Target = 1 project (December 2014)

Policy Objective 2.2: Enhance performance and transparency of inter-municipal bus services

The state has adopted a policy to (A) replace existing permission-based route licenses for inter-municipal bus routes with competitively awarded concessions; (B) institute a performance-based system for managing said concessions; and (C) increase transparency in bus service performance for the public, as evidenced by the State Decree promulgated on October 11, 2013 (Prior Action #8)

The concession bidding documents will identify the bus lines to be sectioned or eliminated in order to prioritize integration with mass transit modes (name, current code, and in the cases of sectioning, new extension, new fleet, and estimated demand). Baseline = 0 lines (December 2013); Target = 19 lines are identified and reorganized or eliminated (December 2014)

Inter-municipal bus lines in the RJMR (previously provided under permission regime) are in operation under new concession agreements incorporating performance monitoring. Baseline = 0 lines (December 2013); Target = 50 percent of the bus lines under the new route plan are awarded to competitively selected concessionaires (December 2014)

Service reliability, fleet modernization, and vehicle occupation requirements are properly monitored and enforced on inter-

46

Area / Objectives Prior Actions Expected Resultsmunicipal bus concessions in operation. Baseline = n.a. (December 2013); Target = 80 percent of bus concessions either comply with requirements or are penalized (December 2014)

Policy Objective 2.3: Ensure the long-term sustainability and effectiveness of the state’s urban transport subsidy/affordability programs

The state has adopted a technical evaluation platform for periodic monitoring of the financial viability and cost-effectiveness of all transport programs requiring ongoing subsidies in an effort to ensure their financial sustainability, as evidenced by Resolution n. 1.113 issued by SETRANS on October 10, 2013 (Prior Action #9)

New operations contract awarded for the cable car system incorporates performance monitoring, availability-based remuneration, and incentives for maximizing ancillary revenue; and results in a reduction in the operational subsidies paid by the State. Baseline= System operated under temporary arrangement with a subsidy of R$14.00 per passenger per trip (2013); Target=new operational contract results in a 5 percent reduction in the subsidy per passenger per trip and is monitored and remunerated using performance indicators (December 2014)


Policy Objective 3.1: Operationalize the Maria da Penha Law by leveraging transport infrastructure for improved delivery and targeting of services

The state has established the Sub-secretariat of Policies for Women (SPM-RJ/SEASDH), as evidenced by the State Decree 44.076 issued on February 20, 2013 (Prior Action #10)

The state has (A) adopted a policy to promote gender equality in the transport sector specifically focused on the women users of the transport system operated by SuperVia; and (B) committed to implement the ‘Programa SuperVia e Teleférico Lilás,’ as evidenced by (A) the Joint Resolution SEFAZ/SETRANS/SEASDH issued on October 14, 2013 and (B) the allocation of US$15 million (equivalent) to the SPM-RJ/SEASDH in the borrower’s 2014 Budget Law Proposal published in the Official Gazette on October 1, 2013 (Prior Action #11)

Number of women benefiting from increased access to information and to social and legal services as per the provisions of the “Lei Maria da Penha” within the SuperVia and Teleférico systems. Baseline = 0 (December 2013); Target = 50,000 (December 2014)

Increase in the number of women benefiting from the integration of the SuperVia and Teleférico systems to the ‘Casa da Mulher Brasileira;’ Baseline = 0 (December 2013); Target = 1,000 (December 2014)

47

Annex 10. Results Framework

Table 10.1. Relevance of the Specific Policy Level Objectives for Policy Area Objectives (the PDOs)

Policy Area Individual Policy Objectives (“Policy Action”)

Rating of Relevance Comments


(1.1) Institutionalize sound financial and debt management policies and practices

Modest

This individual policy action focuses on operational risk, where “Operational risk is defined as the possibility of losses caused by deficiencies of internal processes, staff, systems and external events.” While mitigating operational risks is worthwhile, these are less important than conventional fiscal risks (revenue shortfall, inability to control expenditures) to improve medium-term planning and monitoring of expenditures.

(1.2) Strengthen the definition of the budget resource envelope, reinforcing its hard constraint, under a Medium-Term Expenditure Framework (MTEF)

ModestThis individual policy action also focuses on operational risk and is vague. Because of this, it has only modest relevance in improving medium-term planning and monitoring of expenditures.

(1.3) Implement a framework to deal with fiscal commitments originated from Public-Private Partnerships

SubstantialThis individual policy action is a worthwhile effort that supports the achievement of the policy area of improving medium-term planning and monitoring of expenditures.


(2.1) Foster modal and regional integration in urban transport systems High This individual policy action is highly relevant and supports the

achievement of the policy area objective.

(2.2) Enhance the performance and transparency of inter-municipal bus services

High

This individual policy action is highly relevant and supports the achievement of the policy area objective. Given the state context, the achievement of this individual policy action under the DPO operation time frame can be considered ambitious.

(2.3) Ensure the long-term sustainability and effectiveness of the state’s urban transport subsidy/affordability programs

High

This individual policy action is highly relevant and supports the achievement of the policy area objective. However, the activities selected to implement this individual policy action would not allow for the achievement of this ambitious and high-level objective.

Policy Area 3: Instituting (3.1) Operationalize the Maria da Substantial This individual policy action is substantially relevant and supports the

48

Policy Area Individual Policy Objectives (“Policy Action”)

Rating of Relevance Comments

policies and regulations to improve the availability of targeted social services aimed at reducing domestic and gender-based violence

Penha Law by leveraging transport infrastructure for improved delivery and targeting of gender-specific services

achievement of the policy area objective. However, the activities selected would not allow for the “institution of policies and regulations” according to the policy area objective.

Table 10.2. Relevance of the Indicators for Measuring the Individual “Policy Actions” and PDOs

Specific Policy Objectives Indicator

Relevance of

IndicatorComments


Policy Action 1.1: Institutionalize sound financial and debt management policies and practices

KPI 1: Maintenance of the number of events (occurrences and non-conformities) in which the operational risk policy procedures were not complied with within already achieved satisfactory levels

LowIndicator is operational and unambitious, and therefore, it does not support the institutionalization of sound and financial debt management nor does it improve medium-term planning and monitoring of public expenditures.

KPI 2: Reduction in the number of times in a year that the actual monthly cash balance has remained below the minimum threshold (set at 1.5 times the payroll amount)

Low

Given the cyclical nature of cash balances, the indicator can be achieved even if the state is unable to pay basic commitments at the end of the year. Therefore, it does not support the institutionalization of sound and financial debt management nor does it improve medium-term planning and monitoring of public expenditures.

KPI 3: Reduction in the percentage of payments made after 30 days

High

This indicator does support the institutionalization of sound and financial debt management and the improvement of medium-term planning and monitoring of public expenditures. It is worth mentioning that the activities in this operation did not address the causes for this.

49


Relevance of

IndicatorComments

Policy Action 1.2: Strengthen the definition of the budget resource envelope, reinforcing its hard constraint, under a Medium-Term Expenditure Framework (MTEF)

KPI 4: Reduction in the gap between actual and estimated current revenues, measured as a share of the estimated revenues

Modest

Better ex ante revenue estimates are a useful tool but do not present a budget constraint if no action is taken to correct imbalances. It does strengthen the definition of the budget resource envelope but it does not necessarily allow for reinforcing its hard constraint. Therefore, it modestly improves medium-term planning and monitoring of public expenditures.

KPI 5: Publication in the Internet of revenue estimates documents for the 2014 budget

Modest

This indicator is met by simply completing procedures and while these procedures may be desirable (for transparency reasons), they have no fiscal effect. It does strengthen the definition of the budget resource envelope but it does not necessarily allow for reinforcing its hard constraint. Therefore, it modestly improves medium-term planning and monitoring of public expenditures.

KPI 6: Annual Budget directive issued for the 2015 budget cycle

Modest

This indicator is met by simply completing procedures and while these procedures may be desirable (for transparency reasons), they have no fiscal effect. It does strengthen the definition of the budget resource envelope but it does not necessarily allow for reinforcing its hard constraint. Therefore, it modestly improves medium-term planning and monitoring of public expenditures.

Policy Action 1.3: Implement a framework to deal with fiscal commitments originated from Public-Private Partnerships(PPPs)

KPI 7: A comprehensive report including current and prospective long-term commitments and liabilities associated to PPPs is published as scheduled

LowThis is a worthwhile effort to increase transparency, however, it has no link to implementing the framework or the quality of this framework. Therefore, it does not improve medium-term planning and monitoring of public expenditures.

KPI 8: All PPP-related bidding documents, contracts, and supporting analysis are published on a dedicated website in a timely manner

LowThis is a worthwhile effort to increase transparency, however, it has no link to implementing the framework or the quality of this framework. Therefore, it does not improve medium-term planning and monitoring of public expenditures.

Policy Area 2: Instituting policies and regulations to improve the accessibility, quality and affordability of urban mobility services for the poorPolicy Action 2.1: Foster modal and regional integration in urban transport systems

KPI 9: Permanent Committee for the implementation of PDTU, involving relevant government and civil society actors, meets four

High Despite the meetings being a relatively easy objective to comply with, this indicator was relevant because the fact that these meetings occurred meant that the PDTU was finalized and shared with the interested parties. Therefore, this indicator adequately measures the individual policy action of fostering modal and regional integration and is an important regulation to improving accessibility, quality, and affordability of urban mobility services for the poor.

50


Relevance of

IndicatorComments

(4) times in its first year of existence and presents strategy for the implementation of at least one (1) priority project under PDTU KPI 10: No transport project outside of the PDTU (that is, not endorsed by the PDTU) will be undertaken by the SRJ

High

Despite being an important indicator in the long term, this indicator was only modestly relevant given that in the time frame of this project it was unlikely that the SRJ would have undertaken new projects. In the long term, this indicator adequately measures the individual policy action of fostering modal and regional integration and is highly related to improving accessibility, quality, and affordability of urban mobility services for the poor.

KPI 11: At least one (1) multimodal integration project, connecting state and municipal transit systems, is completed

High

This indicator, although highly relevant given the historical disconnect between state and municipal systems, was unambitious. However, this indicator adequately measures the individual policy action of fostering modal and regional integration and is highly related to improving accessibility, quality, and affordability of urban mobility services for the poor.

KPI 12: State prepares guidelines for the development of bicycle-public transit integration projects to be implemented by metropolitan municipalities and at least one (1) of these projects is implemented with the state’s technical assistance

High

This indicator adequately measures the individual policy action of fostering modal and regional integration and is highly related to improving accessibility, quality, and affordability of urban mobility services for the poor. Although highly relevant, this indicator was unambitious, given the potential health, time savings, and lower emission benefits to citizens of improving connectivity in the municipalities that benefit from public transport infrastructure.

Policy Action 2.2: Enhance performance and transparency of inter-municipal bus services

KPI 13: The concession bidding documents will identify the bus lines to be sectioned or eliminated in order to prioritize integration with mass transit modes (name, current code, and in the

High This indicator adequately measures the individual policy action of enhancing the performance and transparency and is highly related to improving accessibility, quality, and affordability of urban mobility services for the poor. Although highly relevant, this indicator has proven to be ambitious given the project time frame.

51


Relevance of

IndicatorComments

cases of sectioning, new extension, new fleet, and estimated demand) KPI 14: Inter-municipal bus lines in the RJMR (previously provided under permission regime) are in operation under new concession agreements incorporating performance monitoring

High

This indicator adequately measures the individual policy action of enhancing the performance and transparency and is highly related to improving accessibility, quality, and affordability of urban mobility services for the poor. Although highly relevant, this indicator has proven to be ambitious given the project time frame.

KPI 15: Service reliability, fleet modernization, and vehicle occupation requirements are properly monitored and enforced on inter-municipal bus concessions in operation

HighThis indicator adequately measures the individual policy action of enhancing the performance and transparency and is highly related to improving accessibility, quality, and affordability of urban mobility services for the poor.

Policy Action 2.3: Ensure the long-term sustainability and effectiveness of the state’s urban transport subsidy/affordability programs

KPI 16: New operations contract awarded for the cable car system incorporates performance monitoring, availability-based remuneration, and incentives for maximizing ancillary revenue; and results in a reduction in the operational subsidies paid by the state

Modest

This indicator only modestly measures the individual policy action of ensuring long-term sustainability of the system and is modestly related to improving accessibility, quality, and affordability of urban mobility services for the poor. Considering the relative importance of the BUI program compared to the cable car service, it would have made more sense to include an indicator related to the BUI to measure progress in this area. Nonetheless, the SRJ did implement important changes to the BUI program, a much larger social program in the transport sector. These changes contributed to a reduction in fraud and better accountability.

Policy Area 3: Instituting policies and regulations to improve the availability of targeted social services aimed at reducing domestic and gender-based violencePolicy Action 3.1: Operationalize the Maria da Penha Law by leveraging transport

KPI 17: Number of women benefiting from increased access to information and to social and legal services

High

The indicator is highly relevant to operationalize the Maria da Penha Law and to reduce domestic and gender-based violence. However, the way it was measured, using the number of women accessing the information disseminated through the kiosks, does not allow to define how many women benefited from social and legal services as a result of

52


Relevance of

IndicatorComments

infrastructure for improved delivery and targeting of gender-specific services

as per the provisions of the ‘Lei Maria da Penha’ within the SuperVia and Teleférico systems

the project. Given the inexistence of integrated databases, however, this was considered as a reasonable proxy.

KPI 18: Increase in the number of women benefiting from the integration of the SuperVia and Teleférico systems to the ‘Casa da Mulher Brasileira’

High

The indicator is highly relevant to operationalize the Maria da Penha Law and to reduce domestic and gender-based violence. However, the indicator assumed that several service centers for women would have been finalized by the closing date, which did not occur. The indicator was revised during the midterm review, following confirmation that the ‘Casa da Mulher Brasileira’ would not be built by the Federal Government. The World Bank agreed to replace this center with one of the state-run centers to be built as part of the Via Lilac Program, which also faced delays. Considering these delays, the World Bank encouraged SEASDH to implement temporary measures to increase access to services to women by using a mobile service center known as the ‘Lilac Bus.’ This itinerant bus reached the poorest neighborhoods of 20 municipalities on the metropolitan area, serving more than 3,000 women with general information on violence prevention and women’s rights and related services.

53

brazil - enhancing public management for service delivery...

Documents