world bank documentdocuments.worldbank.org/curated/pt/795081468092955011/pdf/icr128… · all 230...

Document of The World Bank

Report No: ICR00001280

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-46850)

ON A

LOAN

IN THE AMOUNT OF US$30.0 MILLION

TO THE

RUSSIAN FEDERATION

FOR A

HEALTH REFORM IMPLEMENTATION PROJECT

December 22, 2009

Human Development Sector Unit Europe and Central Asia Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective August 31, 2009)

Currency Unit = Russian Ruble 1.00 = US$ 0.031 US$ 1.00 = 31.87

FISCAL YEAR

January 1 – December 31

ABBREVIATIONS AND ACRONYMS

Vice President:Philippe H. Le Houerou

Country Director:Klaus Rohland

Sector Manager:Abdo S. Yazbeck

Project Team Leader:Patricio Marquez

ICR Team Leader:Anne Bakilana

RUSSIAN FEDERATION Health Reform and Implementation Project

CONTENTS

Data Sheet A. Basic Information

B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph

Abbreviations and Acronyms ........................................................................................iv1. Project Context, Development Objectives and Design...............................................12. Key Factors Affecting Implementation and Outcomes ..............................................63. Assessment of Outcomes ..........................................................................................104. Assessment of Risk to Development Outcome.........................................................215. Assessment of Bank and Borrower Performance .....................................................216. Lessons Learned........................................................................................................237. Comments on Issues Raised by Borrower/Implementing Agencies/Partners...........24Annex 1. Project Costs and Financing............................................................................1Annex 2. Outputs by Component....................................................................................1Annex 3. Economic and Financial Analysis ...................................................................1Annex 4. Bank Lending and Implementation Support/Supervision Processes...............1Annex 5. Beneficiary Survey Results .............................................................................2Annex 6. Stakeholder Workshop Report and Results.....................................................2Annex 7. Summary of Borrower’s ICR...........................................................................3Annex 8. Comments of Co financiers and Other Partners/Stakeholders ......................18Annex 9. List of Supporting Documents ......................................................................18Annex 10. Additional Indicators...................................................................................19MAP

Abbreviations and Acronyms CAS

Country Assistance Strategy

CIDA

Canadian International Agency Development Agency

CIS

Commonwealth of Independent States

DFID

Department for International Development (United Kingdom)

ECA

Europe and Central Asia Region of the World Bank

EU

European Union

FHIF

Federal Health Insurance Fund

GP

General Practice

HRIP

Health Reform Implementation Project

MHI

Mandatory Health Insurance

MOH

Ministry of Health

MOHSD

Ministry of Health and Social Development

PAD

Project Appraisal Document

PDO

Project Development Objective

RHCF

Russian Health Care Foundation

TACIS

Technical Aid to the Commonwealth of Independent States

THIF

Territorial Heath Insurance Fund

USAID

United States Agency for International Development

WBI

World Bank Institute

WHO World Health Organization

i

A. Basic Information

Country: Russian Federation Project Name: Health Reform Implementation Project

Project ID: P046497 L/C/TF Number(s): IBRD-46850

ICR Date: 12/23/2009 ICR Type: Core ICR

Lending Instrument: SIL Borrower: GOVT. OF RUSSIA

Original Total Commitment:

USD 30.0M Disbursed Amount: USD 27.3M

Revised Amount: USD 27.3M

Environmental Category: C

Implementing Agencies: Ministry of Health and Social Development

Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 02/03/1999 Effectiveness: 12/09/2003 12/09/2003

Appraisal: 11/26/2001 Restructuring(s):

Approval: 03/18/2003 Mid-term Review: 12/11/2006 12/04/2006

Closing: 06/30/2008 04/30/2009 C. Ratings Summary C.1 Performance Rating by ICR

Outcomes: Satisfactory

Risk to Development Outcome: Low or Negligible

Bank Performance: Satisfactory

Borrower Performance: Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Satisfactory Government: Satisfactory

Quality of Supervision: Satisfactory Implementing Agency/Agencies:

Highly Satisfactory

Overall Bank Performance:

Satisfactory Overall Borrower Performance:

Satisfactory

ii

C.3 Quality at Entry and Implementation Performance IndicatorsImplementation

Performance Indicators

QAG Assessments (if any)

Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

No Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Satisfactory

D. Sector and Theme Codes

Original Actual

Sector Code (as % of total Bank financing)

Central government administration 10 10

Health 70 70

Non-compulsory health finance 10 10

Sub-national government administration 10 10

Theme Code (as % of total Bank financing)

Administrative and civil service reform 33 33

Decentralization 17 17

Health system performance 33 33

Law reform 17 17 E. Bank Staff

Positions At ICR At Approval

Vice President: Philippe H. Le Houerou Johannes F. Linn

Country Director: Klaus Rohland Julian F. Schweitzer

Sector Manager: Abdo S. Yazbeck Armin H. Fidler

Project Team Leader: Anne Margreth Bakilana Olusoji O. Adeyi

ICR Team Leader: Anne Margreth Bakilana

ICR Primary Author: Anne Margreth Bakilana F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document) The development objectives of the project are to: (i) establish a system of Federal regulation required for effective governance and management of the health system; (ii) develop and implement strategic approaches to health sector reform in selected regions;

iii

(iii) strengthen the institutional capacity of the MOH as a Federal executive agency; (iv) draw lessons from the implementation of regional programs and disseminate them to other regions; and (v) develop and implement an efficient scheme of restructuring of the health system, with emphasis on increased access, quality and efficiency of health services that will benefit from direct investment under the project. Revised Project Development Objectives (as approved by original approving authority) Project development objectives remained the same. (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Development and approval by the MOHSD of new clinical evidence-based protocols.

Value quantitative or Qualitative)

N/A

All 230 standards and protocols approved and enacted by MOSD.

The MHSD developed and approved: 88 stds for specialized care provision; 40 protocols for medical care provision; 310 standards for hi-tech health services provision; 26 stds for spa and recreation care; 46 stds for emergency care.

Date achieved 12/31/2002 06/30/2008 04/30/2009 Comments (incl. % achievement)

This target was achieved.

Indicator 2 : Approval by the MHSD of new standardization and licensing criteria and procedures.


N/A.

Regulatory and licensing tools in the health system developed.

RF Government Resolution of January 22, 2007, No. 30 #On approval of provisions for licensing in the health sector# (version 10.03.2006.)

iv

Date achieved 12/31/2002 06/30/2008 04/30/2009

Comments (incl. % achievement)

4 others approved, including Order of Roszdravnadzor 3/14/2005, #500-PR/05 #Approval of establishment by the subjects of the RF of committees for licensing in health care, pharma activities and activities related to drugs and psychotropic substances.

Indicator 3 : Development of national and regional models of finance.


N/A National and regional models of finance developed.

Models and relevant software were developed software installed in pilot regions health authorities, MOHSD, Roszdrav TsNIIOIZ. Methodical recommendations and training aids prepared for introduction of national and regional health accounts.


Target was achieved.

Indicator 4 : Development and approval of documents for rational use of human resources at the regional level.


N/A

Documents on rational use of human resources at the regional level developed and approved.

Development of methodologies for measuring labor intensiveness of activities including standards of medical services undertaken by medical workers.



Indicator 5 : Development and approval of health workers payment system at the regional level.


N/A

Health workers payment system at the regional level developed and approved.

For example the following regulatory documents of the federal level was adopted:

v

RF Government Resolution of August 5, 2008, No. 583 #On introduction of new systems of payment of workers of federal budget institutions and federal state bodies";


This goal was achieved.

Indicator 6 : Number of health facilities that meet federal standards of medical care provision


N/A

Health facilities meet federal standards of medical care provision.

Voronezh: 14 federal disease mngt protocols, 82 federal outpatient care stds and 58 inpatient disease mngt stds adapted to regional conditions. Chuvashia: 100 federal outpatient care stds and 120 federal inpatient disease mngt adapted.


This aim was achieved.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Decrease by 10% in the average length of stay in hospitals in pilot regions.

Value (quantitative or Qualitative)

Voronezh: 13.5 (CFO 15.3) Chuvashia: 13.2 (PFO 14.4) Russia: 14.7

10% reduction


Date achieved 12/31/2002 06/30/2008 04/30/2009

vi

Comments (incl. % achievement)

The 10% decrease target was met in Voronezh and in Chuvashia. In both cases the length of stay is lower than of Russia overall.

Indicator 2 : Increase in the percentage of daycare beds out of the total hospital beds in pilot regions.


Total number of inpatient beds: Voronezh: 24152 Chuvash: 11339 Day Care Beds: Voronezh: 2229 Chuvashia: 2043 % of day care beds: Voronezh: 9.23 Chuvashia: 13.3

N/A



The increase in number and percentage of day care beds as a proportion of inpatient beds has been achieved.

Indicator 3 : Reduction of the number of emergency/ ambulance calls (shifting to the primary care), per 1000 pop.


Voronezh: 291.5 Chuvashia:263.5

N/A Voronezh: 321.2 Chuvashia: 255.0


The indicator shows a reduction in Chuvashia and an increase in Voronezh.

Indicator 4 : Reduction of the hospitalization number: i) Number of visits to outpatient facilities (per person); and ii) Number of hospitalizations (per 100 persons).


i)Voronezh: 9.3 Chuvashia: 10.8 ii)Voronezh: 24.4 Chuvashia: 23.8

N/A

i) Voronezh: 9.4 Chuvashia:11.2 ii)Voronezh: 23.8 Chuvashia: 22.8


Indicator 5 : Increase in surgeries in day care hospitals as proportion of total number of surgical operations.



N/A Voronezh: 60.1 Chuvashia:19.9

Date achieved 12/31/2002 06/30/2008 04/30/2009 Comments Indicator shows a large shift in use of day care hospitals for surgical operations.

vii

(incl. % achievement)

G. Ratings of Project Performance in ISRs

No. Date ISR Archived

DO IP Actual

Disbursements (USD millions)

1 04/23/2003 Satisfactory Satisfactory 0.00 2 07/18/2003 Satisfactory Satisfactory 0.00 3 12/22/2003 Satisfactory Satisfactory 0.50 4 01/12/2004 Satisfactory Satisfactory 0.50 5 04/12/2004 Satisfactory Satisfactory 0.64 6 10/19/2004 Satisfactory Satisfactory 0.77 7 01/07/2005 Satisfactory Satisfactory 0.85 8 05/22/2005 Moderately Satisfactory Moderately Satisfactory 0.93 9 02/24/2006 Moderately Satisfactory Moderately Satisfactory 3.37

10 12/28/2006 Satisfactory Satisfactory 8.29 11 07/23/2007 Satisfactory Satisfactory 13.34 12 12/09/2007 Satisfactory Satisfactory 16.07 13 02/28/2008 Satisfactory Satisfactory 17.15 14 10/13/2008 Satisfactory Satisfactory 24.73 15 04/30/2009 Satisfactory Satisfactory 27.34

H. Restructuring (if any) Not Applicable

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal At the time of project appraisal, Russia’s macroeconomic environment had seen significant improvement following the 1998 economic crisis. As a result of sound macroeconomic management and high energy and commodity prices on world markets, the country boasted a budget surplus and a strong current account position. The country had developed a program of legislative measures to tackle social and economic challenges, including deregulation, civil service modernization, public administration reform, fiscal federalism, legal and judicial reform, and restructuring of infrastructure and reforming of the social sectors. Economic growth and strong current account, as well as signs of recovering poverty rates and reduced inequality had changed the nature of the Government's demand for external assistance. This meant that large-scale lending for budget and balance-of-payments support was no longer a priority; rather, in order to implement the reform agenda, there was increased demand for highly skilled and narrowly targeted advisory services in specific areas of institutional design at regional and municipal levels where the main implementation challenges were located. This period was also accompanied by changes in the structure and functions of state authorities, ownership relations, financing and crediting policy, distribution of powers among the various levels of federal administration and local administrative bodies. A number of major changes were also taking place in the health sector that had become highly decentralized as provided in the two major laws ("Fundamentals of legislation in public health care in the Russian Federation" and “The Russian Federation Law On public medical insurance in the Russian Federation"). During the 1990s, Russia had introduced a number of measures to improve efficiency in the health sector and to expand sources of financing. A number of regions had experimented with different types of provider payment systems including those determined by the level or type of service (e.g., capitation for primary care providers, fee-for-service for outpatient specialist care, case-based payment for hospitals) rather than by level of inputs (e.g., beds or staff). These experiments had not been well documented or fully evaluated though the conclusions were that their successes had been mixed. The concept of family medicine had been introduced and in 1992 the MOH had issued a decree calling for expansion of the role of primary care in the health system and instituting family medicine as the foundation for primary care. Implementation of the family medicine program had been slow, and the MOH was concerned that the substitution of inpatient care by primary care was not taking place as expected. In 1991 and 1993, Russia adopted a series of laws introducing payroll-tax financed mandatory health insurance (MHI) to supplement public budgets for health. This involved the establishment of a Federal Health Insurance Fund (FHIF) to oversee the system country-wide, Territorial Health Insurance Funds (THIFs) to implement the system at the regional level, and health insurance organizations (public and private) to receive capitation payments from the THIF on behalf of consumers and, in turn, to purchase services from

2

providers on their behalf. The implementation of MHI had not generated the expected favorable results: i) Its introduction actually led to the erosion of local budgetary allocations for health; ii) Financing, purchasing, and payment arrangements became more complicated with the introduction of Health Insurance Funds and health insurance organizations; and iii) Since each region had adopted its own approach to the introduction of MHI, there was no consistency across regions and no transparency with respect to health care entitlements. At the time of the project preparation, a number of other challenges faced the health sector, including: i. Fragmentation and poor quality of health services. Various restructuring efforts

over past decades had attempted to shift the balance of care provided from specialized inpatient care to outpatient services at the primary level. Implementation had been slow, partly because of ineffective interactions between the Federal Ministry of Health and the rest of the health care delivery system, lack of standardized global best practice training and accreditation of family doctors, and failures to reallocate resources from other parts of the healthcare system, towards the improvement of primary care.

ii. An inequitable health finance regime. The health financing regime was fragmented,

confusing, failing to provide incentives and continuity of funding for high quality and efficient services and did not offer adequate financial protection to consumers. Russia’s regional health departments (oblast health departments) received funds top down from Moscow through the oblast finance departments. The money was allocated across municipalities and rayons as line item budgets using input-based formulation (e.g. number- of beds or staff), which did not allow use of funds across lines or across budget years resulting in less innovation and little responsiveness at the local level. A second source of funding came from the regional health insurance fund, which paid for a "basic benefit package" of primary and secondary care services allocated typically on a service-by-service basis, according to a payment formula which differed by region. A third source of money came from the local governments which typically subsidized utilities and special equipment purchases. A fourth source of money - perhaps the most important - was the patient out-of-pocket spending, estimated at over half of all spending (54%) suggesting risks of impoverishment for theoretically "free" health care services and drugs. Per capita spending variations for the public sector were large across the 89 regions, with rural-urban disparities within regions. While decentralization had allowed regions to take a more active role, the relationship between regional and federal levels remained complex and some regions still expected guidance from the federal level. The majority of regions lacked the capacity to effectively manage health resources or develop appropriate reform strategies.

iii. Financially unsustainable health care system. The sector that was characterized by specialized personnel mostly serving the large hospital sub sector, which limited the possibility for increasing public or private financing. In addition to decentralization, the sector had started to implement reforms to address discrepancies between the low

3

level of resources and the excessive infrastructure through the introduction of more cost-effective and better quality service based on clinical evidence. Still, the reform process faced obstacles because federal level norms set in the 1990s were outdated and was in the form of recommendations rather than guidelines.

iv. Poor health outcomes. The main health problems which threatened economic growth and productivity in Russia were non communicable illnesses and premature deaths among the adult population. The standardized death rate from cardiovascular diseases (for ages 0-64, per 100,000 population) was much higher (211) than the European Union (EU) average of 53. Between 1987 and 1994, life expectancy in Russia had seen a sharp decline, a trend that temporarily reversed in the years between 1994 and 1998, mainly due to a decrease in the death rate among middle-aged adults.

v. Limited Capacity of the Ministry of Health and the sector as a whole. At the federal level, the capacity to carry out the regulatory, policy making guidelines as well as economic and financial analysis was not effectively used. The MOH lacked financial and technical and regulatory tools to influence the regions to undertake consistent reforms countrywide. In many ways, the MOH had been bypassed by the more reform-minded regions. At the facility level, managers were mostly physicians without management training, and were not appropriately trained to plan and manage physical, human and financial resources.

Rationale for Bank Assistance The HRIP was prepared to support a new government strategy, "Concept for Development of Health Care and Medical Science in the Russian Federation" that was adopted in November 1997. The central theme of the strategy was the urgent need to improve the efficiency of health care services by: (i) rationalizing the structure of health services at all levels - with an increased role for preventive measures, primary care services and more efficient use of diagnostic and hospital resources – and improving the interface among all levels and types of care; (ii) improving financial and resource management and introducing appropriate incentives for providers; and (iii) improving sector-wide governance and clarifying governance relationships between different levels of government - Federal, regional, municipal, and district. The strategy was to be led by the Ministry of Health while preserving the autonomy of regions and other local authorities. The HRIP aimed to support the Government’s strategy which focused on the systemic challenges facing the sector. The Project fitted clearly within the context of the Bank’s Country Assistance Strategy (CAS) of May 2002 and fell under the sub-objective of "Improving Health Status, Services and Finance" in the CAS Program Matrix. More broadly, the Project was consistent with the CAS priorities of supporting evidence-based policies including supporting analytical work aiming to improve financial management and resource allocation in the health care system at the federal and regional levels by (i) increasing the financial sustainability by shifting care to outpatient and generalist care,

4

(ii) developing sound and national and regional financing models; and (iii) piloting new approaches for contracting and financing health care personnel.

1.2 Original Project Development Objectives (PDO) and Key Indicators The development objectives of the project were to: (i) design a system of Federal regulation required for effective governance and management of the health system; (ii) develop and implement strategic approaches to health sector reform in selected regions; (iii) strengthen the institutional capacity of the MOH as a Federal executive agency; (iv) draw lessons from the implementation of regional programs and disseminate them to other regions; and (v) develop and implement an efficient scheme of restructuring of the health system, with emphasis on increased access, quality and efficiency of health services that will benefit from direct investment under the project.

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification The Project’s objectives remained unchanged throughout its implementation.

1.4 Main Beneficiaries The PAD identified the following groups as the main beneficiaries: 1. Government, MOH, regional administrations and health authorities in start-up regions, and the Duma, due to validation of key reform concepts; new national norms and standards that would promote efficiency and quality of care; acquired institutional capacity and tools to carry out policy development; sector-wide business planning; gained exposure to new ideas and alternative policy options and policies; additional involvement of multidisciplinary specialists in the sector; and reduction of region-wide financing deficits. 2. Managers of health facilities, due to acquired professional skills and tools for financial and human resource management. 3. Health care personnel, due to acquired skills to improve the quality of care, reduced wage arrears because of efficient use of physical capital and possibly improve remuneration levels, and increased incentives for good performers as performance-based contracting and remuneration is introduced. 4. Population in start-up regions, due to timely diagnosis and treatment, shorter hospital stays, a more efficient health system, and improved managed treatment. 5. Society at large, due to improved policy environment, better leadership from the MOH resulting from access to clinical and managerial tools and training programs.

5

1.5 Original Components The total project cost was estimated at US$ 41.21 million (including IBRD financing of US$30 million and US$11.21 million Borrower financing). The Project had four components as follows: Component 1: Rationalization of Health Services (US$33.92 million, 82.3 percent of total project cost). This component aimed to develop (at the Federal level) and implement (at the Regional level) modern health services and ensure health system’s long term financial sustainability by shifting from in-patient to out-patient care and from specialist to primary health care using modern guidelines. These changes aimed to improve access to health services and quality of care. The component aimed to support the development of: (i) strategic approaches to comprehensive restructuring of health service delivery; (ii) new norms and standards for health service institutions based on the experience of pilot program; (iii) guidelines and best practice manuals for implementation; and (iv) new quality and outcome oriented criteria for licensing and accreditation of health providers. At the regional level, the component aimed to support the implementation of guidelines specified above. Component 2: Strengthening of Financial and Economic Management (US$2.88 million, 7 percent of total project cost). This component aimed to: (i) develop analytical tools, including national and regional financing models and equalization formula, and apply the models to the costing of health benefit packages and regional restructuring plans; (ii) develop training programs in economic and financial analysis; (iii) develop best practice methods and manuals for financial management and business planning; (iv) develop institutional capacity to carry out technology assessment; and (v) develop new approaches to contracting and paying health workers. Component 3: Improving Policy and Governance (US$0.64 million, 1.6 percent of total project cost). The component was to support the creation of necessary conditions for designing and implementing health reforms in pilot regions by providing the appropriate legal framework and regulations at federal level. Component 4: Project Management (US$3.77 million, 9.1 percent of total project costs). This component was to support the implementation of the main project components and finance the management of the project by the Russia Health Care Foundation (RHCF) in terms of technical coordination, procurement, disbursement and financial management. The component was to finance the operating costs of RHCF for the duration of five years, including staff salaries, training of staff, information technologies and office equipment, and other costs of RHCF operations, as well as project audits, procurement assistance, public information activities related to the project, and project monitoring and evaluation.

6

1.6 Revised Components Project components remained the same throughout implementation.

1.7 Other significant changes Project design, scope and scale, implementation arrangements and funding allocations remained the same throughout implementation.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry This Project was designed as a knowledge and process intensive initiative in order to build necessary skills to tackle systemic issues in the health sector which was seen as crucial for the achievement of better health outcomes for the population. The Project was designed to enable the Government of the Russian Federation to speed up county-wide reforms of its health system on the basis of Federal guidelines that had been tested at the local level, with a better skilled Ministry of Health. The rationale to tackle systemic issues was sound; and appropriately responded to the needs of a sector which relied heavily on hospital care and not enough on primary health care and prevention. The design of the project suited the capacity of the Government to implement, and the institutional arrangements that had been identified were suited to an environment that required a thorough consultation process for some of the reforms that were being introduced. This was a third lending operation in Russia, according to project data its preparation begun in FY08. A much larger operation US$500 was originally discussed as was the issue of having more pilot regions covered; but overtime the size of the project was reduced as the project areas of focus emerged. The HRIP was not designed as a successor to Health Reform Pilot Project (HRPP) that had been implemented in Tver and Kaluga regions and which closed April 30, 2004. The design did however draw on lessons learnt, including the fact that reform efforts in the regions were constrained by outdated national-level norms and standards, the federal MOH’s lack of financial and institutional resources to lead reform efforts in the country, and the need for the federal MOH to direct reforms in a decentralized environment. The Project also took into account lessons of implementing other health projects in the ECA region including the fact that health sector reform is a lengthy and political process, the importance of focusing on both technical and institutional aspects of reform, and the importance of paying attention to the political economy of the reform through an effective communications strategy to lawmakers, the medical community and the public. The project design was synergistic with projects supported by other key development agencies. The Project team built on European Union funded TACIS project "Support to Public Health Management", whose focus was on strategic planning at the federal level. The project design was also a result of close collaboration with Canadian CIDA (in Chuvashia), as well as with DFID, WHO and the Soros Foundation. The project

7

preparation, design and quality at entry were coordinated with activities undertaken through the Canadian CIDA project in Chuvash Republic. For example, CIDA had for a number of years supported the introduction and strengthening of the family medicine model as well as health promotion initiatives. CIDA and the Bank collaborated on the development of the Terms of Reference; and CIDA funded the preparation of background concept papers on health system challenges and restructuring plans which formed the background for the Strategic Plans that the HRIP later supported. Excellent collaboration between CIDA and the Bank continued during implementation of the HRIP.

2.2 Implementation The HRIP was by and large implemented according to plan, with no restructuring or major changes, with most project activities completed by the time the Project closed. Implementation did start very slowly, however. After the Project was approved in March 2003, and despite the delay in effectiveness (delayed until December 2003) preparations for implementation by the regions went ahead demonstrating high level of commitment to the Project, a level of commitment that remained throughout project implementation. Progress towards achievement of project objectives was rated satisfactory for most of project implementation though implementation progress stalled following the extensive Government reforms (in Spring of 2004) which created a new Ministry of Health and Social Development and hence the transfer of the responsibility for project implementation to the MOHSD which required an amendment to the Loan Agreement. Implementation progress picked up once the amendment was signed and counterpart funding made available. The project rating was satisfactory for implementation progress and progress towards achievement of objectives for most the implementation period apart from the period between 2005 and 2006 when it was rated moderately satisfactory due to administrative reforms in the organizational structure of the Russian Government. This project was prepared in full recognition of the often challenging political economy of health reforms. One area where project activities encountered challenging views and opposing opinions was whether the adoption of the GP model of care was to marginalize and make irrelevant specialized care in areas such as pediatrics. To advance with the introduction of this model, project activities supported the retraining of specialists willing to become GPs along with the training of a new cadre of GPs. This approach meant in practice that while GPs are now responsible for providing care to all groups at the community level, specialized care is also provided by pediatricians and other specialized physicians on a need basis following new treatment guidelines at the policlinic level.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization The monitoring and evaluation framework was adequately designed to measure progress toward the achievement of project objectives. The framework did carry enough indicators to track both the implementation of activities, results from implementation as well as health system improvements that could be linked to reforms supported by the project. The monitoring framework did contain a sufficient number as well as range of the necessary baseline indicators covering key areas of project activities. Indicators were indeed

8

sensitive to project activities and were relevant to what the project was attempting to achieve. Indicators were easy to collect and the sources of the necessary data had been properly identified right at the beginning. The monitoring framework was amended through the exchange of letters in 2007. The changes to the framework were minor, most indicators remained the same, and some indicators were refined, for e.g. the indicator ‘Percent increase in nursing home utilization’ was rephrased as ‘Increased number of nursing beds’. In addition, a few indicators were dropped. This evaluation uses the revised monitoring framework. A key limitation of the monitoring framework is that it was not designed to enable the separation of the attribution of achievements of development outcomes between the Bank funded activities and those that were due to Borrower efforts. There are several reasons why this is the case. First, the HRIP was an institutional capacity building project, one that invested heavily in building the capacity of the MOH at both the Federal and Regional level to be a better steward of the sector. It is very difficult to measure the impact of capacity building projects let alone to be able to attribute any impacts to Bank funded efforts. Second, the HRIP focused primarily on systemic issues, some of which are influenced by factors outside the control of project activities including already ongoing efforts (in the case of the Chuvash Republic) as well as the environment where a number of complementary Government led efforts were going on at the same time. The project team did acknowledge that it would have been very difficult to separate attribution in achievement of better health outcomes to Bank funded activities as opposed to all other determinants of health outcomes. Same could be said of the difficulty in attributing some of the systemic outcomes to Bank funding given that the Borrower was also investing a considerable sum of financing on activities complementary to the project. A methodology designed to enable better understanding of the contribution of Project financing should have been considered.

Implementation The Russia Health Care Foundation, which was the overall manager of project implementation, together with the Ministries of Health of Voronezh Oblast and the Chuvash Republic were responsible for collecting and reporting on project monitoring indicators. The designed system successfully collected baseline data as well as periodically collecting data during implementation and after project closed. For some indicators the system was successful in capturing comparator data from geographic areas that were affected by project interventions and those that were not as well as the national averages. Utilization The Government was successful in establishing a system to collect project monitoring indicators that were useful in monitoring progress in achievement of project objectives. Most importantly, the Project was successful in instilling a culture of evidence, whereby data, such that on number of beds, ambulance response times, and quality of care

9

provided by physicians are used to make decisions and to further efforts to improve efficiency and quality of care. The Government has been successful in periodically producing indicators from a system developed under the Project and consistently used monitoring indictors in its annual reports. One of the greatest achievements of the Project has been the successful introduction and acceptance of systems of measuring quality, including patient satisfaction surveys, and indicators of quality in the emergency ambulance services, which are routinely used for decision making by management, including creation of efficiency performance ranks by region. This data has also been critical in the introduction of a performance-based system of paying physicians whose salaries are now based on the quality of services provided for which data on quality indicators is a must. The one shortcoming in terms of the project monitoring framework is that the data on project achievements by levels of poverty was also supposed to be collected in order to measure achievement in reaching the poorest segments of the population; this however, was not done, probably due to the fact that this would require additional data collection. It is therefore not possible to document how and to what extent the Project directly impacted issues such as access and quality of health services for the poor in comparison to the non-poor, though the project activities did focus on poor areas of these regions.

2.4 Safeguard and Fiduciary Compliance

This Project was classified as a “C” for environmental category, it did not finance any civil works or goods, and it did not deal with disposal of medical waste and so this safeguard was not triggered. There were no other safeguard concerns. There were no concerns with regards to financial management, no delays were reported with regards to reporting, reconciliation of accounts and all audits were clear. There were no issues reported with regards to failure to comply with Bank policy and procedural requirements for procurement or disbursement.

2.5 Post-completion Operation/Next Phase The HRIP was an institutional reform and capacity building project designed to support a reform program that had been identified by the Government, and which had been rolled out over decades in recognition of the structural inadequacies of the health care system. Analysis of achievement of the Project objective shows that the institutional capacity of the both the Federal and Regional MOHs has been built and the federal MOH has acquired the necessary skills and capacity to be a better steward of the sector. An example would be the capacity to collect and use data in National Health Accounts to make decisions. A noteworthy aspect of the Project is that at the regional level, implementation of project activities was undertaken by regional MOH staffs that are still part of the MOH cadre of staff. The regional project implementation units were more of coordinating units and so there has been no need to make specific transition arrangements as far as continuity is concerned.

10

Demand for borrowing remains limited and currently there are no plans for new loans to support the health sector. Instead, the Bank continues to engage with the country in the health sector through: (i) Fee for Services Advisory Products, particularly narrowly targeted advisory services in specific areas of institutional design combined with international best-practice experience; (ii) sub-national lending at the regional and municipal levels where the main implementation challenges are located. At the end of March 2009, a US$7 million sub-national loan agreement for the Chuvash Republic was approved. This is a significant milestone as the Bank Group’s first sub-national project without sovereign guarantees in the human development sector globally. The Chuvash Republic health project builds on the implementation of the HRIP, and will support the implementation of the second phase of the health system restructuring program in the region. The program concentrates on building service delivery capacity in the Chuvash Republic to deal with cardiovascular diseases and trauma, leading killers and main causes of ill health and disability in the country. The experience and lessons learnt during implementation are effectively used by other regions, especially those practical tools acquired through the reform process in the two pilot regions.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation This evaluation found that the objectives of the Project were consistent with the objectives of the 2002 Country Assistance Strategy (CAS) and remain relevant to the current CAS, current development priorities in the sector, and global priorities. The Project was coherent with the 2002 CAS objective of improving the health status, services and finance in the sector. More broadly, the Project was also consistent with the CAS priorities of supporting evidence-based policies, improving financial management and resource allocation in the health care system at the federal and regional levels by: (i) increasing the financial sustainability by shifting to outpatient and generalist care; (ii) developing sound and national and regional financing systems; and (iii) piloting new approaches for contacting and financing health care workers. The 2002 CAS had put a strong emphasis on the need to strengthen federation-wide functions of: (i) improving equity and efficiency through legislation, standard setting, supervision and support, (ii) high-impact advisory services to reach a consensus on large-scale programs of disease control and health promotion, and (iii) leveraging local and international expertise to reform the public health and disease control system. Overall the decision to focus on systemic challenges facing the sector was sound because the Project aimed to meet key preconditions for an efficient and equitable system. The Project was designed to improve how the health sector is governed by clarifying relationships among key players and institutions and facilitating the introduction of evidence-based health services. The early stages of project preparation had envisaged a larger project. However a final decision to design a smaller project focusing on systemic challenges in the sector as well as testing of new strategic choices was arrived at, with the assumption that a follow-up operation would follow the successful completion of the

11

Project or that the Government would finance any further roll out of reforms using its own resources.

3.2 Achievement of Project Development Objectives Overall, this evaluation finds that the achievement of project objectives is satisfactory.The evaluation was based on analyses of indicators and data linked to the five items listed as objectives of the Project in the PAD, as well as an evaluation of how the activities supported by the Project are linked to the achievement of objectives. Since Project objective number ii) and v) in the PAD are closely linked, and their indicators are also related, the analysis of the achievement of these two objectives is done in tandem. Objective 1. The project developed a system of federal regulation required for effective governance and management of the health system as evidenced by the body of new evidence-based clinical guidelines developed and issued by Ministry of Health and Social Development (MOHSD).

The project supported the development and introduction of standards and protocols in specialized care, general care provision, hi-tech health services provision, spa and recreation care and standards in emergency care provision. The guidelines provide enough flexibility for region specific disease prevalence, geographic conditions, resource constraints and other circumstances making them adaptable to regions with different circumstances. In addition, the Project supported the development of guidelines for human resources remuneration, as well as the development of national and regional models of finance. These federal level guidelines were developed to international best practice standards which were then introduced in the two regions of Chuvashia and Voronezh. The result of the development, testing and adoption of these guidelines and standards is as follows: Voronezh oblast adopted 14 federal disease management protocols, 82 federal outpatient care standards and 58 inpatient disease management standards adapted to regional conditions. Chuvashia Republic adopted 100 federal outpatient care standards and 120 federal inpatient disease management standards adapted to regional conditions. In addition, various Laws and Decrees were issued, necessary for defining the legal basis for undertaking the changes in the health system. All these standards and guidelines and Laws were produced by the Federal Ministry of Health, demonstrating that the project was successful in strengthening the institutional capacity of the MOH as a Federal executive agency which is one objective of the project. The Guidelines and Laws now can be used by other regions in the Federation, adapted to regional realities and conditions.

Objective 2 and 5. This evaluation found that the project successfully supported the development and implementation of an efficient scheme of restructuring of the health system in Chuvash Republic and in Voronezh Oblast. The project satisfactorily developed and implemented an efficient scheme of restructuring of the health system, with emphasis on increased access, quality and efficiency of health services that benefitted from direct investment under the project.

12

a) Successfully intensified and accelerated the restructuring of Primary Health

Care Delivery.

Chuvash Republic was successful in expanding the provision of primary health care through the expansion of the General Practice (GP) model of care. The first 18 GP offices in Chuvashia were opened in 2001-2002, with financial support from the Canadian International Development Agency. It was during that time that training of health workers; health care standards for equipment in GP offices were developed. In addition to training GPs, the Project provided necessary equipment for general practitioner practices, and supported the development of processes necessary for the introduction of economic and other incentives to attract physicians to become GPs through training and retraining of physicians and attracting those with higher medical education. Indicators also show that the growth of the GP model was successful in shifting a proportion of health care services onto outpatient care and further developed other care delivery arrangements such as day and home care, introduced home nursing services, and expanded outpatient surgery and semi-inpatient rehabilitation services. Overall, the reform process also emphasized the importance of prevention in primary care by encouraging and providing incentives for health workers to provide preventive services.

Indicators show that Chuvash Republic was successful in expanding the proportion of services that was provided by GPs. About 54 percent of Chuvashia’s population now receives care from a GP; this is up from 13.1 percent in 2003. GPs are accessed by an even higher percent of rural residents (60 percent). Indicators also show that the proportion of GPs and of nurses that were certified has also increased to more than 96 percent for both GPs and nurses. The result of the intensification of the GP model of care has gradually reduced the proportion of cases that was referred to specialists, data show that the percent of referrals to specialists declined from 8.8 percent to 2.4 percent in between 2002 and 2007. Data show that during the period 2001-2007, the number of referrals to specialists has decreased as GPs increasingly handle some of the cases previously referred to specialists.

Reform of primary health care delivery has also contributed to an increase in the use of outpatient care services and has hence reduced the larger volumes that were handled by inpatient facilities and emergency services. The project supported investments in outpatient care facilities such as day care, outpatient surgery and home based care. The reliance on day care facilities has increased, such that the number of beds in day care facilities increased by 19.4 percent (to 2, 440 day care beds at the beginning of 2008 compared to 2,043 beds in 2002). Home care facilities have been also been expanded, in 2007, they were opened in 57 facilities and 11 455 patients received care in such facilities compared to only 4 049 patients in 2002. Though data for periods before the project are not available and so it is not possible to say what the trend was like before and whether this is purely an impact of the project, the trends in the number of patients treated in alternative care delivery settings has been increasing as has been the proportion of outpatient surgery in the total number of surgeries which has increased by 3.7 percentage points.

13

The Project supported the purchase of equipment for five outpatient surgery centres, which injected up-to-date medical technologies in the outpatient/policlinic units of these surgery centres. In addition to activities directly supported under the Project, the Chuvashia regional primary health care reform plan also focused on increasing preventive services. Some of the activities included the reorganisation of chronic case management including an increase in screening and monitoring; and while these cannot be attributed to the project, they are a demonstration of the complementarities of ongoing efforts that were funded by the Borrower and by other partners. Data show that over the recent five years, the number of continuously monitored patients increased by 9.7 percent to reach 428.7 per 1,000 population in 2007; and in 2007, the share of preventive services amounted to about 40 percent of all visits.

Under its Strategic Plan the Voronezh oblast successfully carried out a number of activities to transform primary care into a GP model of care. By the end of 2008, 346 general practitioner offices had been set up compared to only 7 in 2002. These 346 GP offices include 268 rural offices and 78 offices in the city of Voronezh. Three to five beds day-care hospitals are organized in premises of GP offices to provide treatment for example to post-hospital rehabilitation of patients. Since 2005, the Project supported the procurement of equipment for GP workplaces while oblast budget allocated funds for procurement of medical equipment for doctor outpatient facilities and GP offices as well as the procurement of specialized vehicles for rural doctor outpatient facilities of CRH and health facilities of the city of Voronezh.

As was the case in Chuvash Republic, data show that the composition of primary care services provided by GPs in Voronezh has also changed. The proportion of various classes of diseases which were traditionally referred to narrow specialists has been declining between 2003 and 2008. Data show that the proportion of cases of neurology that were referred by GPs to specialists has declined from over 85 percent to less than 40 percent in 2008. Similar trends were observed in the other groups of cases. Other data show that in GP coverage areas, the number of emergency ambulance calls and referrals to 24-hour hospitals has also declined.

The success of the primary health care reform process is partly due to emphasis that has been put to establish and support centers of learning for GPs such as the Voronezh State Medical Academy and the Family Medicine Department of St.Petersburg Medical Academy of Post-Graduate Education. In addition, GPs received training abroad in Sweden, Denmark and the Netherlands, for example. Training of nurses accompanied the training of GPs; training was provided by the Voronezh Medical College. A total of 945 nurses were trained in GP practice. The oblast set up GP training centers in the Voronezh polyclinic No. 7 and Oblast Clinical Hospital No. 1 for practical training of GPs and nurses.

b) Successful restructuring of the inpatient care delivery model.

In the Chuvash Republic, the restructuring of the inpatient care model has been a success. This was achieved through the reorganization of the network of hospitals by establishing inter-rayon medical centres, making the stock of inpatient beds compliant with the norms set in the basic package of health care benefits, differentiating bed stock

14

by treatment intensity, dividing volumes of health care by care delivery level, reducing the average length of stay, and ensuring efficient use of expensive highly specialized care beds. For the last five years, Chuvashia has consistently worked to reduce its bed stock to meet the set norms and during this period about 10 percent (1,365 beds) of the bed stock above the norms was reduced. At the end of 2007, the availability of hospital beds was 91.8 per 10,000 and during the first half of 2008, additional bed stock was reorganisation led to a reduction to 84.1 beds per 10,000 population. Reforms, such as the introduction of new organisation structures and updated medical technologies led to more intensive use of beds resulting in shorter average length of stay from 13.2 days in 2003 to 12.2 days in 2007 (the average for the Russian Federation is 13.6 days and 13.3 days for the Volga Federal District in 2006), and increased hospital bed occupancy rate from 312.3 days in 2002 to 314.5 days in 2007.

In Voronezh, data show that in addition to introducing the GP model of care, the Project successfully supported the development of outpatient care service and expanded the scope of care provided. Data show, the Voronezh Oblast saw the number of day-care hospitals/ outpatient facilities grow to 17.6 percent of the total number of hospital beds. The number of patients treated in day care facilities increased from 67,046 (2002) to 119,761 (2007) and to 86,512 (nine months of 2008). Day-care bed occupancy rate was 288 days in 2002 and 299.4 days in 2007. Correspondingly, the oblast’s inpatient facilities reduced the number of day-care beds from 829 (2002) to 99 (2007) in line with new guidelines at the Oblast level. Accompanying these changes, 24-hour emergency care units were set up in 50 district hospitals and outpatient facilities; in 2008, these units provided care to over 59,100 residents of the oblast rayons.

Rationalization of inpatient care services led to a reduction of the number of inpatient facilities in the Voronezh oblast. In the period 2002 - 2008 the number of inpatient facilities declined from 163 to 124 and the number of inpatient beds declined from 24152 in 2002 to 19436 in 2008. The average length of stay in hospitals has also declined from 13.5 days in 2002 to 11.8 days in 2008 as modern medical technologies and intensification of the diagnostic and treatment processes were introduced. Data for the city of Voronezh shows that the average stay in municipal hospitals has also declined from 14.1 days in 2002 to 10.1 days in 2007 (compared to 13.6 in the Russian Federation); the average occupancy rate of a hospital bed declined from 320.2 in 2002 to 309.6 in 2007.

The Project successfully supported the reorganization of outpatient surgery centers in Voronezh by further strengthening outpatient facilities at municipal and oblast levels. Currently there are eight functioning outpatient surgery centers; and an additional seven outpatient surgery centers will also open in the oblast rayons. The Project supported the procurement of medical equipment for the outpatient surgery centers including video-laparoscopic stands, arthroscopies, anesthesia equipment and other medical equipment needed to expand surgery services in outpatient facilities. In the first nine months of 2008 when data is available, outpatient facilities accounted for 59.7 percent of surgeries. The outpatient surgery center in City Polyclinic No. 7 is also a training center for sixth-year students, interns and residents in outpatient surgery.

15

c) Successful improvement of the Management of the Information System.

The Project supported improvements in the management information systems for the primary health care system including the supply of up-to-date information technology hardware and software for emergency care. It also supported the development of an information system for the additional drug supply program for eligible groups and supported the introduction of the telemedicine system. The project supported the training of various types of specialists contributing to improvements in monitoring of compliance with health care standards that were developed and introduced under the project. The project also supported installation of an information system was installed in primary health care physicians and specialists offices who were trained to record the volumes of care provided to a patient from the primary care level and ensure compliance with the outpatient care standards; monitor the quality and completeness of compliance with approved standards based on the case-based records of provided services.

Reforms that were supported under the Project resulted in a number of improvements in services that were provided. For example, the Project supported Chuvashia’s efforts to reduce the incidence and effects of road traffic accidents. The Project also supported efforts to improve timeliness and to improve the quality of medical care provided to victims of road accidents through the supply of telecommunication equipment to the emergency/ ambulance departments. In addition to renewal of the ambulance fleet, a unified dispatch centre, using satellite geo-positioning systems was introduced. Average waiting time for an emergency team declined from 20.9 to 13.4 minutes in the first 9 months of 2008 making Chuvashia one of the top performers in this aspect. Accordingly, the proportion of calls that were responded to by timely arrival of emergency care staff was 85.8 percent on average. This was achieved through the establishment of a single system for all emergency calls ensuring effective communication between emergency/ ambulance services, fire service, and utilities/ gas services.

With the support of the project Voronezh was also successful in improving the performance of its emergency care service. The Oblast’s emergency care service comprises an independent ambulance station in the City of Voronezh; 33 emergency care departments in the Central Rayon Hospital and 80 emergency rooms in district level facilities. Through the support of the Project 40 Standards for equipment and medicines were introduced and in addition communication equipment and modern vehicles were procured. The Priority National Health Project which is financed by the Federal Government was complementary to the Project’s efforts in this area by providing additional equipment to the 24 hour care services. Evidence on better performance is still mixed. The City of Voronezh seems to have better performance indicators as measured by the average number of calls per 1000 population (311.2 in 2007) probably attributed to better access to primary health care services compared to Oblast rayons data that show the average number of calls per 1,000 of population was 355.3 in 2007 and 240.6 in nine months of 2008 (compared to 307.9 in 2002); while in the oblast rayons this indicator is higher than the oblast average: 355.3 in 2007 (307.9 in 2002). On average 83 percent of all emergency calls were responded to by the arrival of an emergency team within the established norm of emergency ambulance service of 15 minutes.

16

d) Successfully contributing to improvements in the Financial Sustainability of the Health Care System

One of the many challenges that faced the sector was the existence of a large and very specialized hospital infrastructure and a cadre of medical personnel that was too hospital based; a number of the activities supported under the Project successfully contributed to improving the financial sustainability of the health care system by making it more efficient, through cost reductions. For example: (i) between 2002 and 2008, the number of more costly inpatient bed stock was reduced in Voronezh from 24,152 to 19,436, and less so in the Chuvash Republic from 11,339 to 11,172; and (ii) the average length of hospital stay in Chuvashia was reduced from 13.2 to 12.2 days between 2003 and 2008, and hospital bed occupancy rates increased from 312.3 to 314.5 days between 2002 and 2007. The project also supported activities that led to the successful reduction in the share of funding for more costly inpatient care while increasing that for more cost efficient primary health care.

With the support of the Project, Chuvashia was successful in transitioning the sector to a primarily single-channel system of health finance (pooling of funds from different sources) and hence contributing to less fragmentation of sector finances as well as increase in financing to the sector. Due to a number of factors, such as, iincreased contributions for non-working population, reallocation of funds from municipal budgets, the deficit of the Regional Benefit Package was reduced from 37.7 percent (as of the beginning of 2007) to 17.9 percent in 2008. This increase health sector financing channel financing through the MHI system made it possible to expand the list of health services expenditures for health facilities to include previously excluded items such as utility and asset maintenance services. This transition to a single-channel of MHI financing increased efficiency in resources management, it enabled the full restructuring of care provided by allowing the settlement of health facilities accounts on full tariffs.

In addition to the above, the Project successfully improved the capacity of the Ministry of Health to carry out economic and financial analysis. For example, a national health financing model was developed and is used to assess the impact of alternative reform policies on revenues and expenditures and on the fiscal balance of the health sector countrywide. The Project also supported the development of short and long-term training programs in financial and economic analysis and made them accessible to specialists and managers at the national and regional levels. The project supported the introduction of a system of health accounts, consisting of a matrix of social and economic indicators on aspects of the health system at the national or regional revel. Using such data it is possible for example to analyze the Chuvash regional health accounts, according to which the overall health expenditures funded from all sources of finance increased by 34.6 percent between 2002 and 2005. In Voronezh, various activities were implemented to improve finance and expenditure planning models in the oblast and specifically to improve availability and use of information necessary for sound assessment and use of financial resources in the sector. The Project set out to improve availability of analytical tools for analysis of various reform options; methods for evaluating impact and hence the selection

17

of options. One of such tools developed was the macroeconomic models of finance and expenditure planning that provides projections of proposed health reform outcomes at the federal and regional levels. The project team shared the experience of developing the models with various audiences domestically and internationally such as at the National Health Accounts for the CIS Workshop. In addition to the development of analytical tools, the Project successfully contributed to building the capacity in training in the field of economics. The Project supported the development of new university courses; and university curricula have been brought to the standards of leading universities. A federal health economics resource center was established in the Russian State University of Peoples’ Friendship (UPF) to provide training and to improve the quality of teaching. This center is part of UPF’s further and vocational education system that retrains and upgrades the qualifications of health care sector personnel in economics. This center is expected to continue to develop teaching of health economics targeted at trainers in health economics. Using additional resources obtained from the Federal Government, health care specialists were trained through the regional WBI organized flagship courses in health reform and sustainable financing that were held in: Cheboksary, Chuvash Republic with 41 participants from the health care sector, including 31 from the pilot regions, and in Voronezh, Voronezh oblast with 35 participants from the pilot regions. In addition, key health sector personnel, including chief physicians and economists working at health facilities have received training in basic financial management, cost accounting, financial accounting, budgeting and business planning. In addition to improving the efficiency of the system, the Project also supported the development and implementation of mechanisms that created incentives conducive to more efficient services by care providers. Some of the most critical changes in the provider payment mechanisms included: i) the linking of salary level to service-required qualification, rather than strictly on the qualifications of the individual. This meant that salaries are now based on the requirements and responsibilities of the position rather than strictly on the individual’s qualifications; and facility managers retain the discretion to adjust salary level to individuals’ qualifications. ii) Managers now have the discretion to create positions that are competitive in the labor market, a discretion that allows them to adjust coefficients such that particular wages for certain position can be more competitively priced in the regional labor market; iii) Workers would now earn incentive-based payments as these would cease to be automatically paid making this one of the tools for human resources management

Objective 3. The project was successful in strengthening the capacity of the Ministry of Health as a Federal Executive agency.

As was discussed in some of the section above, a significant proportion of the loan proceeds was earmarked for capacity building activities. This evaluation found that the Ministry of Health has been strengthened in the following ways:

18

• Strengthened skills in developing national norms and standards in efficiency and quality of care, for a list of norms and standards developed see Annex 2;

• The MOH and the sector as a whole has acquired institutional capacity as well as the tools to carry out policy development, sector-wide business planning, legislation and regulation, such as the National and Regional Health Accounts systems. The MOH capacity is strengthened in the area of financial and economic resources management and in addition two resource centers that participate in making decisions on key issues of health economics and organization have been set up.

• It has gained exposure to new ideas, and alternative policy options and policies through the various study tours; courses; collaborations with international organizations;

• The reform process that was supported by the Bank has opened up the involvement of multidisciplinary specialists in the sector including accountants, economists and financial management specialist all which will contribute to better management of the sector; under the project health facilities acquired financial managers. The project supported the training of specialists in legal issues in the sector; the skills were clearly well applied as evidenced by the new legal frameworks for example in human resource remunerations (new provider payment mechanisms), etc.

• Through the implementation of this project the MOH and the sector as a whole has acquired skills for comprehensive restructuring of the sector, and has developed procedures for reforms that were then used for the development and successful implementation of regional strategic restructuring plans which were successfully implemented in Chuvashia and Voronezh;

• In the case of Chuvashia, the relationships that have been built at the municipal level between the MOH; transport, education etc, have been instrumental in alleviating the profile of Chuvashia as a Health City and its efforts in improving health life styles.

Objective 4. The Project successfully drew lessons from the implementation of regional programs and disseminated them to other regions.

• The two pilot regions have become the go-to regions for other regions that wanted to learn about the experience of implementing comprehensive reforms in the sector. All the guidelines and laws produced during the reform process have been made available through a website from the MOHSD (TsNIIOIZ) and are accessible to all regions. The project team has been very active in disseminating the lessons learnt through its participation in various workshops and seminars, as well as participation in various working groups such as the one for the development of a federal health accounts model of the Coordination Council on state guarantees of free medical care.

• Other dissemination activities under the Project included a very well attended

HRIP closing conference that was held at the end of 2008 which drew great

19

interest from stakeholders in the health sector1 . The Conference drew the interest of senior political representatives from around the country, as well as from other international bodies2. The event presented key outputs from the Project including laws and guidelines, and protocols and procedures that were developed to implement reforms and which were available to all regions.

• Project inputs and initial experiences at the regional level were taken into

account for the preparation of the 2005-2007 Presidential Priority Health Project National Program. Both Chuvash Republic and Voronezh were beneficiaries of additional investments provided under this national program. In addition, the Chuvash Republic has also benefited from its participation in the Pilot Project implemented over the 2007-2008 period to improve the quality of services in the health care sector.

3.3 Efficiency An economic analysis to HRIP was not carried out for the ICR. The Russia HRIP was mainly an institutional capacity building project and, as mentioned in the PAD, it would not be appropriate to attribute changes in aggregate health outcomes to the Project alone. The Project served as a spring board for many of the changes that occurred, mostly by financing strong and relevant technical assistance to guide and support Government efforts. Additionally, the Project represented a financially small component of the overall budgetary outlays that financed the reform process. While the PAD attempted to undertake some economic analysis, it was not particularly extensive, and covered one region that was eventually replaced by another. While an economic analysis was not carried out data presented in this evaluation shows that there has been a positive trends in terms of greater efficiency in the health sector (i.e. a shift away from specialized care into primary care, greater efficiency in the use of daily hospital beds, reduction in length of hospital stays, professional salaries linked to functions, etc.), as discussed in previous sections, and the ICR team believes that as these reforms are disseminated and adopted in other regions of the country, there would be even greater efficiency gains.

3.4 Justification of Overall Outcome Rating

1 Including those from Dagestan, Tatarstan, Tyva, Khakassia, Chuvashiya; Krays: Krasnodar, Krasnoyarsk, Stavropol, Perm; Oblasts: Astrakhan, Bryansk, Vologda, Volgograd, Voronezh, Kemerovo, Kirovskaya, Kostroma, Orel, Samara, Tula; and Autonomous counties: Khanti-Mansi and Yamalo-Nenets.

2 Including the Deputy Governor of the Yamalo-Nenets autonomous County, two representatives of the MHSD, 20 heads/deputy heads of the regional health authorities (health ministries, departments, committees and chief departments), five heads/deputy heads of territorial MHIF; Six leading specialists of territorial medical information and analysis centers; Seven professors of higher education institutions; Five specialists of scientific research institutes; Seven workers of medical organizations. As well as World Bank, USAID Health Department, representative of the Canadian Embassy and delegation from Azerbaijan participated in the conference. The total number of conference participants reached 75 persons.

20

Rating: Satisfactory. The rating is based on both relevance of project objectives and a sound design that focused on systemic issues. The rating is also based on the full achievement of PDOs as discussed above in section 3.2. The project was implemented according to plan with no shortcomings. While an evaluation of efficiency using traditional measures was not deemed practical, and so this evaluation cannot verifiably rate the extent to which PDOs were efficiently achieved, data do show that the project did contribute to the successful shift of care from one that is predominantly an inpatient care delivery model to one that increasingly provides more accessible care in outpatient care or lower-intensity inpatient services. The project also successfully contributed to transferring financial resources from inpatient to outpatient services, all these systemic changes will result in efficiency gains in the sector.

3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development

Data to monitor the impact of the project on poverty, gender aspects was not collected; this is one of the deficiencies of the monitoring framework that makes it difficult to make verifiable conclusions on poverty and gender impacts. It is however very likely that the system improvements achieved with the contribution of the project, such as those on improving access to primary care providers, will have an impact in improving access to populations of all socio economic groups, including the poor and women. The project was, however, very effective in reaching the intended project beneficiaries such as the Government, MOH, regional administrations and health authorities in start-up regions, and the Duma, Managers of health facilities who acquired professional skills for financial and human resource management, best practice manuals in financial management, accounting, business planning, contracting staff. Other beneficiaries that were reached by the project include Health care personnel, the population in Chuvashia and Voronezh as well as the society at large, due to improved policy environment and better leadership from the MOH.

(b) Institutional Change/Strengthening (particularly with reference to impacts on longer-term capacity and institutional development)

One of the major achievements of this project is the degree to which the institutional capacity of the Federal Ministry of Health has been built, coupled with the development and introduction of a legal framework as well as an expansive list of norms and regulations for the sector (please see Section 3.2 on achievement of Objective 3). The Project was also responsible for supporting reforms in primary health care delivery and provider payment mechanisms, and these have also been institutionalized making it unlikely that these outcomes could be reversed. This project was instrumental in supporting the development and adoption of new guidelines; the training of staff; and dissemination and sharing of the tools that were developed as discussed in the sections above.

21

(c) Other Unintended Outcomes and Impacts (positive or negative)

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Not applicable.

4. Assessment of Risk to Development Outcome Rating: Negligible/ Low. As mentioned in section 3.5b above the project had a deep impact on institutional reforms that have been integrated into the regular functions of the health sector, both at the Federal Level and also in the two regions, and are now being taken up by other regions in the federation. As such, the risk that they would be reverted is considered low. The other factor that justifies this rating is that the project did not introduce activities that generate additional recurrent costs –on the contrary it led to greater efficiency in the health sector—so there are no budgetary concerns in terms of its sustainability.

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry

Rating: Satisfactory. The Bank’s performance in ensuring quality at entry is rated satisfactory because: • The project was very relevant to objectives of the CAS as described in section 1.1

above. • Of sound identification of the main challenges facing the sector. While it would have

been possible to focus on disease control as a way to deal with the poor health status of the population, it was the right decision to focus on systemic deficiencies which were limiting access to prevention and care services for the population.

• The project design was sound, and identified activities were clearly linked to objectives that the project intended to achieve.

• The project drew on the lessons from the previous project which had identified the need for guidelines at the federal level to guide the improvement of quality of care at the regional level.

• The identified project was well suited to support country-led initiatives. This project enjoyed the highest level of political support at both the federal and regional levels, a factor that greatly influenced the ultimate achievement of objectives.

• As described in section 1.1 above, the macroeconomic situation at the time of appraisal had made borrowing very much lowered the demand for borrowing, it was thus very critical that the Bank team was able to identify areas of support that would add real value to the achievement of a more efficient health care system.

• It ensured that the project was synergistic with projects supported by other key development agencies. The Project team built on European Union funded TACIS

22

project "Support to Public Health Management", whose focus was on strategic planning at the federal level. The project design was also a result of close collaboration with Canadian CIDA (in Chuvashia), as well as with DFID, WHO and the Soros Foundation.

• The design drew on the lessons of other health reform projects that had been supported in the Europe and Central Asia Region.

(b) Quality of Supervision

Rating: Satisfactory. The supervision of the project was satisfactory, with regular missions throughout implementation supported by a team of competent specialists and competent specialists in areas supported by the project. The supervision team included international experts in areas ranging from information technology to primary health care. The team maintained very close consultations with the RHCF in between missions and missions were of adequate length to address both operational and technical issues. The project has a set of well documented mission reports including detailed mission reports on progress and issues encountered as well as commendable efforts to ensure continued Bank engagement to support the sector in absence of limited federal level demand for borrowing. The Bank was proactive in supervising fiduciary issues; all FMRs were produced in a timely manner. (c) Justification of Rating for Overall Bank Performance

Rating: Satisfactory. Overall Bank performance is rated as Satisfactory on the basis of: • The Bank’s role in ensuring quality at entry as described in section 5.1 above; • Sound supervision of the project throughout implementation; • Commendable efforts in ensuring continuity of Bank support in the health sector.

5.2 Borrower Performance (a) Government Performance Rating: Satisfactory. Government performance during project preparation was Satisfactory. There was strong motivation on the part of the Federal MOH as well as the regional Ministries of Health to tackle systemic challenges that had been identified. Overall, the Government ensured adequate and timely funding of the counterpart funding. In addition, during implementation, the sector benefitted greatly from increased funding directed to the sector such as that implemented through the National Health Priority Project which contributed to increases in salaries for health care personnel as well as improvement in the technology base of the sector. The two project regions enjoyed very strong political support from Regional Government authorities which ensured continued attention to details throughout project implementation. (b) Implementing Agency or Agencies Performance

23

Rating: Highly satisfactory. The performance of the Russian Health Care Foundation is rated as Highly Satisfactory. The RHCF was staffed with a competent cadre of staff that had years of experience in supporting the reform process; the RHCF maintained very productive relationships with the Federal as well as the Regional MOH and was able to be very effective in supporting the reform process. The RHCF was also very successful in implementing a project progress monitoring system that documented and provided timely evidence for the results of the project. (c) Justification of Rating for Overall Borrower Performance

Rating: Satisfactory. The overall performance of Borrower performance is rated as Satisfactory on the basis of the strong performance of the RHCF as well as the performance of the Government, especially in guaranteeing high level political support that was also demonstrated through increased funding for the sector. Though it is not possible to demonstrate with hard evidence, it is very likely that the satisfactory performance of the project was facilitated by the additional resources that the Government directed to the sector. The Government created an environment for the sector that was conducive to better performance of health sector staff which included increased levels of salary and improvement in availability of high quality medical equipment.

6. Lessons Learned Data and evidence are critical in a challenging political economy. The successful implementation of health sector reforms and the supporting role that the HRIP played in the reform process is well documented; and this was possible through the strengthening of the use of evidence in policy design and management. The success has been well disseminated in the Russian Federation; and Chuvash Republic and Voronezh Oblast have become the demonstration model, the go-to-regions on the reform process. This is due to the fact that outputs from the project, such as the guidelines and norms that have been developed; efficiencies that have been gained have been measurable and the tools developed under the project have been made available to other regions. A significant barrier on how to undertake reforms has been removed and the overall experience has shown that complex reforms, such as those undertaken, can be successful.

Evaluations of institutional reform and capacity building projects in the health sector are complex, but need to be tackled if the attribution of Bank funded project is to be adequately assessed. This project spent a significant proportion of project financing on building the institutional capacity of the MOH as an effective steward of the sector and transforming the sector into a more efficient one. While there is evidence the reform process was successful and the project met objectives, as is common in health sector projects supported by the Bank, the project monitoring and evaluation design did not allow for a ‘with-project versus without-project’ evaluation which would have given more evidence on the impact of the project. Moreover, while the project did focus on poor regions as well as on rural communities with less access to health care, this

24

evaluation did not have hard data to demonstrate how the poor have benefitted in comparison to the non poor. Complex reforms require holistic and well sequenced approaches. The HRIP was able to achieve success because reforms implemented spanned the key areas of the health care system starting with the stewardship functions of the Federal and Regional MOH, to the financing mechanisms and provider payment systems, to the way that care is delivered including the changing the balance between the different levels of the care system. The lesson learnt was that partial reforms can produce incentives at one level and might trigger reactions in other levels of the care system. For example, reducing the number of beds in the care system needs to be balanced by changes in the primary care level; in the case of the HRIP attempts to reduce tertiary care beds necessitated the increase in care provided in the primary level. The role of champions, both at regional and Federal level, was critical in ensuring the success of the reforms. The reform process in the health sector received the highest level of support from officials at the federal MOH who believed in the need to reform the sector and who were critical in the development of the 1997 strategy. At the regional level, both Chuvashia and Voronezh enjoyed support from the highest levels of political administration who took keen interest in the sector as a whole as well as in specific developments within the sector. The Bank’s resources, both human and financial, can provide important leverage to support government strategies thus increasing likelihood of success and sustainability of reforms. The added value of the Bank loan was not only financial; the Bank’s involvement in the sector was instrumental in leveraging additional resources, both human and financial. The Bank was instrumental in obtain additional resources from other partners, such as CIDA to fund several Flagship courses in support of the reform process; and throughout project life organization of video conferences on a number of key topics on health reform.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies

(b) Co-financiers

(c) Other partners and stakeholders (e.g. NGOs/private sector/civil society)

1

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent)

Components Appraisal Estimate (USD millions)

Actual/Latest Estimate (USD

millions)

Percentage of Appraisal

COMPONENT 1. RATIONALIZATION OF HEALTH SERVICES

33.92 23.80 70.17

COMPONENT 2. STRENGTHENING OF FINANCIAL AND ECONOMIC MANAGEMENT

2.88 2.50 86.80

COMPONENT 3. IMPROVING POLICY AND GOVERNANCE

0.64 1.60 250.00

COMPONENT 4. PROJECT MANAGEMENT

3.77 9.10 241.38

Total Baseline Cost 41.21 37.00 89.78

Physical Contingencies 0.00 0.00 0.00

Price Contingencies 0.00 0.00 0.00

Total Project Costs 0.00 37.00 Front-end fee PPF 0.00 0.00 .00 Front-end fee IBRD 0.00 0.00 .00

Total Financing Required 0.00 37.00

(b) Financing

Source of Funds Type of Co-financing

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of Appraisal

Borrower 11.2 9.7 86.5 International Bank for Reconstruction and Development

30.0 27.3 91.0

1

Annex 2. Outputsby Component

Expected outcomesbycomponents

Outputs Achievements Remarks

1) Rationalization ofmedical care provisionsystemA. Federal regulatory andmethodical documents forsystem restructuring

A.1 Development and testing in twopilot regions of strategic approaches tocomprehensive restructuring, theirapproval by theMHSD

Strategic approaches to comprehensive restructuring weredeveloped at the federal level. Theseapproaches wereused asa basis fordevelopment of regional strategic restructuring plans for thehealth sector. Methodological recommendationswereprepared andapproved:“Substantiation of the public health optimization and preparation of long-term development scheme for the health facilities of theregion”Methodical recommendations were prepared and approved:

Achieved

A.2 Approval by the MHSD ofguidelines and manuals oncomprehensive restructuringimplementation

• Improvement of primary care organization;• Ambulance service optimization;• Improvement of effectiveness of diagnostic services• Issues of inpatient care restructuring.

Achieved

A.3 Approval by the MHSD of neworganizational norms and standards forhealth facilities

MHSD Orders on organization of public health care service provision:- Procedure of organization of public health care service provision based on the district principle (of August 4, 2006, No.584),- Organization of district therapists work (of December 7, 2005, No.65),- Organization of medical care provision (of October 13, 2005, 633), - Approval of the procedure for organization of primary care service provision (of July 29, 2005, No.487),- GP performance evaluation criteria (of May 11, 2007, No.325),- GP nurse performance evaluation criteria (of May 11 2007, No.326).

Achieved

A.4 Testing, evaluation and approval bythe MHSD of at least 40 new clinicalmanuals based on adoptedmethodology, and adoption of 12manuals in the regions

Achieved

A.5 Stable financing of organizationsresponsible for development ofmanuals, implementation of evidence-based medicine principles,pharmacoeconomic studies andtechnology evaluation; implementationof activities by them in accordance withagreed action plans

The MHSD established successfully functioning organizations responsible for preparation of disease management protocols and clinicalguidelines, implementation of evidence-based medicine principles, etc., including:- Health standardization unit, Public Health and Management Institute, Sechenov Medical Academy;

- Standardization, licensing and accreditation laboratory, Public Health and Management Institute, Sechenov Medical Academy;

- Central health organization and informatization research institute, etc.

Besides, interregional public organization “Pharmacoeconomic Studies Society”, interregional public organization for assistance instandardization and medical care quality improvement were established.

Achieved

A.6 Approval by the MHSD of newlicensing criteria and procedures

RF Government Resolution of July 4, 2002, No. 499 “On approval of provisions for licensing in the health sector” (version of10.03.2006.)Order of Roszdravnadzor of March 14, 2005, No. 500-PR/05 “On approval of provisions for establishment by the subjects of the Russian

Achieved

2



Federation of Roszdravnadzor committees for licensing in health care, pharmaceutical activitiesand activities related to drugsandpsychotropic substances circulation”Order of Roszdravnadzor of May 4, 2005, No. 1115-PR/05 “On issue of licenses for manufacturing of medical equipment”Order of Roszdravnadzor of April, 2006, No. 979-PR/06 “On creation of the Central Licensing Committee of the Federal Health Careand Social Development Supervision Service”

A.7 Implementation of the newlicensing procedure in the pilot regions

In connection with establishment of the Federal Service for Supervision in Public Health Care and Social Development, new licensingbodies were reorganized in the regions and new licensing procedures implemented. Territorial licensing authorities organize and conductlicensing of medical and pharmaceutical activities.

Achieved

B. Restructuring regionalhealth care systems

B.1 Availability of the regional healthcare restructuring plan meeting strategicapproaches, agreed upon with theregion, MHSD and Bank, andimplemented in accordance with theagreed schedule

Voronezh oblastRegional plan for comprehensive restructuring of the Voronezh health care system was developed for 2003-2010 in line with strategicapproaches; it was agreed upon with and approved by the oblast administration and RF MHSD. The plan is being implemented inaccordance with the plan/schedule. The strategic plan was finalized (2nd stage) taking into account the new legal framework andcomplemented with municipality restructuring plans.Chuvash RepublicThe strategic plan for Chuvash health care system restructuring in 2003-2010 was developed and approved by the Cabinet of theChuvash Republic. The plan is being implemented in accordance with the plan/schedule. The strategic plan was finalized (2nd stage)taking into account the new regulatory legal framework.

Achieved.

B.2 Implementation of the plan in partpertaining to investment in equipment,buildings and human resources training

Voronezh Health Care System Restructur ing 2004-2008

RUB $U.S. (Approximate)*

Rehabilitation of theGP offices 134.11 million 5.417 million

Equipment of GP offices 79.2 million 2.97 million

Procurement of 249 ambulancevehicles,26 cars, and equipment for theambulanceservice

143.72 million 19.703 million(RUB 143.72 million +USD 331 774)

-Procurement of equipment for interrayon centersincluding: 20 fetal monitors, 13 laparoscopic stands, X-ray

420.3 million 22.15 million(RUB 420.3 million + USD 599 476)

Health facility building rehabilitation 149.9 million 5.6 million

Training: 106 GP (family doctors), 783 doctor, and 105GP nurses were retrained

5.8 million 21 804

Chuvash Republic

3



Procurement of equipment for interrayon centers includingtraining equipments

424.488million

15.95 million

Procurement of equipment for specialized care 40.5 million 1.5 millionTraining: 166 GPS were trained

B.3 Implementation of theGPperformance-based contracting system

Voronezh oblast: GPperformance-based payment regulations weredeveloped and arebeing implemented

In the Chuvash Republic, GP contracts, developed by republican specialists, were introduced.

Achieved

2) Strengthening offinancial and economicresourcesmanagement

2.1. Application of financing model atthe regional level for evaluation of:- thebenefit package cost;- financial effect of changes in clinicalguidelines;- financial effect of changes in medicalcare organization guidelines;- effect of health care financing reform

MHSD and thepilot regionsuse the model of statebenefit program cost evaluation and health finance reform impact evaluation.The following productswereprepared and provided to thepilot regions and theMHSD:- network rationalization and social and economic impact evaluation model;- software.

Achieved

2.2. Inclusion of regional healthaccounts in the regional financingmodel

Simulation modelsweredeveloped for analysisand projection of aggregated health expendituresand sources of their finance for themedium and long-term (5-10 years) for a separatesubject of the Russian Federation; and their software. Thesoftware wasprovided tothepilot regions. In thepilot regions, interagency working groups were established for development and implementation of the regionalhealth accounts.

Achieved

2.3. Setting up of two resourcecentersthat may participate in making decisionson key issues of health economics andorganization

Two centers were established under contract No/ 4685/B.3/01 of August 2, 2007, “Provision of services for organization of a resourcecenter of administration and financial management in the health sector, for development of qualification upgrading programs,preparation of learning, methodical and practical aids on financial management” and contract No. 4685/B/2/01 of August 2, 2007“Provision of services on organization of resource center of health economics”.FRC internet sites were designed:- administration and financial management - http://www.frc.ane.ru/;- health economics - http://www.economzdrav.mednet.ru

Achieved

2.4. Proportion of GPs working under aperformance-based contract Chuvash Republic Voronezh oblast:

100% of GPs work under the performance-based contracts.

Achieved

2.5. Proportion of health facilities withfinancial managers trained under theproject

32 specialists of the pilot regions were retrained under the regional flagship course on the health reform and sustainable financing(September 11-19, 2006, Cheboksary).37 specialists of the pilot regions were retrained under the regional flagship course on the health reform and sustainable financing (May28 – June 5, 2007, Voronezh).About 150 specialists of the health sector participated in testing of training modules prepared by the federal resource centers.

2.6. Preparation of financialmanagement aids and materials adaptedto Russian conditions and theircirculation in the regions

Administration and financial management FRC prepared 24 training and methodical aids covering topical issues of financialmanagement in the health sector, based on up-to-date academic and practical data.The key training and methodical aids were handed out to the participants of the National Project closing Conference (December 10,2008)

Achieved

4



Multi-level modulequalification upgrading programs covering the issues of health administration and financial management weredeveloped and tested:- 30 programs for trainers of respectivedisciplines;- 54 programs for practical workers of financial and economic servicesof thehealth sector institutions,including: 3 long-term program of professional retraining, 11 medium-term and 40 short-term programs of professional upgrading.

• Currently thesiteprovidesarticles on topical issues of health administration and financial management.• 14 reviews of literature on general theoretic and practical issues of health administration and financial management were

prepared

3) Health policy andlegislation

3.1. Submission for review to respectivebodies of theconcept of improvementof thepackage of legal and regulatoryframework of the federal and regionallevels for their approval/consolidationin acode. Completion ofimplementation of changes introducedin theregulatory acts

Theproject preparation stage included analysisof the current federal and regional legal framework. This review wasprovided to theState Duma Health Protection Committee. Based on this review, theMHSD jointly with the State Duma Health Protection Committeeprepared ahealth lawmaking plan.

Achieved

3.2. Approval and implementation ofother legal and regulatory acts theneedfor which will bedefined during theproject.

Based on theoutputsof theproject, legal regulatory acts in thehealth sector areadopted (RF Government resolutions, MHSD ordersandmethodical recommendations)

Achieved

3.3. Training of specialists in legalissues and application of the acquiredknowledge and skills for developmentof new approaches in lawmaking in thehealth sector

3.4. Improvement of interaction andcoordination of relationsbetween theMHSD, regional health authorities,federal and territorial MHI fundsandhealth facilities

4) Project management 4.1. Key stakeholdersparticipate in theproject and receive information about it

MHSD and the Russian Healthcare Foundation ensure interaction between the interested parties required for theproject implementationand provide relevant information on theproject progress.

4.2. Performanceof theprojectmanagement functions (planning,implementation, monitoring andevaluation, reporting, disbursement andaccounting) in accordance with theIBRD requirements

MHSD and the Russian Healthcare Foundation fully perform thenecessary project management functions.

1

Annex 3. Economic and Financial Analysis N/A

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/Specialty

Lending Olusoji Adeyi Lead Health Specialist ECSHD TTL De St Antoine, Jean Jacques Lead Operations Officer ECSHD Former TTL Teresa Ho Lead HD Specialist ECSHD Former TTL Karl Skansing Sr Procurement Specialist ECSCS Procurement Veronica Kabalina Social Scientist N/A Consultant Armin Fidler Health Sector Manager ECSHD Sector Manager Annette Dixon Human Development Sector Director ECSHD Sector Director James Christopher Lovelace Former HD Sector Director ECSHD Sector Director Maria Gracheva Operations Officer ECSHD Operations

Anna Jouravleva Financial Management Specialist ECSPS Financial Management

David Freese Senior Finance Officer N/A Financial Management

Robert Hecht Peer Reviewer N/A Reviewer Simon Blair Peer Reviewer N/A Reviewer

Supervision/ICR Patricio Marquez Lead Health Specialist, TTL ECSHD TTL Rifat A. Atun Consultant ECSHD Public Health Anne Margreth Bakilana Economist ECSH1 Economist, ICR Alexander Balakov Procurement Specialist ECSC2 Procurement Jan Bultman Lead Health Specialist ECSHD Public Health Shiyan Chao Sr. Economist (Health) LCSHH Economist Willy L. De Geyndt Consultant AFTS3 Public Health Alberto Gonima Consultant ECSHD HIS Maria E. Gracheva Senior Operations Officer SASHN Operations Olesya Klimenko Team Assistant ECCU1 Team Assistant Alyona Korneva Consultant N/A Consultant Galina S. Kuznetsova Sr. Financial Management Special ECSC3 Financial Mngt John C. Langenbrunner Lead Economist, Health EASHH Economist Tatyana Loginova Operations Officer ECSHD Operations Aziz Mamatov E T Consultant N/A Consultant Jennifer Manghinang Senior Program Assistant ECSHD Program AssistantIrina Reshetnikova Program Assistant ECCU1 Program AssistantSevil K. Salakhutdinova Consultant ECSHD Public Health Nikolai Soubbotin Sr. Counsel LEGEM Legal Maria D. Zhorova E T Temporary ECCU1 N/A

2

(b) Staff Time and Cost Staff Time and Cost (Bank Budget Only)

Stage of Project Cycle No. of staff weeks USD Thousands (including

travel and consultant costs)Lending

FY98 - 46.45 FY99 - 210.83 FY00 19 49.71 FY01 33 73.69 FY02 46 232.86 FY03 31 189.66 FY04 - 0.03 FY05 - 0.00 FY06 - 0.00 FY07 - 0.00 FY08 - 0.00

Total: 129 803.23 Supervision/ICR

FY03 6 18.93 FY04 24 106.47 FY05 29 116.81 FY06 23 72.22 FY07 17 117.14 FY08 30 104.35 FY09 24 0.00

Total: 153 535.94

Annex 5. Beneficiary Survey Results N/A

Annex 6. Stakeholder Workshop Report and Results N/A

3

Annex 7. Summary of Borrower’s ICR

Project description The project is aimed at providing technical, regulatory, legal and institutional frameworks to implement the national health reform program in the Russian Federation. The efficiency of the health sector is improved through: i) Rationalising and restructuring the system of health care provision at all levels and improving the coordination and cooperation among these levels; ii) Improving the management of financial, physical and human resources, developing and implementing a new system of employment contracts for health workers; and iii) Improving the policy and governance in the health sector at all levels.

Financial implementation arrangements A major part of the cost of the goods and services procured under the project was funded out of the IBRD loan. The Russian Federation provided counterpart funding from the federal budget and the budgets of the pilot regions in an amount of about US$ 10 million, which was spent primarily to pay Russian taxes and duties as well as to fund civil (construction and renovation) works in health facilities in the pilot regions. The federal budget financed the servicing and repayment of the Loan extended to fund activities in support of the national health reform policy development, and activities related to the project management and implementation at the federal level. Costs of project management in the pilot regions are covered from the budgets of these regions.Some implementation-related activities were incrementally financed from the MOHSD’s research budget and other sources under the federal budget. The pilot regions caused local current funding to be provided to: prepare premises for equipment installation, supply consumables and reagents for their health facilities, implement training activities for medical personnel and support the regional project management units.

Regional project participants In accordance with the Loan Agreement (Section 1.01 (g)), the project participating regions are the Voronezh Oblast and the Chuvash Republic. In 1998, during the project preparation, two regions were selected: the Novgorod Oblast and the Chuvash Republic. The selection was based on the following criteria: 1) The regional administration signed a three-party agreement with the MOH and the Federal Fund of Mandatory Health Insurance (FFMHI) on cooperation in health care management; 2) The region had its regional health reform program; 3) The region had started to implement this program; 4) The political situation in the region was stable (with the Governor and members of the legislative authorities already elected); 5) The regions should differ in terms of their patterns of population distribution (be with high and medium / low population densities) to offer different models of health care management. During the preparation for the negotiations, it was decided against inclusion of the Novgorod Oblast in the project as a pilot region due to its outstanding debts to the federal budget.

4

Project evaluation

A. Rationalisation of health services

A.1.1 Comprehensive restructuring of health services Activities implemented under this sub-component included:

1) Development of strategic approaches to comprehensive restructuring of health service delivery

2) Development of the Strategic Plan of Comprehensive Restructuring of Health Services in the Voronezh Oblast (Stage 2)

3) Preparation of the Regional Health Restructuring Plan in the Chuvash Republic (Stage 2).

4) Study tours/training events for managers and specialists of economic units and practising physicians

5) Development of a monitoring system for restructuring of health services at the regional level

6) Designing of a GIS-based management information system in support of decision-making on regional health care

7) Procurement of a software package (called PK START) and its use for purposes of collection and processing of medical statistics, and personnel training in its use

8) Transfer of the health management information system for collection and analysis of health statistics to the START system

9) Information support in highlighting the comprehensive restructuring of the regional health system (based on the example of the Chuvash Republic)

A.1.2 Standardisation, licensing and accreditation in the health sector

Under the sub-component, the following activities were implemented:

1) Elaboration of a regulatory and legal framework for developing and implementing case management protocols

2) Development of information and software products to maintain databases of health care standards and treatment protocols

3) Computer equipment (servers) were procured and supplied to the Ministry of Health and Social Development of the Russian Federation and the Licensing Resource Centre of the Federal Service for Oversight in Health Care and Social Development.

4) Collected Standards of High-Tech Health Care (in 2 volumes) were produced, printed and disseminated.

5) Collected Standards of Specialist Health Care (in 2 volumes) were produced, printed and disseminated.

6) Consultation and methodological assistance was provided to Regional Project Management Units to implement activities under this sub-component.

5

A.2.1 Chuvash Republic: Support in the implementation of the regional comprehensive health restructuring plan

Since 2000, the Chuvash health system has been undergoing significant changes to ensure access to and high quality of health care for each resident. This comprehensive reform is meant to restructure and streamline specific components of health care and improve the system of health finance. The Strategic Plan of Health Restructuring for 2003-2010 was developed in 2002 by specialists of the Chuvash Ministry of Health with technical assistance provided under the Chuvash-Canadian Health Reform Project funded by the Canadian International Development Agency (CIDA). The Restructuring Plan is based on a system-wide approach; it was approved by the Chuvash Republic Cabinet of Ministers and endorsed by specialists from the federal MOHSD and experts of the World Bank. On September 21, 2005, the Chuvash Cabinet of Ministers, the federal MOF and the federal MOHSD signed a Subsidiary Loan Agreement, under which US$ 10 million was allocated to the Chuvash Republic to implement the Health Reform Implementation Project. In 2006, a team of project consultants developed the 2nd stage of the Strategic Plan, taking into account the changes in the regulatory and legal framework and the newly launched National Priority Health Project.

Development of a new health promotion policy Key health strategies are defined in the Health Care Concept of the Chuvash Republic for 2004 - 2010 as adopted by Decree of the President of the Chuvash Republic # 65 of 18.06.2004. Since 2004, the Chuvash Republic has been participating in the WHO European Network of Healthy Regions; the cities of Cheboksary and Novocheboksarsk are accredited participants of Phase IV of the WHO’s Healthy Cities Movement. All the municipalities are united in a regional network of Healthy Cities, Rayons and Settlements. The Ministry of Health and Social Development of the Chuvash Republic (with technical assistance provided under the WHO’s Russian Health Policy and Management Project funded by the Canadian International Development Agency), developed a methodology for active involvement of municipalities in the process of setting up an integrated inter-sectoral system of health care and health promotion in a give rayon or city with due regard to local specifics. To ensure sustainable functioning of interagency teams, a School of Professional Training in the Area of Health Care and Health Promotion (further on referred to as ‘the School’) was established under the Institute of Continuous Education for Physicians of the Chuvash MOHSD. The School is called to: implement training programs in public health, train consultants/trainers from the School to act as interagency team supervisors and as trainers for interagency teams (training of trainers).

Primary health care restructuring

Chuvashia has been consistently developing general (family) practice. The first 18 GP offices were opened in 2001-2002, with financial support for this purpose provided by the Canadian International Development Agency. In Chuvashia, primary health care is developed in accordance with Decree of the President of the Chuvash Republic # 14 of February 25, 2003 On Additional Measures for Family Medicine Development in the Chuvash Republic, under the Family Medicine Subprogram of the Chuvash Rural Health Care Program for 2002-2006, approved by Resolution of the Chuvash Cabinet of Ministers # 261 of November 30, 2001, and the Chuvash Targeted Program to Improve Primary Health Care to achieve key targets of the National Priority Project in the Chuvash Republic for 2006–2010, approved by Resolution of the Chuvash Cabinet of Ministers # 301 of November 29, 2005. In 2008, primary health care was provided in Chuvashia by 55 city hospitals, 21 central rayon hospitals, 2 catchment area hospitals, 430 general (family) practice departments, and 573 feldsher and midwife stations.

6

As of 01.10.2008, general practitioners delivered care to about 54% population in Chuvashia (versus 13.1% in 2003). In rural areas, general (family) practitioners provide care to over 60% of the population. About RUR 17 million from the loan proceeds was spent to procure 160 sets of equipments for general (family) practices. Municipal budgets funded civil works to retrofit premises and construct new premises for GP offices. The regional consolidated budget provided funding to procure equipment for all newly opened GP offices and health care vehicles for some of them. In accordance with the integrated regional strategy, a standardised approach was implemented to train GPs, equip their workplaces, and define volumes and quality of their services which ensures actual provision of health benefits while delivering high-quality health care closer to the places of patient residence. Over the recent 5 years, the total capacity of outpatient/policlinic care increased by 2.1% to each 36,434 visits per shift in 2007.

Remuneration of general/family practitioners

In 2004, to set up mechanisms for primary health care improvement, a performance-based system of incentive payments for GPs was introduced in Chuvashia; it is based on employment contracts between the administration of the health facility and the GP, specifying a basic salary rate and additional contracts specifying a variable part of remuneration directly dependent on performance. The additional variable part of remuneration for general/family practitioners is defined on a monthly basis as a function of performance evaluated on the basis of about 30 performance indicators, reflecting medical, social and economic constituents of health care. The care delivery process is promptly evaluated using such indicators as volumes of outpatient/policlinic care provided, shares of preventive visits and active home visits, number of arranged ‘schools’ for patients, etc.; immediate and medium-term results (coverage with occupational health examinations and screenings, incidence of diseases with temporary disability, hospital admission rates, timeliness of cancer detection, etc.), as well as long-term outcomes (impact on the length of life, maternal and infant mortality, mortality in working age).

Restructuring of inpatient care

In Chuvashia, restructuring of inpatient care includes such steps as phased restructuring of the bed stock; reorganisation of inefficient hospitals; establishment of inter-rayon centres of secondary (specialised) care; rationalisation of tertiary care in (clinical) hospitals; intensification of treatment processes through introducing up-to-date managerial and medical technologies. As of the beginning of 2008, Chuvashia had 59 hospitals, including 43 municipal (20 rural and 23 urban) and 16 public sector hospitals. In 2002-2007, the reorganisation affected 22 catchment area hospitals with the total bed capacity amounting to 343 beds; they were transformed into general/family practice departments for 1,199 visits and 98 day care beds. In Cheboksary, the number of hospitals was reduced from 18 in 2002 to 14 in 2006, and 7 buildings were divested from the city health sector. For example, to improve the efficiency of resource capacity use in tertiary health facilities, ensure efficient use of financial resources made available for their maintenance and implement the coordinated public policy of social disease prevention, 9 out of 12 specialized health facilities (3 TB Dispensaries, 4 Skin and Venereal Disease Dispensaries, and 2 Narcological Dispensaries) were merged into 3 public sector facilities of the narcological, skin and venereal and TB services. The number of beds in day care facilities increased by 19.4% to reach 2,440 day care beds as of the beginning of 2008 (versus 2,043 beds in 2002); they include 1,178 day care beds (48.0%) opened in hospitals, 1,269 beds in 67 outpatient/policlinic facilities and their units. Home care facilities have been also put in place. In 2007, they were opened in 57 facilities, and provided care to 11,455 patients (versus 4,049 patients in 2002). Over the recent 5 years, the total number of patients treated in alternative care delivery settings

7

increased by 9.8%. By 2007, the volumes of care provided in alternative care delivery settings reached 551.3 days per 1,000 population. Since 2002, the share of outpatient surgery in the total number of surgical services increased by 3.7% (from 45.7% in 2002 to 47.4% in 2007 .). The World Bank project procured equipment worth of RUR 13 million for 5 outpatient surgery centres, which enabled them to introduce up-to-date medical technologies in the outpatient/policlinic units of the health facilities. The new model helps to reduce the pressure on inpatient facilities and emergency services, and shifts part of care volumes and financial flows to the outpatient sector; therefore, since 2002, it resulted into savings of over RUR 500 million. These savings were reinvested into improvement of primary care, and strengthen of diagnostic services in policlinics.

Continuous health care quality management

Since 2004, Chuvashia has been going through a new stage in the development of performance evaluation methods for health care and social support, as required in the federal Government’s regulations on introducing performance-based budgeting in the Russian regions. Every year, the Chuvash Republic prepares a report on performance and key activities of the Ministry as an entity of budget planning; it is set forth therein that the main goals for the health sector are to improve the demographic situation and the quality of and access to health care. Law of the Chuvash Republic # 44 of July 19, 2007, On Regional Quality Standards for Public Services, defines key requirements to the contents of, development procedures for and application of regional quality standards for public services provided by budget-funded institutions and financed from the regional budget. In accordance with the federal legislation, these standards are subject to approval by the regions.To ensure timeliness and high quality of public services, to develop an information base in support of managerial decision-making on the volumes and scope of public services provided by health facilities, to predict and optimize public expenditures, Resolutions of the Cabinet of Minister of the Chuvash Republic # 42 of 16.03.2007 and # 118 of 28.05.2007 were issued to approve the list of health services subject to needs assessment for such services and the procedures for evaluating the quality compliance of actually provided health services with the performance standards and key requirements. Under a key activity of the pilot project, provisional regional lists of equipments were developed for different departments of public sector and municipal inpatient facilities on the basis of the health care standards approved by Order of the Chuvash MOHSD # 392 of 29.05.2007 (Provisional Regional Standards of Equipping Inpatient Departments of Health Facilities).

Regional information and communication system

With intensively pursued structural reforms, and introduction of up-to-date organisational and medical technologies, coordination and manageability of the sector is largely dependent on availability of modern management information systems. The Chuvash health system shares information with other regional ministries and agencies, involved in the implementation of the Public Health Development Concept, under the single regional information and analysis system connecting all the public authorities and local self-governance bodies.

Human Resources Development

Following the Project Implementation Plan, 40 specialists from Chuvash health facilities (including GPs) participated in study tours to Sweden, Denmark, and the Netherlands to get acquainted with their systems of general practice and heath systems as a whole.In the period of

8

2003-2007, a total of RUR 16,524,920 was spent to procure up-to-date equipment to introduce effective training technologies; these costs were funded from the budget, extra-budgetary sources and out of the IBRD loan). An indicator of health training quality is the share of certified specialists. The training in the above training centres helped to increase the share of certified physicians by 28.5% compared with 2002, and the share of certified nurses rose by 55.4% during the same period to reach 87% and 86.6%, respectively. In 2006, the Cheboksary Medical College enrolled a group of 6 students to be trained as sisters of mercy in nursing care as a specialty under the patronage of the Cheboksary-Chuvash Eparchy. In addition to training in accordance with the national education standard, sisters of mercy will study religious disciplines; upon completion of the training in the College, they will receive nationally-recognized diplomas of specialists in nursing care and certificates of sisters of mercy from the Cheboksary-Chuvash Eparchy.

The project included videoconferences of the World Bank and the Global Development Learning Network under the following headings: the UK’s experience in implementing primary health care reform (14.03.2005); Licensing and accreditation of health workers: Lessons learnt in the USA and other countries (08.04.2005), Primary health care: how to ensure its priority? Lessons learnt in Bosnia and Herzegovina (27.06.2005), Reform in the health system of the Veterans Health Administration of the US Department of Veterans Affairs: transformation of the inefficient public sector health care system to a holistic healthcare system under which high quality, evaluation, effectiveness, commitment and information are in the forefront of health reform (13.04.2007), Dying too Young: Addressing Premature Mortality and Ill Health due to Non-Communicable Diseases and Injuries in the Russian Federation (03.02.2006), National Response to High Incidence of Injuries and Wounds from Traffic Accidents (21.05.207).

Financial sustainability of the health system

The Chuvash Republic is annually increasing the funding for the Benefit Package: as a result, the allocated amounts increased form RUR 1,470.8 per capita in 2002 to RUR 3,403 per capita in 2007. In 2007, the funding from the budget accounted for 54.7% in the actually allocated amount to cover the cost of the Benefit Package, with the remaining 45.3% covered from mandatory health insurance proceeds. Under the project, Chuvashia accomplished a phased transition to a primarily single-channel system of health finance (pooling of funds from different sources). Owing to increased contributions for non-working population and reallocation of funds from municipal budgets, the deficit of the Regional Benefit Package was reduced from 37.7% (as of the beginning of 2007) to 17.9% in 2008. In 2007, the total amount of funding for the Benefit Package was RUR 4,520.3 million (or 114.6% of its approved cost). During the period from July 1, 2007 to July 1, 2008, an additional amount of RUR 930.1 million was channelled to the MHI system to pool financial resources (Figure 1).

In 2007, the mandatory health insurance contributions for non-working population increased 1.5 times and part of proceeds from the budget was reallocated to the MHI system; therefore, the share of health finance from the budgets and that from MHI changed: the share of MHI increased from 39.5% of the total amount of funding (July 2007) up to 67.3% (December 2007).

Figure 1: Health expenditures in the Chuvash Republic, by public administrator of health finance in accordance with the regional health accounts, %

9

Emergency care restructuring

In 2007, emergency care was provided in Chuvashia by 4 city ambulance stations, 20 emergency departments under rayon and city hospitals and 19 affiliated units of emergency care departments under catchment area hospitals with 394 ambulance-based and 98.5 24-hour teams. The availability of emergency care teams is 0.76 per 10,000 population. In rural areas, it is higher than in urban areas (0.86 versus 0.65, respectively). The number of timely served ambulance calls depends on the radius of operation zones, availability of teams, and ‘density’ of ambulance calls and varies from 61.4% in the Alikovo Rayon to 95.7% in Kanash. The average regional number of timely served calls increased from 77.2% in 2004 to 85.8% in 2007. In 2007, the optimized operation of the ambulance service reduced the average time of waiting for an ambulance team to 13.8 minutes. The average regional load per 24-hour team is 9.8 calls per day (during 9 months of 2008, it was 9.4 calls per day); in urban and rural areas, it is 13.8 and 7.0 calls per day, respectively (9 months of 2008:12.99 and 7.3). This project procured 92 ambulances for Chuvashia, and under the National Priority Health Project, 161 ambulances (including 4 resuscitation ambulances) were supplied, which enabled the region to renew its ambulance fleet. In 2008, the project helped to fully up-date the information system enabling to automate the operation of the emergency/ambulance service. In accordance with the Strategic Plan, training has been provided to drivers and staff of life support services so that they could acquire para-medical skills to provide first aid. During the period of 2005-2008, over 1,000 people were trained.

Reorganisation of diagnostic services.

The following new services were opened in Chuvashia in support of diagnostic processes: Functional diagnosis, X-ray diagnosis, Laboratory diagnosis, Ultrasound diagnosis, and Endoscopy. The volumes of diagnostic services have been increasing from year to year, which is accounted for by the provision of additional equipment to health facilities and intensified preventive activities of physicians to detect diseases at early stages. So, in 2002-2007, the number of laboratory tests per day increased by 10.7% to reach 103,400 in 2007; the number X-ray examinations increased by 7.3% to reach 4,643; the number of examinations per machine per year increased by 39.6% to reach 4,855; and the number of ultrasound examinations increased by 39.8% to reach 2,850. In the recent 5 years, the number of functional diagnostic examinations increased by 38 % to exceed 4 million. Outpatient diagnostic services account for 65 %, and

10

inpatient diagnostic services account for 32 % of the care provided. The most frequently provided services are electrocardiography, rheography, examination of external respiration. The laboratory diagnostic service consists of 136 clinical diagnosis, 15 bacteriological, and 5 serological laboratories. Centralized biochemical and immunological laboratories were set up under the Regional Diagnostic Centre.

Implementation difficulties and next steps: Difficulties of making inter-budget settlements of accounts to establish a centralized laboratory service due to insufficient funding. Underestimated share of diagnostic testing in the structure of the health care tariff, inadequate coverage of costs. In 2009, it is intended to organize a centralized laboratory under the Regional Clinical Hospital. A centralized operation and maintenance center for diagnostic equipment will be established and equipped. In municipal health facilities, the problem of high depreciation of diagnostic and laboratory equipment (over 60%, particularly, of X-ray equipment) was to some extent addressed through provision of equipment under the National Priority Health Project and its procurement funded from the regional budget. However, similar modernization is required in the regionally-owned health facilities which act as interayon centers for many specialties. To improve efficiency of budget expenditures, procurement of diagnostic equipment and consumables was centralized at the level of the Ministry of Health and Social Development the Chuvash Republic, which resulted into saving of 15% of resources.

A.2.2 Voronezh Oblast: Support of Regional Comprehensive Health Care System Restructuring Plan

Health promotion is a priority of the social policy of the Voronezh oblast administration. Under the HRIP, the regional health care system was reformed in accordance with the main sector development objectives reflected in the Strategic plan of comprehensive restructuring of the health care system in the Voronezh oblast (2003-2013, stages I-II), agreed with the RF MHSD, the oblast Duma and approved by the Chief Health Department.

Primary Care Reform

The Strategic plan of comprehensive restructuring of the medical care provision system in the Voronezh oblast (Strategic Plan) included a package of activities aimed at increasing the priority of the primary care (including priority allocation of funds for its development, strengthening of physical assets and human resources provision, improvement of workers’ payment system in the district service, organization of new forms of primary care, their interaction with other levels of medical services, development of general practices so that they gradually become the focus of the primary care and acquire a greater role in the overall health care system, strengthening of preventive care). Under the Strategic Plan the Voronezh oblast carried out a number of activities to transform primary care into general practitioner (family doctor) primary care, to improve quality of and access to medical care, to implement the oblast targeted program “Formation of healthy life style among Voronezh oblast population for 2002-2006” approved by oblast Duma resolution No. 476-III- D of October 31, 2002.

Organization of doctor outpatient facilities

Under the Strategic Plan implementation, as of the end of 2008, 346 general practitioner offices were set up (2002, 7), including 237 opened in 2006 and 2007 and 100, in the first six months of 2008. These 346 GP offices include 268 rural offices and 78 offices in the city of Voronezh. Organizationally and legally, the GP offices of the Voronezh oblast are structural units of health

11

facilities. Three-to-five beds day-care hospitals are organized in premises of GP offices to provide treatment to dispenser group of patients, finalization of treatment and post-hospital rehabilitation of patients. Since 2005, the project provided support to procurement of 337 sets of equipment for the GP workplaces worth RUR 70.1 million. In addition, the oblast budget allocated funds for procurement of medical equipment for doctor outpatient facilities and GP offices: in 2004, RUR 13.1 million; in 2005, RUR 14.4 million; in 2006, RUR 15.6 million; in 2007, RUR 20.3 million.

In accordance with the Project Implementation Plan, early in 2006, the Voronezh State Medical Academy named after N.N. Burdenko (VSMA) started retraining of doctors in general practice (family medicine). A contract was signed for training general practitioners (family doctors) of the Voronezh oblast in the VSMA; retraining plan was approved; six training districts were defined; as of the date, 357 GPs were retrained (through the project funds). In parallel the oblast budget also financed retraining of GPs. Totally 679 GPs were retrained; 35 GPs continued training in 2008. The number of doctors trained as general practitioners (family doctors) in a two-year training program: 2006, five doctors; 2007, two doctors; 2008, six doctors; three doctors continue training.

Under the project, the Family Medicine Department of St.Petersburg Medical Academy of Post-Graduate Education provided training to 15 outpatient facility doctors from the city of Voronezh in general practice (family medicine). The objective of this cycle was to train tutors in general practice (family medicine) for health facilities of the Voronezh oblast. In 2006, under the GP nurse training contract the Voronezh Medical College provided retraining to 105 specialists with secondary medical education for work in GP offices and wards. Totally 945 nurses of the oblast were trained in GP practice. In accordance with the Project Implementation Plan and Contract No. 4685/ .0/01 signed for organization of training activities in 2007-2008, 80 specialists of health facilities of the city of Voronezh and oblast rayons (including GPs) were trained in organization of general practices and health care system in general in Sweden, Denmark and the Netherlands.

In 2008, the number of day-care hospitals in the oblast outpatient facilities grew from 217 (2002) to 259 (9 months of 2008). The number of day-care beds grew from 2229 (2002) to 3768 (9 months of 2008) and amounted to 17.6% of the total number of hospital beds. The number of patients treated in them grew from 67046 (2002) to 119761 (2007) and to 86512 (nine months of 2008). Day-care bed occupancy rate was 288.0 days in 2002 and 299.4 days in 2007.

The oblast inpatient facilities reduced the number of day-care beds from 829 (2002) to 99 (2007) in line with the oblast health authorities decision regarding inexpediency of their use. 24-hour emergency care units were set up in 50 district hospitals and outpatient facilities; in 2008, these units provided care to over 59100 residents of the oblast rayons. Primary care doctors use 82 medical care provision standards developed at the federal level.

Follow-up activities include further implementation of general practices; creation of new jobs for general practitioners (family doctors); provision of necessary equipment and devices to them; turning them gradually into the core of the primary care sector and increasing their role in the overall health care system. Strengthening of preventive care within the primary care sector; reorganization of polyclinics into consultant-diagnostic and rehabilitation centers; and development of new forms of primary care and new forms of interaction with other levels of the health care sector. Continuation of training and retraining of specialists with higher medical education in general practice (family doctor) program; retraining and qualification advancement of GP nurses; public information and education in advantages of the general practice model; and active efforts to transfer the maximum possible volumes of care to outpatient care; further development of day-care hospitals, home inpatient care; setting up home care services; expansion

12

of outpatient surgery service; development of various forms of medical and social services (hospices, nursery homes, home care); setting up outpatient rehabilitation services as close to the place of residence as possible. Integration of pediatrics and obstetrics/genecology into the GP system; development of obstetrics/pediatrics therapeutic centers, i.e. transition to family doctor service; implementation of a GP payment system based on partial fund holding or incentives for achievement of the structural and qualitative performance indicators; and further development of GP the fund-holding principle.

Diagnostic Services

During the reporting period of the project, the strategic objectives of comprehensive restructuring of the Voronezh oblast diagnostic service included improvement of its organizational structure, setting up of inter-rayon diagnostic units and three inter-rayon (county) diagnostic centers and provision of modern equipment to them. Units for ultrasonic examinations, MRT, computer tomography, laparoscopic surgery, immunologic diagnosis, etc. were set up. The volumes of proven diagnostic examinations have increased. The Voronezh oblast Clinical Consultations and Diagnosis Center (VOCCDC) is the organizational, methodological and consultant center in the area of medical diagnostics. Off-the-staff-schedule diagnostic specialists are heads of respective departments in this center and other oblast health care facilities.

Diagnostic service issues and follow up activities: Inadequate staff provision; Lack of protocols for effective diagnostic equipment use; Lack of care continuity throughout all stages: from preventive check-ups in outpatient facilities and ending with treatment in specialized health facilities. Follow-up activities: Setting up a computer information system for individual laboratory and instrumental examination records; Bringing modern examination techniques to the primary care and their provision in the required volumes; Setting up of an automated workstation for endoscopy diagnosing; In general, it is necessary to approximate hi-tech diagnostic services to the primary care and provide access to qualified diagnostic care for the rural population; Development and broad implementation of diagnostic examination protocols; Setting up a quality assurance system for the diagnostic service; Unification of examination methods at all levels of medical care; Development of standardization system: development and implementation of standards for provision of laboratory and diagnostic equipment to health facilities; Continuation of procurement of modern medical equipment to gradually renovate physical assets of the diagnostic service; Streamlining the organizational structure of the inter-rayon (county) diagnostic centers; Improvement of operation of the oblast medical equipment maintenance service; development and implementation of training programs for the staff attending imported equipment; Organization of monitoring and evaluation of data about availability and technical condition of the laboratory and diagnostic equipment; Organization of diagnostic service staff training for work in new conditions. Inpatient CareThe project provided for rationalization of the inpatient care with due consideration of the Strategic plan goals and objectives. The number of inpatient facilities of the Voronezh oblast reduced over the period of 2002 to 2008 from 163 to 124 (by 39 hospitals or 24%). In 2008, overall the inpatient facilities had 20851 beds, including 15874 (69.2 %) in the municipal facilities (including 9549 in rural rayons and 6325 in the city of Voronezh); 6700, in the oblast facilities (21.9 %) and 2150, in the departmental facilities (8.9 %). Bed provision rate (without departmental hospitals) has reduced in the oblast from 120.8 in 1992 to 92.8 per 10 thousand population in 2007. In 2007, the city of Voronezh indicator reduced from 78.3 in 2002 to 49.4 per 10 thousand population in 2007 (in the Russian Federation, 96.4 in 2006; in CFO, 97.7 in 2006.

13

On the average, this indicator in the rural administrative rayons in 2008 (nine months) reached 60.4, and in the city of Voronezh, 49.4 (without beds of the oblast facilities). Inpatient care restructuring resulted in reduction of bed provision rate per 10 thousand population to 90.6 over nine months of 2008 (in the oblast rayons, to 60.4, in the city of Voronezh, to 49.4 in nine months of 2008). Implementation of modern medical technologies and intensification of the diagnostic and treatment process shortened the average stay in the hospital from 13.9 in 2002 to 11.8 days in 2008. The city of Voronezh shows a positive trend in the average stay in municipal hospitals: this indicator reduced from 14.1 days in 2002 to 10.1 days in 2007 (in the Russian Federation, 13.6l; in the CFO, 14.0 in 2006). The average occupancy rate of a hospital bed reduced from 320.2 in 2002 to 309.6 in 2007, while in the municipal hospitals of the city of Voronezh this indicator increased from 326.6 in 2002 to 330.8 in 2007 (in the Russian Federation, 317.0; in the CFO, 309.0 in 2006)

Issues of inpatient care organization and follow-up activities: Inpatient care volumes still exceed the norms recommended by the state benefit program. Lack of capacity of the outpatient care sector to take up additional medical care volumes, including outpatient surgery. Certain mentality of medical staff and patients who believe that inpatient care is more qualified and more effective. Inadequately developed network of rehabilitation and nursery-care hospitals. The proportion of beds of these profiles is also insignificant. Substantial difference in access to specialized and hi-tech medical care exist for residents of different municipalities. Follow-up activities include: Further reduction of the number of unjustified hospitalizations; Division of health care provision by levels: primary, secondary and tertiary to secure continuity of medical care provided to patients; Brining the inpatient bed stock in line with the estimated norms required for provision of state guaranteed volumes of medical care; Setting up of eight inter-rayon (county) hospitals (centers, specialized wards); Reduction of the average stay in a 24-hour inpatient facility (hospitalization on the date of operation); Implementation of modern treatment technologies of the evidence-based medicine; Rational use of tertiary level beds with brining their number to the estimated norms; transferring part of their current volumes to the county (inter-rayon) level; selection of patients for treatment; quoting places in tertiary inpatient facilities for residents of municipalities; Development of hi-tech care to respond to the health problems of the oblast population; Improvement of staff labor organization and wages; Improvement of primary and inpatient care providers payment methods.

Streamlining Provision of Human Resources for Medical Organizations

The oblast created a system of continuous postgraduate education (Institute of continuous postgraduate education) in the Voronezh Medical Academy named after N.N. Burdenko of the Federal Agency for Health care and Social Development. In 2005-2007, 5097 doctors upgraded their qualification.

In accordance with the project implementation plan and contract No. 4685/ .0/01 for organization and conducting training activities in 2007-2008, 25 health specialists of the city of Voronezh and oblast rayons were trained in Norway (studying experience of organization of hi-tech medical care), in Germany (financing of state-guaranteed medical care programs), Israel (studying experience of information support to medical care organization).

Emergency Care Service

In 2007, the Voronezh oblast emergency care service employed 218.25 round-the-clock teams (195 in 2002), including 60.9, in the city of Voronezh. There are 1.1 of such teams per 10 thousand population on the average in the oblast (while the norm is 1.0 per 10 thousand

14

population); and 0.68 team per 10 thousand population in the city of Voronezh. Professionally the ambulance teams are divided into feldsher, general doctor and specialized doctor teams. Over the last decade, the role and proportion of feldsher teams predominantly grew in the general structure of the service. The average number of calls per one thousand of population was 333.8 in 2007 (in 2002, 291.5); in nine months of 2008, 240.6. This indicator in the city of Voronezh is somewhat lower: 311.2 in 2007, which may be attributed to a better performance of the primary care; while in the oblast rayons this indicator is higher than the oblast average: 355.3 in 2007 (307.9 in 2002).

Timely arrival of the ambulance team is an integral indicator of the emergency care accessibility. The legally established norm of arrival is 15 minutes. The number of timely attended cases depends on the coverage radius, availability of teams and frequency of calls. In the oblast on average 83 % of calls are attended within the recommended 15 minutes (the overall timeliness was recorded in 80% of cases). The average time required for attending one call was 48.1 minutes. In 2007, the workload of a 24-hour team on the average in the oblast was 15 calls per day (in 2002 – 9.9). In 2002, in the city and in the rural areas this load was 11.3 and 10.9, respectively. The average hourly load that characterizes efficiency of the ambulance team performance is 0.38 in the rural areas, 0.46 in the city of Voronezh (in 2002, 0.47 in the city and 0.45 in the rural areas).

Follow-up activities: Continuation of modernization of physical assets of the emergency care service (ambulances, medical equipment, modern means of communication, telecommunications); Transfer of a certain volume of emergency care to the primary care; Involvement of the life support service staff (police, road police, fire brigades, industrial resuscitation instructors) and population in emergency care provision (at the pre-doctor level); Gradual transition from the emergency care principle “Provision of the maximum health care at the pre-hospital stage” to principle “Support of patient’s life functions and his prompt transportation to a health facility”.

B. Strengthening of Financial and Economic Management

B.1 Development of finance and expenditure planning models in health care system Subcomponent B.1 provided support to development of the basics of the social and economic analysis – macroeconomic models of finance and expenditure planning that provide projections of proposed health reform outcomes at the federal and regional levels. The following are some of the activities implemented within the framework of development of the first model:

1. Activities related to participation of representatives of Russia in seminars: National health accounts for the CIS, including the HIV/AIDS programs, held by the World Bank, USAID, PHRplus Project, UK DFID, ZdravPlus Project (Kazakhstan, Alma-Aty, December 8 - 13, 2003; Armenia, Yerevan, October 14 - 15, 2004; Georgia, Tbilisi, December 7 - 9, 2005; Kyrgyzstan, Bishkek, June 14 - 16, 2006) 2. 4. Technical assistance was provided to the MHSD in preparation of the regional health accounts implementation program. 5. Information and analysis system (IAS) was developed for analysis and projection of public health expenditures. 6. The software and The User and Administrator Manuals: Health Expenditures Analysis and Projection were provided to various specialists.

15

7. Outputs of Subcomponent B.1 (development and implementation of health accounts under) are used by the MHSD and Chamber of Accounts for preparation of reports.

8. Computers were provided to the MHSD for installation of the model in other subjects of the Russian Federation.

The following major activities were implemented within the framework of development of the second model (Contract No. 4685/ .1/02):1. Baseline data were analyzed and systematized to resolve the issues of the network rationalization and evaluation of the social and economic impact of restructuring; data sources and indicators were defined at the federal and regional levels. 2. Principles and forms of health care system governance and key legal regulatory acts regulating the health care system directly or having an indirect influence on it were reviewed. 3. Regional variations of the health care systems in two pilot regions (Voronezh oblast and Chuvash Republic) were studied. 4. Options of modeling and parameters of the network rationalization and restructuring impact evaluation model were defined, as well as software for the regional and municipal levels. 5. Prototype of model system and network rationalization and restructuring impact evaluation software was developed; key estimated norms were defined.

6. Training seminars were held for the health care system administration specialists of the Russian Federation and the pilot regions under a specially prepared program.

7. The software and model complex were piloted to identify inaccuracies, normally, caused by errors in the initial database.

8. The software was installed on the workstations of the users – specialists of the pilot region health care administration authorities and in the MHSD.

B.2 Health Economics Education

Professors for the health management, economics and planning sub-departments of doctor upgrading departments of Moscow higher education institutions were selected; training was financed for eight professors in health reform and sustainable financing at the flagship course organized by the World Bank Institute and Association of University Health Administration Programs (Moscow, May 24 – June 4, 2004). Health care specialists were trained through the regional flagship courses in health reform and sustainable financing that were held in: Cheboksary, Chuvash Republic (September 11-19, 2006) with 41 participants from health care sector, including 31 from the pilot regions; and in Voronezh, Voronezh oblast (May 28 – June 5, 2007) with 35 participants from the pilot regions. Training programs were well balanced: theoretical issues were complemented with reviews of practical experience of foreign countries and various regions of Russia. B.3 Strengthening financial management at institution level

1. Organization of the Federal Resource Center of Health Administration and Financial Management (Contract No. 4685/ .3/01)

2. Development of health sector management accounting system and software (Contract No.

4685/ .3-02)

16

B.4 Health Workers Payment and Contracting

The following major activities were implemented under this sub-component:1. Development of key workers payment and contracting reform areas (the” key areas”). 2. Development of analytical and methodical materials for pilot implementation of health workers payment system based on wage ranges by qualification levels of professional groups in state (municipal) health facilities 3. Development of methodical and analytical materials on labor norms in the health sector

. Health Policy and Governance

Legal regulatory framework under component C was primarily developed in the area of informatization of the health sector. To unify and standardize the process of development of medical information systems, the following activities were implemented.

1. Drafting national standard: “Informatization of the health sector. Data exchange formats for mutual settlement of accounts for treated patients. Requirements to medical information systems” 2. Drafting national standard: Informatization of the health sector. Data exchange formats for development of the consolidated register of the insured. Requirements to medical information systems”

3. Drafting national standard: Informatization of the health sector. Data exchange formats for development of the register of health facility electronic passports. Requirements to medical information systems”

4. Drafting national standard: Informatization of the health sector. Data exchange formats for collection and analysis of medical statistics. Requirements to medical information systems”

5. Translation of international standards on informatics (Contract No. 4685 / .0/02).

Project Closing Conference

National HRIP closing conference was held on December 10, 2008, in Moscow in the Ararat Park Hayat Hotel conference hall. The conference attracted great interest among workers of the RF health sector, which may be confirmed by the list of participants. Representatives of the following subjects of the Russian Federation attended the conference: i) Republics: Dagestan, Tatarstan, Tyva, Khakassia, Chuvashiya; ii) Krays: Krasnodar, Krasnoyarsk, Stavropol, Perm; iii) Oblasts:Astrakhan, Bryansk, Vologda, Volgograd, Voronezh, Kemerovo, Kirovskaya, Kostroma, Orel, Samara, Tula; and iv) Autonomous counties: Khanti-Mansi and Yamalo-Nenets.

The level of conference participants included Deputy Governor of the Yamalo-Nenets autonomous County; Two representatives of the MHSD; Twenty heads/deputy heads of the regional health authorities (health ministries, departments, committees and chief departments); Five heads/deputy heads of territorial MHIF; Six leading specialists of territorial medical information and analysis centers; Seven professors of higher education institutions; Five specialists of scientific research institutes; Seven workers of medical organizations.

Experts of the World Bank, USAID Health Department, representative of the Canadian Embassy and delegation from Azerbaijan participated in the conference. The total number of conference participants reached 75 persons.

The conference discussed the key outputs of consultant services provided under the project. Two pilot regions made presentations about their health systems development and noted a significant

17

project contribution in improvement of medical care organization and provision, in provision of health facilities with modern equipment. Representatives of the pilot regions also noted that training provided under the project to health specialists had a positive impact on the quality of medical care in the Chuvash Republic and Voronezh oblast.

Overall, conference participants noted great importance of the project outputs for development of the health reform methodical and institutional framework in the Russian Federation. The key project outputs will be posted on the site of TsNIIOIZ of the MHSD accessible to all interested organizations.

Factors that Influenced the Project Progress

1. Budget Situation

The budget situation in the Russian Federation as a whole and in the pilot regions in particular was rather stable. Co-financing funds were transferred against the RHCF requests on time and in full.

2. Government Health Care Initiatives

The major government initiative is the priority National Health Project (NHP) that is being implemented starting from 2006. The key areas of the NHP for 2009 – 2012 include: i) Promotion of healthy life-style; ii) Development of the primary care and improvement of preventive care; iii) Improvement of access to and quality of specialized care, including hi-tech medical care; iv) Improvement of mother-and-child care. Hence, the key activities aimed at development of the primary care under the National Health Project fully coincide with the Health Reform Implementation Project and contribute to implementation of these prospective policies across the country.

18

Annex 8. Comments of Co financiers and Other Partners/Stakeholders No additional comments.

Annex 9. List of Supporting Documents

1. Russia Health Care Foundation. 2006. Intermediate Project Evaluation Report.

2. Russia Health Care Foundation. 2007. Implementation Status Report.

3. Russian Health Care Foundation. 2008. Project Completion Report.

4. World Bank. 2002. IBRD IFC Country Assistance Strategy for the Russian Federation. Report No.24127.

5. World Bank. 2003. Loan Agreement between Russian Federation and

International Bank for Reconstruction and Development.

6. World Bank. 2003. Project Appraisal Document on Proposed Loan to the Russian Federation for a Health Reform Implementation Project.

7. World Bank. 2003-2009. HRIP Aide Memoires.

8. World Bank. 2003-2009. HRIP Implementation Status and Results Reports.

9. World Bank. 2003-2009. HRIP Mission Back to Office Reports.

10. World Bank. 2006. IBRD, IFC, MIGA Country Partnership Strategy for the

Russian Federation for the year FY2007- FY09. Report No. 37901-RU.

11. World Bank. 2007. Project Performance Assessment. Russian Federation Health Reform Pilot Project.

12. World Bank. 2009. IBRD, IFC, MIGA Country Partnership Strategy Progress

Report for FY2007- FY09. Report No. 49119-RU.

19

Annex 10. Additional Indicators.

Indicator Baseline Value Original Target Values (from approval documents)

Formally Revised Target

Values

Actual Value Achieved at Completion or Target Years

Indicator 1 : Development and approval by the MHSD of new clinical evidence-based protocols.


N/A 230 standards and protocols approved and enacted by MOHSD.

The MHSD developed and approved: - 88 standards for specialized care provision; - 40 protocols for medical care provision; - 310 standards for hi-tech health services provision;- 26 standards for spa and recreation care; - 46 standards for emergency care provision



Indicator 2 : Approval by the MHSD of new standardization and licensing criteria and procedures.


Not applicable. Regulatory and licensing tools in the health system developed.

RF Government Resolution of January 22, 2007, No. 30 “On approval of provisions for licensing in the health sector” (version 10.03.2006.) Order of Roszdravnadzor of March 14, 2005, No. 500PR/05 “On approval of provisions for establishment by the subjects of the Russian Federation of Roszdravnadzor committees for licensing in health care, pharmaceutical activities and activities related to drugs and psychotropic substances” Order of Roszdravnadzor of May 4, 2005, No. 1115PR/05 “On issue of licenses for manufacturing of medical equipment” Order of Roszdravnadzor of April, 2006, No. 979-PR/06 “On creation of the Central Licensing Committee of the Federal Health Care and Social Development Supervision Service”



Indicator 3 : Development of national and regional models of finance.


National and regional models of finance developed.

Models and relevant software were developed. The software was installed in the health authorities of the pilot regions, MHSD, Roszdrav TsNIIOIZ. Methodical recommendations and training aids were prepared for introduction of national and regional health accounts; software was developed for introduction of the regional health accounts at the level of the subject of the Russian Federation. Based on this, the MHSD and the Chamber of Accounts have data for analysis of health expenditures efficiency.



Indicator 4 : Development and approval of documents for rational use of human resources at the regional level Value quantitative or Qualitative)

Documents on rational use of human resources at the regional level developed and approved.

Development of methodologies for measuring labor intensiveness of activities including standards of medical services undertaken by medical workers.

20

Identified time standards to perform basic and comprehensive medical services and prepared corresponding guidelines. Defined rational standards of workload depending on specialty and location.



Indicator 5 : Development and approval of health workers’ payment system at the regional level


Health workers’ payment system at the regional level developed and approved.

The following documents were prepared and approved by the Labor Relations and State Civil Service Department of the MHSD: - draft of sample provisions for workers’ payment and incentives in state (municipal) health facilities; - draft methodical recommendations on organization of the new payment system for health facility managers;- package of methodical documents on pilot implementation of health workers’ payment and incentives system. The following regulatory documents of the federal level were adopted: RF Government Resolution of August 5, 2008, No. 583 “On introduction of new systems of payment of workers of federal budget institutions and federal state bodies";

MHSD Order of August 14, 2008, No. 424n “On approval of recommendations for contracting workers of federal budget institutions and contract sample ";

MHSD Order of August 14, 2008, No. 425n “On approval of recommendations for development by the federal government authorities and institutions – main federal budget administrators of sample provisions for payment of workers of subordinate federal budget institutions "; MHSD Order of August 27, 2008, No. 450n “On approval of methodical recommendations for development by the federal government authorities of conditions of payment of their workers and workers of their territorial bodies". Chuvash Republic:- Resolution of the Cabinet of the Chuvash Republic No. 236 of September 25, 2007 “On procedures for payment of workers of government institutions of the Chuvash Republic participating in implementation of the pilot project aimed at improvement of health care quality” - Order of the ChR MHSD No. 628 of September 26, 2007, “On performance indicators of staff of state health facilities of the Chuvash Republic participating in implementation of the pilot project aimed at improvement of health care quality” Voronezh oblast:Decree of the Voronezh oblast administration of January 21, 2005, No. 19 (with amendments of November 3, 2006, and September 25, 2007) “On

21

payment of health care system workers in the Voronezh oblast”



Indicator 6 : Number of health facilities that meet federal standards of medical care provision:


Health facilities meet federal standards of medical care provision.

Voronezh oblast:Use in health facilities of oblast disease management standards, 14 federal disease management protocols, 82 federal outpatient care standards and 58 inpatient disease management standards adapted to regional conditions. Chuvash Republic:100 federal outpatient care standards and 120 federal inpatient disease management standards adapted to regional conditions.



Indicator 7 : Decrease by 10% in the average length of stay in hospitals in pilot regions.



Ongoing process of quality improvement.



The 10% decrease target was met in Voronezh and in Chuvashia. In both cases the length of stay is lower than of Russia overall.

Indicator 8 : Increase in the percentage of daycare beds out of the total hospital beds in pilot regions.




Total number of inpatient beds:

Voronezh: 19436 Chuvash: 11172 Day Care Beds: Voronezh: 3798 Chuvashia: 2449 % of day care beds: Voronezh: 19.5 Chuvashia: 18.0


The increase in number and percentage of day care beds as a proportion of inpatient beds has been achieved.

Indicator 9 : Average occupancy rate of inpatient beds.


General Therapy:Voronezh: 315.5 13.1 Chuvashia: 333.4 13.2


General Therapy: Voronezh: 306.7 11.2 Chuvashia: 338.5

22

Cardiology: Voronezh: 345.9 13.8 Chuvash: 341.1 14.2 Cardio-surgery:Voronezh: 346.0 13.8 Chuvashia: 328.0 16.1 Urology:Voronezh: 327.5 11.6 Chuvashia: 286.0 13.2

11.8 Cardiology: Voronezh: 343.9 11.7 Chuvash: 331.8 12.5 Cardio-surgery:Voronezh: 311.4 9.9 Chuvashia: 340.9 13.7 Urology:Voronezh: 344.7 10.6 Chuvashia: 326.3 11.6


Occupancy rate of a hospital bed largely depends on its profile. Data on bed profiles with different intensity of the treatment process show a common trend of reduction of the average stay in hospital in both regions.

Indicator 10 : Reduction of the number of emergency/ ambulance calls (shifting to the primary care), per 1000 pop Value quantitative or Qualitative)



Voronezh: 321.2 Chuvashia: 255.0


The indicator shows a reduction in Chuvashia and an increase in Voronezh.

Indicator 11 : Reduction of the hospitalization number: i) Number of visits to outpatient facilities (per person); and ii) Number of hospitalizations (per 100 persons)


i) Voronezh: 9.3 Chuvashia: 10.8



ii) Voronezh: 24.4 Chuvashia: 23.8

Voronezh: 23.8 Chuvashia: 22.8


Indicator 12 : Reduction of the percent of patients repeatedly hospitalized due to same condition within three months. Value quantitative or Qualitative)





Indicator shows a reduction in both Chuvashia and Voronezh.

Indicator 13 : Increase in the proportion of surgeries in day care hospitals as proportion of total number of surgical operations. Value Voronezh: 5.5 Ongoing process of quality Voronezh: 60.1

23

quantitative or Qualitative)

Chuvashia:12.1 improvement. Chuvashia:19.9


Indicator shows a large shift in use of day care hospitals for surgical operations.

Indicator 14 : Increase in the number of nursery care beds. Value quantitative or Qualitative)

Voronezh: 250 Chuvashia: 45


Voronezh: 115 Chuvashia: 85


In Voronezh oblast the number of nursery care beds reduced due to transfer of some district hospitals that operated as nursery care hospitals to the social protection system

Indicator 15 : Increase in the proportion of primary and outpatient care expenditures.


Primary Care Voronezh: 42.2 Chuvashia: 30.5 Inpatient Care Voronezh: 57.8 Chuvashia: 69.5


Primary Care Voronezh: 53.0 Chuvashia: 46 Inpatient Care Voronezh: 47 Chuvashia: 54


I. NORTH1. Arkhangel2. Nenetz3. Karelia4. Komi5. Murmansk6. Vologda

III. CENTRAL12. Bryansk13. Ivanovo14. Kaluga15. Kostroma16. Moscow17. Moscow City18. Orel19. Ryazan20. Smolensk21. Tver

IV. CENTRAL CHERNOZYOM25. Belgorod26. Kursk27. Lipetsk28. Tambov29. Voronezh

IX. WESTERN SIBERIA61. Altai62. Gorny Altai63. Kemerovo64. Novosibirsk65. Omsk66. Tomsk67. Tymen68. Khanty-Mansi69. Yamalo-Nenets

VI. VOLGA40. Astrakhan41. Kalmykia- Khalmg Tangch42. Penza43. Samara44. Saratov45. Tatarstan46. Volgograd47. Ulyanovsk

VII. VOLGO-VYATKA48. Chuvash49. Kirov50. Mariy El51. Mordov52. Nizhniy Novgorod

X. EASTERN SIBERIA70. Buryat71. Chita72. Agin Buryat73. Irkutsk74. Ust-Ordyn Buryat75. Krasnoyarsk76. Evenk77. Khakas78. Taimir79. Tuva

XI. FAR EAST80. Amur81. Kamchatka82. Koryak83. Khabarovsk84. Jewish AO85. Magadan86. Chukot87. Primorski88. Sakhalin89. Sacha (Yakut)

VIII. URAL53. Bashkortostan54. Chelyabinsk55. Kurgan56. Orenburg57. Perm58. Komi-Permiak59. Udmurt60. Sverdiovsk

II. NORTHWEST7. Novgorod8. Pskov9. Leningrad10. St.Petersburg City11. Kaliningrad

V. NORTH CAUCASUS30. Chechen31. Daghestan32. Ingush33. Kabardino-Balkar34. Krasnodar35. Adygeya36. North Ossetia37. Rostov38. Stravropol39. Karachaevo-Cherkess

22. Tula23. Vladimir24. Yaroslavl

31

3032

33

39

36

35

34

3841 40

37

46

29

25

44

28

42

19

27

2218

26

12 1416

17

51

5223

13

20

621

78

9

10

3

5

1

24

15

4958

4

5759

5048

47

43

45 60

5554

53

56

67

68

2

69

66

65

78

76

75

64

61

62

77

63 74

79

73

70 71

80

83

84

87

88

72

89

85

81

82

86

11

0 100 200 300 400 500 MILES

0 200 400 600 800 KILOMETERS

IBRD 27188R

LITHUANIA

L AT V I A

ESTONIA

UNITEDKINGDOM

DENMARKNETHERLANDS

GERMANY

CZE

CH

REP

.

SLOVAK REP.

P O L A N D

HUNGARY

N O R W A Y

S W E D E N

F I N L A N D

BELARUS

MO

LDO

VA

GEORGIA

ARM

ENIA

AZERBAIJANISLAMIC REP.OF IRANIRAQ

SYRIANARAB REP.

UZBEKISTAN

T U R K E Y

K A Z A K H S T A N

D.P.R.OF

KOREA

C H I N A

M O N G O L I A

JAPAN

UNITED STATES OF AMERICA

East SiberianSea

BeringStrai t

ChukchiSea

B e r i n g

S e a

L a p t e v

S e a

K a r a S e a

B a r e n t s

S e a

N o r w e g i a n S e a

G u l f o f B o t h n i a

B a l t i c S e a

N o r t hS e a

B l a c k

S e a

Caspian

Sea

AralSea

LakeBaikal

S e ao f

J a p a n

S e a o f

O k h o t s k

Am

ur R

.

Kolym

a R.

Indigirka

R .

Lena R.

Amur

R.

Enisei R. Angara R.

Ob R.

Ob R.

Tobo

l R.

Volga

R.

RUSSIAN FEDERATIONREPUBLICS, KRAIS, OBLASTS, OKRUGS

NATIONAL CAPITAL

RIVERS

ECONOMIC REGION BOUNDARIES

OBLAST, KRAI, OR REPUBLIC BOUNDARIES

AUTONOMOUS OBLAST, OKRUG, OR REPUBLIC BOUNDARIES*

INTERNATIONAL BOUNDARIES

Arc t i c C i rc le

Moscow

The boundaries, colors, denominationsand any other information shown onthis map do not imply, on the part ofThe World Bank Group, any judgmenton the legal status of any territory, orany endorsement or acceptance ofsuch boundaries.

U K R A I N EROMANIA

RUSSIANFEDERATION

DECEMBER 1995

RU S S I A N F E D E R A T I O

N

* Including republics of Adygeya, Altai, Karachaevo-Cherkess, and Khakasiya