P R O J E C T P E R F O R M A N C E A S S E S S M E N T

December 2013

IFAD Internal Printing Services

People's Republic of China

Rural Finance Sector Programme

International Fund for Agricultural Development

Via Paolo di Dono, 44 - 00142 Rome, ItalyTel: +39 06 54591 - Fax: +39 06 5043463E-mail: evaluation@ifad.orgwww.ifad.org/evaluationwww.ruralpovertyportal.org

ifad-un.blogspot.com

www.facebook.com/ifad

www.twitter.com/ifadnews

www.youtube.com/user/ifadTV

Independent Offi ce of Evaluation

December 2013 Report No. 3041-CN Document of the International Fund for Agricultural Development

Independent Office of Evaluation

People’s Republic of China

Rural Finance Sector Programme

Project Performance Assessment

This report is a product of staff of the Independent Office of Evaluation of IFAD and the findings and conclu-sions expressed herein do not necessarily reflect the views of its Member States or their representatives to its Executive Board. The designations employed and the presentation of material in this publication do not imply the expression of any opinion whatsoever on the part of IFAD concerning the legal status of any country, terri-tory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. The designa-tions “developed” and “developing” countries are intended for statistical convenience and do not necessarily express a judgement about the stage reached by a particular country or area in the development process.

All rights reserved. ©2013 by the International Fund for Agricultural Development (IFAD)

Preface

China is the fourth largest country in the world and home to more than 1.3 billion

people. Since the start of far-reaching economic reforms in the late 1970s, China has

witnessed unparalleled economic growth that has fuelled a remarkable increase in per

capita income and a decline in poverty. The economy has performed well in recent years,

even against the backdrop of the worldwide financial crisis and soaring food prices.

Despite its remarkable progress in economic and social development and poverty

reduction, China still faces many challenges to reduce residual poverty. Disparities in

income among provinces and between urban and rural areas have been widening and

poverty remains primarily a rural phenomenon.

The purpose of this project performance assessment is to assess the results and

impact of the Rural Finance Sector Programme and gather lessons that would improve

the design, implementation and impact of future IFAD operations.

The project contributed to increased access to finance by the poor, women and

small producers. It also contributed to enhancing the livelihoods of rural households in

the project counties by an increase in household assets and production, improvement in

food security, and in the strengthening of human and social capital and socio-economic

status of women. However, the project originally designed as a sector project to

participate in systemic policy reforms could not do so due to a lack of proper institutional

arrangements, rapid changing nature of the ongoing reform process and inadequate

grasp of the reform issues and institutional complexity of the rural finance sector in

China.

Special thanks are due to the project management offices of Chongqing Rural

Commercial Bank and Shaanxi Provincial Rural Credit Cooperative Union, and senior

management and field officials of both organizations for their excellent cooperation, for

providing data and assistance without which it would have been impossible to complete

this assessment. The beneficiaries deserve special mention for sharing their views and

experience with the evaluation mission.

This project performance assessment was prepared by Mark Keating, Evaluation

Officer, with inputs from Dewan A. H. Alamgir, rural finance specialist consultant and

Linda Danielsson, Evaluation Assistant. Konstantin Atanesyan, former Senior Evaluation

Officer, and Anne-Marie Lambert, Senior Evaluation Officer provided comments on the

draft report. This Office is grateful to the Asia and the Pacific Division of IFAD for the

input and support provided throughout the evaluation process, as well as to the

Government of China for its constructive collaboration. The evaluation would like to

thank Sun Yinhong and Weijing Wang of the IFAD China Office for their excellent support

during the field visits.

Contents

Currency equivalent, weights and measures ii

Abbreviations and acronyms ii

Map of the project area iii

Executive summary iv

I. Background and methodology 1

II. The project 2

A. The project context 2 B. Project implementation 4

III. Review of findings 11

A. Project performance 11 B. Rural poverty impact 16 C. Other performance criteria 19 D. Performance of partners 22 E. Overall project achievement 23

IV. Conclusions and recommendations 23

Annexes

I. Rating comparison 26

II. Basic project data 27

III. Terms of reference 28

IV. Methodological note on project performance assessments 33

V. Definition of the evaluation criteria used by IOE 37

VI. List of key persons met 38

VII. Bibliography 39

ii

Currency equivalent, weights and measures

Currency equivalent

Currency unit = Chinese CNY

US$ 1.00 = CNY 6.3 (as of June 2012)

Weights and measures

1 kilometre (km) = 0.62 miles

1 metre (m) = 1.09 yards 1 hectare (Ha) = 10.000 m2 (0.01km2) 1 hectare (Ha) = 2.47 acres 1 acre (ac) = 0.405 hectares (ha) 1 kilogram (kg) = 2.204 pounds 1 pound (lb) = 450 grams (gr)

1 ton = 1000 kg

1 gallon (gl) = 3.785 litres (lt) 1 m3 = 35.3146 ft³ 1 m3 1000 litres (lt)

Abbreviations and acronyms

CBRC China Banking Regulatory Commission

CRC Bank Chongqing Rural Commercial Bank

GDP Gross Domestic Product

IFAD International Fund for Agricultural Development

MTR mid-term review

NPL non-performing loan

PCR project completion report

PCRV project completion report validation

PMO Project Management Office

PRCCU Provincial Rural Credit Cooperative Union

RCB Rural Commercial Bank

RCopB Rural Cooperative Bank

RCC rural credit cooperative

RCCU rural credit cooperative union

RFSP Rural Finance Sector Programme

RIMS Results and Impact Management System

UNOPS United Nations Office for Project Services

iii

Map of the project area

iv

Executive summary

1. The Independent Office of Evaluation of IFAD (IOE) conducted a project

performance assessment (PPA) of the IFAD-funded Rural Finance Sector

Programme (RFSP) in China in June-July 2012. The assessment comprised the

review of key project and other official IFAD documents, as well as other relevant

literature, and a short mission to China where the evaluation mission met with

project partners, relevant stakeholders and beneficiaries, and observed ongoing

activities following project closing. The RFSP was designed in 2002/03, approved

by the IFAD Executive Board in April 2004 and became effective in September

2005. The project completed all of its activities in March 2010, six months after the

original completion date, and submitted a project completion report (PCR) prepared

by the Ministry of Finance in 2010.

2. The project was designed at a time when the Government of China had been

carrying out a major reform of its rural financial sector. The RFSP concept

envisaged participation in the reform process by assisting the government to

introduce systemic changes in the rural finance sector. At the time of project

design, the rural finance sector in China was managed by government-owned,

underperforming networks of rural credit cooperatives (RCCs). The China Banking

Regulatory Commission (CBRC) spearheaded reforms that focused on

a) developing clear corporate structures for the RCCs at provincial and county

levels; b) developing mixed ownership (government and private); c) improving

profitability by enhancing operating efficiency and reducing non-performing assets;

and d) increasing access to credit. These objectives have largely been achieved by

converting many RCCs into rural commercial banks and cooperative banks.

3. In this context, the RFSP was designed to partner with CBRC to contribute to

reforming the rural financial sector. This was to be achieved by converting the

network of rural credit cooperatives into sustainable microfinance institutions that

are accessible to poor rural people, women in particular. The programme

comprised five components: (a) policy development; (b) institutional development;

(c) operational development; (d) financing; and (e) programme management.

4. Overall, the evaluation found that the project contributed to strengthening

institutional, financial, and operational capacity of the two implementing financial

institutions – Chongqing Rural Commercial Bank and Shaanxi Rural Credit

Cooperative Union network – that led to increased access to finance by the poor,

women and small producers. It also contributed to enhancing the livelihoods of

rural households in the counties covered by the project. Impact studies showed

increase in household assets and production, improvement in food security, and

strengthening of human and social capital and socio-economic status of women. At

the same time, the positive impacts on households cannot be fully attributed to the

project alone. The two financial institutions are institutionally and financially viable

to continue offering financial services to poor smallholders. However, the

participation of the project in the systemic policy reforms led by CBRC did not fully

materialize due to a lack of proper institutional arrangements, rapidly changing

nature of the on-going reform process and inadequate grasp of the reform issues,

and institutional complexity of the rural finance sector in China. The project

concept was relevant but design was ambitious and weak in terms of institutional

arrangements.

5. The evaluation noted that there is a need and scope for future reform in the rural

finance sector in China. A number of issues, such as: (i) lack of strong competition

in the rural areas; (ii) potential effectiveness of village and township banks as they

are initiated by RCCs or RCBs; (iii) ownership structure of various institutions and

prudential regulatory issues; and (iv) strengthening of governance are all areas

where further research and reforms are needed. The evaluation recommends for

v

IFAD to maintain regular interactions with CBRC and the Ministry of Finance to

explore possibility of participating in future rural finance sector reforms.

1

People’s Republic of China Rural Finance Sector Programme Project Performance Assessment

I. Background and methodology 1. This report presents the findings and recommendations of the project performance

assessment (PPA) of the Rural Finance Sector Programme (RFSP) funded by IFAD

in the People’s Republic of China. IFAD’s Independent Office of Evaluation (IOE)

conducted the PPA during June-July 2012. The RFSP was designed in 2002/03, it

was approved by IFAD’s Executive Board in 2004 and became effective in 2005.

The project completed its activities in 2009/10 and the Government prepared a

project completion report (PCR) in 2010 through the implementing agency, the

Ministry of Finance. IOE conducted a PCR validation (PCRV) of the project and

selected it as a sample project for conducting a full PPA.

2. The PPA is carried out to: a) provide an independent assessment of the overall

results and impact of the project for accountability and management purposes;

b) distil lessons learned identifying key explanatory factors of project performance

and poverty reduction results for learning purposes; and c) gather additional

evidence on the major information gaps, inconsistencies or analytical weaknesses

of the PCRV. Under the current IFAD Evaluation Policy of 2011,1 IOE undertakes

PPAs for selected projects instead of project evaluations as was done in the past.

The findings and recommendations also benefit the design and implementation of

ongoing and future operations within the country.

3. IOE conducts a PPA as a follow-on step after producing a project completion report

validation (PCRV) report through desk review of various documents, including the

PCR, official project documentation, impact studies and other informative literature.

The PCRV serves the following purposes: (i) independent verification of the

analytical quality of the PCR; (ii) independent review of project performance and

results through desk review; and (iii) extrapolation of key substantive findings and

lessons learnt for further synthesis and systematisation. A PPA includes country

visit in order to complement the PCRV findings and fill in information gaps

identified by the PCRV.

4. The PPA applies the evaluation criteria outlined in the IFAD Evaluation Manual.2 In

view of the time and resources available, a PPA is generally not expected to

undertake quantitative surveys. Based on the initial findings of PCRV process, the

PPA rather adds analysis based on interviews at IFAD headquarters, individual

interviews and discussions with stakeholders in the country including project

beneficiaries, and direct observations in the field. The PPA process relies on the

data available from the project monitoring and evaluation system.

5. Methodology and process. The following methodology and process were followed in

conducting the PPA: a) review of relevant documents (see bibliography in annex

2); b) interviews with project staff and other key resource persons from the

executing and implementing agencies; c) interviews with sample project

beneficiaries; d) interviews with IFAD staff in IFAD Headquarters and in country

office; and e) interviews with sector experts as well as other multilateral agencies

involved in poverty reduction and the rural finance sector.

6. Prior to undertaking the PPA mission, key areas and data gaps were identified to be

addressed during the country visit. Additional primary data were collected in the

field to validate the findings and conclusions of the PCRV and to allow for an

independent assessment of project performance. Given time constraints, a

1 Available at: http://www.ifad.org/gbdocs/eb/102/e/EB-2011-102-R-7-Rev-1.pdf.

2 Please refer to annex 1 in this document.

2

qualitative approach was adopted for data collection. As the project continued its

activities after project completion in 2010, the two implementing financial

institutions – Chongqing Rural Commercial Bank (CRCBank) and Shaanxi Provincial

Rural Credit Cooperative (RCC) – were able to provide the PPA team with additional

data, in some cases covering activities up to 2012.

7. The PPA field mission3 in China was undertaken between 12 and 26 June, 2012.

Following a series of meetings in the capital city of Beijing, the evaluation team

visited: a) Chongqing Municipality and Youyang county, and b) Shaanxi province

(Xi’an city) and Xixian county to observe activities of the CRC Bank, the Shaanxi

Provincial Rural Credit Cooperative Union, and the Xixian Rural Cooperative Bank

(PRCCU) respectively. Two debriefing meetings were held in Chongqing and in Xi’an

city following the field visits. A wrap-up meeting was conducted in Beijing with the

Ministry of Finance to brief the Government about the preliminary findings of the

PPA mission and gather government’s views. The mission schedule and list of

persons met is attached to this report as annex 3.

II. The project

A. The project context

8. Poverty reduction and economic development. RFSP was designed and

implemented in an era when China experienced fast economic growth and rapid

poverty reduction. Contributing factors included special government programmes

to reduce rural poverty by focusing on farmers and numerous other reforms that

encouraged local initiatives. Financial sector reforms were also part of the overall

reform effort. China maintained an average annual growth of 10 per cent Gross

Domestic Product (GDP) for nearly thirty years (1980-2008). In 2009, despite the

global financial crisis, the country recorded a GDP growth rate of 8.7 per cent. GDP

per capita increased as well, from US$ equivalent 3,324 in 2008 to 3,710.72 in

2009. Rural income per capita in 2009 followed a similar trend, amounting to

US$756,68 or an 8.3 per cent increase from the previous year.4 The Government

announced the successful reduction of the number of absolute poor from 260

million in 1978 to about 14 million in 2008, based on the previous poverty line of

CNY785 as rural income per capita. In 2008, the Government increased its poverty

line standard to CNY 1,196. This in turn expanded the category officially recognized

as the poor to more than 40 million people and included also the low-income

stratum, which had been accounted for separately in previous years. The

Government of China has been drafting an action plan for the period 2011-2020 to

ensure the eradication of absolute poverty in the country. To this end, substantial

financial and other resources are expected to be allocated for this purpose. It is

important to note in this context that poverty in China also holds a geographical

dimension: central and western provinces are still considered poorer than eastern

provinces. Any future intervention by IFAD and/or other donors needs to consider

the geographical concentration of poverty, especially in remote and isolated areas.

9. Rural financial sector reform. RFSP was designed when the Government undertook

major reforms of the rural financial sector. The sector has been dominated by a

few key actors, most notably the Agricultural Bank of China (ABC), the Agricultural

Development Bank of China (ADBC), as well as thousands of rural credit

cooperatives (RCCs). All these institutions were initially regulated by the People’s

Bank of China (PBC - Chinese Central Bank). A critical institutional reform was the

establishment of the Chinese Banking Regulatory Commission (CBRC), which was

mandated, among others, to manage RCCs. Later on in 2005/06 CBRC has become

the only regulatory body of RCCs and banks instead of directly managing them.

3 Team Composition: Dewan A.H. Alamgir, Rural Finance Specialist (Consultant) and Mark Keating, Evaluation Officer,

IOE (IFAD). Mr Konstantin Atanesyan, Senior Evaluation Officer, IOE (IFAD) joined the mission on 22 June 2012 in Xi’an, Shaanxi province. Mr Sun Yinhong, IFAD Country Presence Officer and Ms. Weijing Wang, Program Officer supported the mission in planning and organization and shared their experience of the project. 4 Economist Intelligence Unit. China Country Profile, 2011.

3

10. A summary of the several changes which took place in the management of ABC

and RCCs is provided to clarify the context and performance of RFSP. RCCs

originally acted as branches of ABC. During this period, the RCC network suffered

from high share of non-performing assets, high losses, undercapitalized RCCs and

inadequately skilled staff members. These issues were related to the prevailing

policies of directed credits, controlled lending interest rate, unclear ownership, lack

of incentives to make profits and the focus on providing loans to state-owned

enterprises. The challenge of the reform implemented by CBRC was to give the

RCCs a corporate structure with clear ownership to be driven by performance

(profitability).

11. The reform process started with the release of the RCC Reform Plan by the State

Council in June 2003. The main objectives were to restructure and define

ownership and governance of the RCCs of different sizes and performance

categories and to establish a more efficient rural finance system. Since 2005, the

reform accelerated and selective RCCs were restructured into rural commercial

banks, rural cooperative banks, and rural credit cooperatives. This process

continues to date with more and more RCCUs at county level being converted to

RCBs. Also, former provincial RCCUs (RCC unions) were re-organized into regional

financial institutions, in the form of rural commercial banks, rural cooperative

banks or provincial RCCUs, depending on the market potential and asset quality of

each provincial network. County RCCUs were converted to RCBs, as sub-branches

of provincial RCBs or status quo RCCUs, and township RCCs were restructured as

service outlets of county RCCUs. By 2010 RCC organizational conversion or

restructuring was completed in most provinces. Notably, operating efficiency and

network capacity differs from one province to another, mainly due to market

conditions, capitalization of the RCCs and their differing traditions of service in the

different provinces of China. Following years of reform and change, RCC networks

remain the most extensive service providers in the rural areas: As of June 2012

China has 230 rural commercial banks, 180 rural cooperative banks and 2,265

RCCs.

12. In addition to the above, in the period 2007-08 new types of smaller rural financial

institutions were licensed: village and township banks (VTB), microcredit

companies, and rural mutual fund associations. One requirement for VTB licensing

is that it has to be initiated by a rural commercial bank. As a very successful and

one of the largest rural commercial banks, the Chongqing Rural Commercial Bank,

an RFSP partner, has already established two VTBs outside the Chongqing

province. As of mid-2012, more than 726 VTBs are in operation.

13. Impact of CBRC-led reforms:

Ownership is clear because of well-defined corporate structure and

shareholdings. Private investors have also bought shares of rural

commercial banks. Notably, the Chongqing Rural Commercial Bank

developed into a commercial bank from an RCC as consequence of the

reform process and in 2011 enlisted in the Hong Kong Stock Market;

Profitability has improved. Average return on assets and return on equity

are estimated to be 1.2 per cent and 19.2 per cent respectively;

Non-performing assets have significantly declined, and according to CBRC,

the share of non-performing loans is 1.8 per cent;

Management capacity and internal management systems were improved

and streamlined. Management information systems are computerized; and

Capitalization has improved as per new requirements.

14. Weakness in rural finance sector. Even with these different types of financial

institutions, access to finance for the poor is still low, especially in western and

central provinces. RCBs, Rural Cooperative Bank (RCopBs), RCCs have traditionally

serviced private clients that were considered non-poor as well as larger corporate

4

rural clients, including state-owned enterprises. Without outside stimulus, it is

likely that the RCC networks will continue to neglect lower income farmers, rural

households and rural micro enterprises, above all since the pressure for generating

profits has increased for the RCCs after the reforms.

15. Future direction of reform. The reform agenda is driven centrally by CBRC and the

following still needs to be considered for future reform steps: liberalization of

savings and lending interest rates; further privatization of rural commercial banks

and cooperative banks; risk categorization and loan loss provision in line with

international standards.

16. RFSP Design. The Rural Finance Sector Programme (RFSP) was designed in

2002/2003, approved by the IFAD Executive Board on 21 April 2004 and became

effective on 13 September 2005. The programme financing was estimated at a

total of US$ 21.3 million, with an IFAD loan amounting to Special Drawing

Rrights 9.95 million (US$ 14.7 million equivalent). Due to appreciation of the

Chinese currency CNY, the project cost in US$ terms has changed to US$30.4

million at completion. The programme objective was to enable poor rural people to

improve their livelihoods by increasing their access to financial services.

17. The project was designed at a time when the government and CRBC had been

undertaking major reforms of their own for the whole rural finance sector. To

reflect the reform agenda, the project design document (appraisal report) had been

substantially revised with the expectation that the project would be implemented

by CBRC and lessons from the project could be fed into the reform process.

However, that expectation did not materialize as CBRC focused on the regulatory

framework for the whole sector rather than becoming involved in the management

of RCCs, and therefore declined to host and implement the project. At the ultimate

client level, the entry rationale for the project was the significant unmet demand

among the lower and low income sections of the rural population in the two project

areas. IFAD supported the rural finance sector and government’s policy to reform

rural credit cooperatives (RCCs). The purpose of the project was to turn the credit

cooperatives into sustainable microfinance institutions for the benefit of poor rural

people and women in particular. The project was designed for a period of four

years, initially with two counties participating from each project province

(Chongqing and Shaanxi). To accelerate IFAD loan disbursements, two additional

counties joined the project in Shaanxi in 2007, as reflected in the Loan Amendment

and following the recommendation of the Mid-Term Review.

18. Initial project implementation up to MTR was very slow due to various institutional

reasons, and the Mid-Term Review in April 2007 proposed a number of changes

and implementation modalities to accelerate implementation. After MTR,

implementation progressed but disbursement was still slow as the initial completion

date of 30 September 2009 was approaching fast. Upon the request of the

Government, IFAD extended RFSP’s completion to 31 March 2010 and closing to 30

September 2010.

B. Project implementation

19. Implementation Approach. In the above context, RFSP adopted a strategy of

working in concert with the CBRC to contribute to policy reforms, experiment with

the impact of these reforms in two provinces, strengthen institutional and

operational capacities of the two provincial RCC networks, and enhance access to

finance for the poor, women and farmers. To achieve these objectives, the focus

was on providing technical assistance in policy areas, capacity building assistance

to upgrade systems and human resources and lines of credit to expand outreach of

services. Credit lines were provided specifically to help catalyze innovative

microfinance practices and to demonstrate the operational viability of well-designed

credit products. But, as mentioned earlier, CBRC eventual non-participation forced

5

the project to work directly with the two provincial RCC networks. This affected

outcomes of Component 1 of the project.

20. Components at appraisal. The Appraisal Report included the following five

components, to which some adjustments were made at the time of the MTR:

Component 1: Policy development. key outputs and activities were:

(a) Lending rate liberalization;

(b) Microfinance group lending, women window;

(c) New product development;

(d) Technical and consultancy support; and

(e) Policy support and technical advice at national level.

Component 2: Institutional development. Key outputs and activities

were:

(a) Infrastructure and equipment;

(b) Staff training; and

(c) Reorganizing legal structure of RCCs.

Component 3: Operational development. Key outputs and activities

were:

(a) Accounting system and loan policies; and

(b) Staff incentive system.

Component 4: Financing (Line of credit). A credit line (re-financing)

was provided to help test innovative products, and to promote loans to the

poor and women.

Component 5: Programme management. Unusually, two parallel Project

Management Offices (PMOs) were established at provincial level with County

PMOs in each county covered by RFSP. The PMOs were to ensure

programme implementation. A National Coordination and Management

Office (NCMO) was proposed to maintain the overall supervision of the pilot

activities in the programme provinces. It was also to advise on national

policies and policy reform, coordinating technical assistance for studies and

staff training of the programme RCCUs. The NCMO including CBRC was

never constituted and the two provincial PMOs had to take the lead in

programme implementation.

Review of performance

1. Component 1- Policy development. The RFSP differs from other IFAD-supported

interventions, which are area-based poverty reduction projects, in the past often

containing lines of credit for pro-poor RCC outreach. In RFSP, IFAD was aiming at

helping to institute systemic, sector-wide change through influencing on-going

reform in the sector under the leading role of CBRC. This did not materialize, and

the pivotal Component 1 has become the least successful of all components. The

PCR does not clearly describe the reasons behind such situation. The following

paragraphs briefly present an analysis of the related issues and factors that led to

such situation to benefit future IFAD project design and implementation.

2. At the time of design, the Ministry of Agriculture was the proposed executing

ministry for the project, but later handed over the project to the Ministry of

Finance. This created a gap regarding the understanding about the purpose,

process of implementation as well as scope of the project. Representatives from

both the Ministry of Finance and the CBRC had associated with IFAD design

missions, but their participation did not percolate up to their respective institutions.

6

Consequently, no ownership for RFSP was generated out of their mission

participation suggesting that the participation might have been on individual rather

than institutional capacity, especially in the case of CBRC. The design document

repeatedly mentions CBRC as the executing agency but in practice it never

materialized. The project struggled in terms of achieving the objective of this

component, of day to day supervision and could not provide feedback on lessons

learned and experience gained to CBRC to bring about systemic changes in the

sector. Discussions with IFAD staff and consultants provide two probable reasons:

a) consultations/discussions of the design team with CBRC at the time of design

missions did not lead to formal approval of the project within CBRC’s internal

official system as the host and main implementer; and b) CBRC had been

managing the networks of RCCs but later, as a result of institutional reform by the

government, turned from managing RCCs into playing a regulator role. CBRC

therefore kept away from directly managing a reform project that included

activities of two RCCs and a line of credit. This decision has practically de-linked

the project from CBRC although it officially remained as the executing agency.

Therefore, RFSP could not contribute to the sector reform process without any

formal mechanism for interacting with CBRC in the context of RFSP.

3. There was no adequate discussion and understanding with the two RCCUs about

their role in policy reform. These networks operate under policies of CBRC and are

not in a position to influence CBRC policy reform agenda.

4. The MTR recommended the formation of a policy consultative mechanism under the

project framework, comprising one representative from central CBRC, one senior

manager, two senior professionals and one support staff from each program

PRCCU. There was no follow up on this proposal as well. However, eight sessions of

policy consultation between the two project RCC networks were organized, mostly

to regularly share experiences in rural lending and programme implementation

management.

5. New product development. The two implementing agencies made significant

progress of their own in product development after MTR guidance from 2007

onwards. The PCR reports that the two provinces developed 29 new loan products,

of which 26 were applied and rolled out through the provincial networks.

Meanwhile, other products such as savings products and intermediary services

were designed and tested in the rural market and 45 other new products were

reportedly applied in the market, of which four are related to savings.

6. In the course of the PPA mission, the CRC Bank reported developing 35 new loan

products, of which 17 are related to agriculture. In particular, four products were in

high demand: a) loan for returnee migrant workers to start business; b)

management of forestry products; c) joint loan; and d) production loan for women.

All these products were not only introduced in the two project counties (Youyang

and Younyang) but also in all other counties of the province through the rural

branches. Similarly, the PRCCU in Shaanxi also reported developing and rolling out

loan products in the four project counties covered in this province. Important to

note is that the two organizations could develop and roll out financial products of

their own. CRC Bank reported that its ‘management of forestry’ product has been

promoted by CBRC in other provinces and it enjoys strong support from the

regulator.

7. Loan pricing. This activity area was considered particularly relevant at appraisal.

The Appraisal Report states that interest rates charged by RCCs for their loans are

below market clearing levels, and their main effect is the provision of relatively

cheap resources to the State enterprise sector, the banking sector’s main client.

There was no progress made on this, as the interest rates levels at RFSP start up

and loan closing demonstrated. The Shaanxi RCCU and Chongqing RCB adopted a

more flexible pricing policy and decentralized pricing decision-making at county

7

sub-branch level since 2007. Some training sessions were reported as programme

activities to support the introduction of such decentralized pricing; there was no

substantial fluctuation in lending interest rates in the rural areas afterwards as

CBRC has provided a band for variation of interest rate for rural commercial banks,

rural cooperative banks, and RCCs. Such stable rural lending rate typically ranged

from 7 to 11 per cent for on-farm financing, and up to 15 per cent for trade and

off-farm income-generating activities. Both agencies followed the regulator’s

guidelines in this respect. See table 1 for interest rates determined by CBRC over

the years.

Table 1 RCB and RCC savings and lending rates as determined by the central bank

Year Central bank base rate,

savings Central bank base rate,

lending Rcb & rcc av. Applied

lending rates

2007 3.87% 7.29% 14.58%

2008 2.52% 5.58% 11.16%

2009 2.25% 5.31% 12.21%

2010 2.50% 5.56% 12.79%

2011 2.75% 5.81% 13.36%

Source: Project Completion Report (2011)

8. Overall assessment of the component. The main objective of the component to

influence systemic policy change was not achieved due to inappropriate

institutional arrangements; yet, the two implementing agencies developed and

rolled out financial products to suit the demand of the small borrowers. No

progress was made on making interest rates for RCC deposits and loans more

market oriented. The reform agenda has been driven nationally and centrally by

the CBRC without any link to the project.

9. Component 2 - Institutional development. Of the three designated activities –

infrastructure and equipment, staff training, and support reorganization of legal

structure of RCC- under this component the first two were successfully completed.

A total of 78 service outlets in two provinces were rehabilitated by renovating office

building and upgrading the computer and surveillance system. The PPA mission

visited a few RCC outlets in Youyang county (Kiampo market). For distant clients

CRC Bank has outstations (a cash dispensing machine and a person) located in

shops. All these outlets and branches enjoy computerized accounting and

management information systems that help provide prompt services to the clients

and improved internal control and management of the branches. The local bank

officials reported that, due to physical upgrading, the number of clients is

increasing.

10. Staff development training. A series of topics for training were identified, mainly

focusing on accounting and financial management, credit management, and

microfinance. Over the project period, 5,860 staff members (3,260 in Chongqing,

of which 27 per cent women, and 2,600 in Shaanxi PRCCU, with 15 per cent

women) have received training. It is important to note that both organizations

designed and developed training materials internally and resorted to RFSP financial

assistance only when necessary. The PPA mission had an opportunity to meet

several officials who received such training. All of them were positive about the

contents of the courses, and utilized the knowledge acquired in day-to-day

business, for example, in selecting clients, disbursement and collection of loans,

and maintenance of accounts.

11. Reorganizing legal structure of RCCs. The initiative for legal restructuring was

undertaken by the RCCs themselves under the reform agenda of CBRC and

therefore this activity was not implemented by the project. The MTR re-allocated

8

the resources initially allotted for legal restructuring. In Chongqing, former county

RCCUs were converted to sub-branches of the Chongqing Rural Commercial Bank.

Interestingly, the bank has been so successful that it had enlisted in the Hong

Kong stock market in 2011. The PRCCU in Shaanxi now has four rural commercial

banks, 12 cooperative banks and 98 RCCs as members, all of which have separate

legal identity. The success of such transformation shows the effectiveness of the

reform engineered by CBRC.

12. Overall assessment of the component. Only two of the three activities have

been completed successfully; one activity was beyond the rim of the project and

was readjusted during MTR.

13. Component 3 - Operational development. This component dealt with making

accounting principles and practices compatible with internationally-accepted

accounting standards. The significant exception is classification of impaired assets

and loan loss provisioning. During the project implementation period, there were

even developments away from a strict time based loan loss provisioning. Risk

categorization for loan portfolio at risk and attendant loan loss provisioning are less

in line with international standards in the RCCs than they were a few years ago.

14. At RCC level, loan polices and loan management needed to be revised to match

demand for new loan products. Notably in the area of staff incentive systems, no

progress seems to have been made. The impact of revised loan policies can be

seen in the development of new loan products. The accounting system and

presentation of financial reports conforms with national standards.

15. Operating self-sufficiency. The impact of the overall reorganization of RCCs can

be seen in improved operations. Both provincial networks reported improved

operating self-sufficiency,5 for Chongqing from 120 per cent in 2005 to 126 per

cent in 2009, and for Shaanxi from 115 per cent to 152 per cent during the same

period. The improvement in operating efficiency derives from the improved

productivity of staff members. Staff productivity has improved in the programme

counties since the start of the RFSP. In 2005, an average 160 to 250 active

borrowers per credit officer were reported, while in 2009 programme lending

recorded 631 active borrowers per credit officer in Shaanxi and 807 in Chongqing.

The main contributor behind improved operating efficiency is the drive to make

profit by reducing cost. This high efficiency was also possible due to the fact that

clients repay mostly in one instalment, that is, at the end of loan period and a

credit officer need not to visit clients too frequently.

16. Overall assessment of the Component. One activity has been completed

successfully with overall adequate impact on financial and operational performance

of the two implementing agencies.

17. Component 4 – Financing (Line of Credit). Under Component 4, the project

provided a line of credit to the two implementing agencies to expand loans to rural

low income clients and to try out microcredit group lending techniques. The credit

reimbursed on-lending funds for pro-poor lending through the two institutions. The

microcredit lending under the project was executed mostly through the RCC

operations of individual microcredit lending, with more conscious targeting of the

poor and the women as differentiated from the RCC conventional microcredit

portfolio. In 2007, the MTR proposed a village credit model to be experimented in

the programme counties. The PPA mission found no evidence of the application of

such village credit model and both rationale and use for the IFAD targeted clients is

not discernible.

18. Clients and client selection. The two institutions follow systematic procedures to

identify potential clients and assess risk. While discussing with sample clients, it

5 A percentage which indicates whether or not enough revenue has been earned to cover the total costs – operational

expenses, loan loss provisions and financial costs.

9

was found that they either learned about the financial services offered by the CRC

Bank and PRCCU from their neighbours, from visiting credit officers or from market

campaign organized by them. This indicates systematic efforts to reach rural and

women clients. The credit analysis includes indicators such land ownership, total

asset, purpose of the loan, reputation of the applicants and the loan size is

determined accordingly. For poor applicants loans start with smaller amounts but

gradually increase. The PCR reports that 36,224 clients were given loans during the

project period; of these, 47 per cent was classified as poor and low income. Twenty

eight per cent of total beneficiaries were women. The sample of clients interviewed

by the PPA mission seem to be satisfied with the services provided; loans were

disbursed by RCCs within a week of application, and they plan to access further

loans in future.

19. Loan disbursement and recovery. Most of the loans are only for a 12 month

duration and probably not well attuned to client’s requirements for term financing.

Only loans for house construction could be up to three years. The absence of term

lending facilities for small scale investment finance is all the more surprising when

the unusually long terms of RCC term deposits are considered. Loans are recovered

with interest and principal amount at maturity; such arrangement is the preferred

one by the clients, although can increase risk if clients miss repayment dates. Term

lending for small scale investments and repayments in instalments should be

preferred depending on the purpose of the loan.

20. Loan range. Loans up to CNY 70,000 (US$ 11,000) have been disbursed for house

construction. Most of the loans reportedly range between CNY 10,000 to 30,000.

CRC Bank has the following policies regarding loan range, as illustrated in table 2.

Table 2 CRC Bank: Loan products, terms and conditions

Name of the loan products

Duration of the loan (in years) Range of loan amount (CNY)

1 Loan for migrant workers (business start-up)

1 to 3 20,000-200,0000

2 Start-up loan for women 1 to 3 10,000-50,000

3 Forestry ownership management loan 1 to 5 20,000-200,0000

4 Joint loan 1 to 3 10,000-50,000

Source: CRCBank

21. Interest rate. Interest rates are within the range stipulated by CBRC which, in the

case of CRCBank and PRCCU vary between 7 to 15 per cent. As noted earlier, the

rates are determined by the regulator, and not determined by cost or market

considerations.

22. Use of loans. In line with economic conditions in the six project counties, most of

the loans - except those provided for house construction - are directly or indirectly

linked with agricultural production. Table 3 below provides aggregate CRCBank

loan information for 2011.

10

Table 3 Microfinance portfolio of CRCBank in 2011

Loan category Households Amount (million CNY) Percentage (%)

Crop growing 48,937 880.73 41.9%

Livestock raising 21,793 481.14 22.9%

Commercial and trade 18,292 365.84 17.4%

Miscellaneous 16,652 373.75 17.78%

Total 105,674 2101.46 100%

Source: CRCBank

23. It is important is to note that the two financial institutions are continuing their

microfinance product related activities beyond the project period. Table 4 below

provides information on the microcredit portfolio of CRC Bank in two project

counties for the 2010-2012 period.

Table 4 CRC Bank RFSP-Microcredit information beyond project period in two project countries

Year and county

Loan disbursement during the year

[Million CNY] Number of borrowers

Man Women Total Man Women Total

2010 Youyang 16.55 1.81 18.36 1,341 185 1,526

Yunyang 15.96 1.91 17.87 1,004 205 1,209

Total 32.51 3.72 36.23 2,345 390 2,735

2011 Youyang 17.42 2.59 20.01 1,024 249 1,273

Yunyang 16.41 2.75 19.16 902 237 1,139

Total 32.51 3.72 36.23 1,926 486 2,412

2012

(Jan.-Apr.)

Youyang 12.36 1.57 13.93 689 153 842

Yunyang 12.76 1.63 14.39 612 178 790

Total 25.12 3.2 28.32 1,301 331 1,632

Source: CRCBank

24. After more than four years of implementation, PMOs in Chongqing and Shaanxi

accounted 36,224 direct beneficiary households from their programme microcredit

lending, of which 10,246, or 28 per cent, were women. Indirect beneficiary

households were estimated to be approximately 830,584, which is equivalent to

the number of savings households after deducting the overlapping rate of 15 per

cent. However, the PPA mission found no supporting evidence to the claim that

each and every depositing household in the counties covered in the two provinces

is either a direct or indirect beneficiary of RCC project loans. In addition, the PPA

mission questions the comparatively low number of 36,224 direct beneficiary

households in relation to the number of indirect beneficiaries as high as 830,584.

25. Overall assessment of the component. The component has been successfully

implemented, namely expanding access to credit for the poor, women and farmers,

both by using project credit lines as well as the own resources of the two RCC

networks, which even surpassed the project credit funds. However, the high

amount of financing coming forward from the partner institutions puts a question

mark behind the very rationale of operating a credit line, a point further

accentuated in the most recent IFAD Rural Finance Policy of 2009.

11

26. Component 5 Project Management. The PMOs in the two provinces completed the

Results and Impact Management System and benchmark surveys twice during the

programme period, respectively in 2007 and in 2010. A number of management-

related training sessions were reportedly organized and conducted for managers

and staffs. Female participation in those training sessions was estimated at 15 per

cent of the total.

27. Overall assessment of the component. PMOs managed well all activities that fell

within their capacity, especially after implementing the recommendations contained

in the MTR. They were not in a position to implement most of the activities under

Component 1 due to the reasons previously described.

III. Review of findings

A. Project performance

Relevance

28. At the time of design in 2002/3, the RFSP was relevant in terms of identifying the

general direction of reform agenda for networks of RCCs as well as strengthening

capacity of two partner RCCUs while they were undergoing restructuring. The

concept of designing a project to assist the Ministry of Finance or the Chinese

Banking Regulatory Commission (CBRC) remains relevant to date to implement

future reforms. At the same time, the project was designed and implemented at a

time when the Government was carrying out a major reform of the rural finance

sector. Some of the stated objectives of the RFSP equaled the objectives of the

reform program: increasing access to credit, improving the profitability of rural

credit cooperatives (RCCs), and promoting institutional reform of the RCCs. The

RFSP had a policy component which was designed to assist in ongoing and planned

policy reforms. However, the institution responsible for policy reforms (the China

Banking Regulatory Commission, CBRC) did not agree to be the implementing

agency for the project, and thus the reform process proceeded “in parallel” with

the project.

29. Up to the 2007 MTR, the progress was slow reportedly due to procurement related

issues, and delays in initial mobilization. The MTR recommended reallocation of

resources to speed up IFAD loan disbursements; notably, despite changes in the

priority areas of investments, the RFSP remained relevant to its overall objectives.

30. Under Components 2 and 3, the project assisted to upgrade management policies,

institutional and operational capacities, and sustainable lending practices to target

segments. The project has been supportive of organizational conversion in

Chongqing during 2007/08 from Rural Credit Cooperative Union to Rural

Commercial Bank (CRCBank). The project’s assistance helped the newly formed

Rural Commercial Bank to keep some focus on providing financial services to the

poor, women and farming while the bulk of its loans had been disbursed to larger

businesses; this was possible due to the project’s priorities in management,

operations and rural targeting. In Shaanxi, according to the provincial and county

level officials, the project helped upgrading systems, developing financial products,

and keeping the focus on the poor and farmers. The products developed for the

project counties have been replicated in other counties.

31. The project proved to be directly relevant to the poor, women and small borrowers.

Both partner financial institutions extend financial services (savings and credit) to

the poor, women and farmers. The RFSP paid particular attention to product

development, by trying to promote microcredit products with poverty and gender

sensitivity.

32. The selection of the RCC network as implementing financial institutions has been

appropriate and will remain so for the future as well. The RCC network is the only

practical way of reaching large rural areas with financial services. They are rooted

in the rural area, and had a long history of supporting agricultural and rural

12

development. Any future rural finance project, if undertaken in poorer provinces in

the central and western provinces has to be partnered with respective RCCs.

Whether net asset increases for poor productive people can be achieved without

lines of credit as in the most recent additions to the IFAD investment portfolio

remains to be seen.

33. Rating of relevance. The PPA finds the project concept relevant both at the time of

design as well as at present, although the design process lacked full

comprehension of the reform process, and proper institutional arrangements for

influencing systemic changes which led to some extent to partial underperformance

of the project. Relevance is therefore rated as moderately satisfactory (4).

Effectiveness

34. The effectiveness of a project is defined as ‘the extent to which the development

intervention’s objectives were achieved taking into account their relative

importance’. Since there was no logframe at the time of design, the project later on

produced a logframe at MTR (see annex 4) that does neither provide a statement

by component, nor any physical target for measuring progress. The PPA mission

has first used the overall objective of “enabling poor rural people to improve their

livelihoods by increasing their access to financial services”, and the overall project

purpose “to turn the credit cooperatives into sustainable microfinance institutions

for the benefit of poor rural people and women in particular” as a reference point to

assess the effectiveness of the project.

35. Components. As per resource allocation, the line of credit was the most important

component to actually increase access to finance by providing matching capital to

the two financial institutions. Policy reforms, institutional and operational

development activities – with the exception of infrastructure (civil works and

computer hardware) did not require large sums of money but contributed to overall

capacity building of the institutions and the sustainability of the financial services.

Resource allocation as a percentage of total costs was as follows: (1) Policy

Development (4.65%); (2) Institutional Development (12.85%); (3) Operational

Development (2.62%); (4) Financing (line of credit for microfinance) (76.62%);

and (5) Management (3.26%).

36. Effectiveness in relation to the overall objective. The project contributed to

improving livelihoods of the poor people by increasing access to financing services.

The two RIMS studies clearly show that borrower families have higher income and

increased assets. It should be noted that increased income cannot be fully

attributed to access to credit (attribution problem) as exogenous factors may have

influenced higher levels of incomes as well as increased assets.

37. Effectiveness in relations to overall purpose. The overall project purpose ‘to

turn the credit cooperatives into sustainable microfinance institutions for the

benefit of poor rural people and women in particular’ was rather ambitious.

Reforms of the RCC networks were carried out which led to greater clarity in the

corporate structure and ownership, improved profitability, a significant decline in

non-performing assets, and improved capitalization. Participating financial

institutions trained staff and developed new financial products suitable for the poor,

women, and farmers. However, because the project was de-linked from CBRC, and

because Participating financial institutions trained staff and developed new financial

products on their own, it is not clear to what extent these outcomes can be

attributed to the project. However, the project can reasonably take credit for

organizing a line of credit for microfinance related activities, assistance in

infrastructure development and human resources development. Both institutions

are financially sustainable and their overall profitability will allow and encourage

them to continue with microfinance operations as a part of their portfolio.

13

Effectiveness by project components

Component 1 - Policy development. This component partially met its

objectives. The only effective activities carried out were inter-provincial

exchanges between the two programme networks, and some training and

internal consultations in lending policy development. The new product

development has been carried out by the two financial institutions.

Component 2 - Institutional development. This component was effective as

the project contributed to infrastructural development such as renovation of

outlets, computerization of internal systems and human resources development

through a large number of training sessions. As a result, it contributed to

attracting clients, enhanced competitiveness of the institutions and enhanced

internal capacity, and made the credit officers of RCCs effective in targeting

poorer clients.

Component 3 - Operational development. This component was partially

effective as it contributed to revising internal policies, yet the review of the staff

incentive system could not be achieved.

Component 4 - Line of credit. The most effective component of the project

was finance for microcredit. Both the IFAD loan and the matching funds from the

two financial institutions were used. A number of reasons made this component

effective, among these the fact that it was not subject to any form of

procurement, and that line of credit was the main business of the two financial

institutions already prior to project start up. Their capacity to develop and

promote successful new loan products was another important factor to be

considered.

38. Regarding the review of the financial achievements, table 5 indicates that the line

of credit component achieved 100 per cent of the implementation target, as it

adopted the RCC’s own methodologies. However, the requirements for accessing

IFAD finance were perceived as burdensome when it came to procurement for

consultancy services and outsourced training.

Table 5 Financial achievements, overall programme (in US$ equivalent)

Overall programme targets Overall programme achievements

Overall percentage of programme achieved

IFAD RCC Total IFAD RCC Total IFAD RCC Total

Components Policy dev. 2,130,325 36,069 2,166,394 1,344,959 70,491 1,415,449 63% 195% 65%

Institutional dev. 4,249,329 358,324 4,607,653 3,626,424 281,590 3,908,014 85% 79% 85%

Operational dev. 853,624 18,416 872,040 757,960 39,893 797,852 89% 217% 91%

Financing 9,014,740 13,115,311 22,130,051 9,394,080 13,915,261 23,309,341 104% 106% 105%

Management 1,388,962 89,764 1,478,727 764,980 226,366 991,346 55% 252% 67%

Total 17,636,981 13,617,884 31,254,865 15,888,403 14,533,600 30,422,003 90% 107% 97%

Categories

Civil work 985,619 51,875 1,037,494 931,103 49,005 980,108 94% 94% 94%

Equipment & materials

1,712,203 199,513 1,912,316 1,327,827 147,536 1,475,363 78% 74% 77%

vehicles 209,511 43,214 252,725 196,793 30,925 227,717 94% 72% 90%

Studies 660,317 14,530 674,847 322,204 16,388 338,592 49% 113% 50%

Training & consultancies

4,281,885 134,906 4,416,791 3,107,609 160,091 3,267,700 73% 119% 74%

incremental credit

9,014,740 13,115,311 22,130,051 9,394,080 13,915,261 23,309,341 104% 106% 105%

Staff salaries 238,249 28,974 267,223 318,746 199,128 517,874 134% 687% 194%

Operating costs 534,456 29,562 564,018 290,042 15,265 305,307 54% 52% 54%

Total 17,636,980 13,617,885 31,255,464 15,888,403 14,533,600 30,422,003 90% 107% 97%

Source: Project Completion Report (2011)

14

39. Overall effectiveness. The project could be considered effective judged by impact

on livelihoods and expanded access to credit. However, in terms of policy reform

and its contribution to transforming RCCs into sustainable institutions, the extent of

the project contribution is not fully clear. At the same time, given the successful

achievements with regards to component 4, the criterion of effectiveness is

considered to be satisfactory (5).

Efficiency 40. Efficiency is a measure of economical use of funds, expertise and time of the

project to achieve objectives. As discussed earlier the efficiency was affected due

to delay in implementation. As noted in the MTR, not much progress was made

before 2007. Besides, significant amounts of time were lost between loan approval

and effectiveness - 16.8 months, which is well above the regional average of 9.2

months and the IFAD average of 12.4 months. The implementation period was

extended by six months, bringing the current completion date at 31 March 2010

and the current closing date at 30 September 2010.

41. Fund disbursement. Due to the appreciation of the Chinese currency and the

Special Drawing Rights against the US Dollar during implementation, the project

made more financial progress than physical progress. Table 6 below presents

revised estimates and actual financial performance. The revised project estimates

at the time of the MTR amounted to US$ 31.25 million while actual achievement

was US$30.422 million, that is, 97 per cent fund utilization. The IFAD loan figure

was revised to US$17.64 million against the original figure of US$14.7 million at

the time of appraisal. Against the revised estimates, actual IFAD loan disbursement

was US$15.89 million, which is 90 per cent of adjusted financial value. Similarly,

RCCs have contributed US$14.533 million against a revised estimate of US$13.617

million (107 per cent), that is, the RCCs financial contribution has even exceeded

the revised estimate. RCCs contribution even look much higher if compared to

appraisal estimates. At appraisal, the IFAD loan was estimated to be US$14.7

million, that is, the actual disbursement was 8 per cent higher whereas RCC’s

contribution was estimated at US$6.6 million, and the actual disbursement was

120 per cent higher. Therefore, the IFAD contribution to the total financing was

weighted for 52 per cent instead of 70 per cent at appraisal, and the final

counterpart finance averaged at 48 per cent as compared to the initial figure of

30 per cent. During implementation, both provincial RCCUs/RCBs provided

continued financial support through matching funds to support programme

implementation.

Table 6 Programme costs, financing and matching in US$

IFAD RCC Total

Adjusted cost and achievement

Adjusted programme target 17 636 980 13 617 885 31 255 464

Actual realizations 15 888 403 14 533 600 30 422 003

Percentage of actual realizations 90% 107% 97%

Consolidated programme matching percentage 52% 48% 100%

Province-wise matching in Shaanxi 43% 57% 100%

Province-wise matching in Chongqing 66% 34% 100%

At appraisal

Appraisal estimation 14 700 000 6 600 000 21 300 000

Changes in percentage* 8% 120% 43%

Source: Project Completion Report (2011)

* Total change is caused by the valuation of Chinese currency; increased counterpart percentage is caused by both currency valuation and RCC additional funding.

15

42. The MTR reallocated funds due to procurement difficulties expressed by both

provinces in executing a number of activities related to consultancies, studies and

policy development. The line of credit was increased but allocations for training and

consultancies at provincial levels reduced. The actual project cost by component is

given below in table 7. This reallocation has contributed to a more efficient use of

funds as credit disbursement component has exceeded appraisal targets and

directly contributed to achieving the project objective and purpose. However, the

contribution of the credit line should also be seen in comparison with total assets of

the two financial institutions. It is estimated, as of 2009, the project credit fund

amounted to only 3 per cent of total assets. This small size prohibits rigorous

efficiency analysis. The project acted as catalyst, or influencing agent, to develop

new financial products for the poor and encourage the financial institutions to keep

in their portfolio.

Table 7 Actual fund use by component (in US$ equivalent)

All programme achievements

Components IFAD Gov./RCC Total % of Total

Policy dev. 1 344 959 70 491 1 415 449 4.65

Institutional dev. 3 626 424 281 590 3 908 014 12.85

Operational dev. 757 960 39 893 797 852 2.62

Financing 9 394 080 13 915 261 23 309 341 76.62

Management 764 980 226 366 991 346 3.26

Actual total 15 888 403 14 533 600 30 422 003 100.00

Source: Project Completion Report (2011)

43. The management cost of the project is very low (3.26 per cent of total cost), which

allowed more than 96 per cent of resources to be allocated for program activities.

But the probable reason for such low management cost is that the microcredit

management cost has been absorbed by the financial institutions as their regular

operating costs. In any case, partnership with appropriate institutions has made

the project more efficient.

44. The PCRV compares RFSP with two other IFAD-funded projects in China which were

on-going in the same period and having a rural finance support component. As

shown by data in table 8 below, RFSP did reach a much smaller number of direct

beneficiaries. The main reason probably lies in loan size (amount disbursed per

loan per person), that is, RFSP disbursed larger loans per person than the other

two projects and the beneficiaries targeted by the other two projects are probably

much poorer that the ones targeted by RFSP.

Table 8 Comparison between IFAD projects

Project RF Costs

CNY million Total project cost

CNY million Proportion

investment in RF Borrowers

reached by RF

Qinling Mountain Area Poverty- Alleviation Project 129 695.64 19% 459 300

West Guanxi Poverty-Alleviation Project 101 730 14% 239 000

Rural Finance Support Programme 126 126 100% 36 224

Source: IOE interim evaluation Qinling mountain area poverty-alleviation project

16

45. Overall efficiency. Taking into account the financial progress against the physical

progress and the slow start-up, this PPA rates efficiency as moderately satisfactory

(4).

B. Rural poverty impact

46. Data sources and limitations. The following analysis is based on data presented

in the project completion report and complemented by additional information from

the PPA field visits. The project conducted two RIMS surveys in mid-2007 and early

2010. However, no baseline data and comparison with control groups are available

to attribute that the impacts are directly correlated with the project interventions,

especially enhanced access to financial services. A random selection of 900

households in the project villages from the two provinces (180 households in

Chongqing and 780 households in Shaanxi) was reviewed. The results represent

overall changes in the households of the selected villages but do not attribute the

changes to the project interventions. A more appropriate methodology would have

been to compare changes in beneficiary households with control groups to attribute

the changes to project interventions.

Household income and assets

47. No specific data were collected regarding household income and income generating

activities. The PCR noted an overall reduction in poverty in project villages. For

example, in Chongqing, poor or low-income households decreased from 16 per

cent in 2007 to less than 5 per cent in 2009; in Shaanxi, surveys recorded 11 per

cent of poor and low-income people in 2007, which was reduced to about 8 per

cent in 2009. However, it also noted that the improvements correspond with the

overall poverty reduction in the country. That makes it difficult to segregate the

impacts of the project on income and poverty reduction. The PPA mission

interviewed a small number of direct beneficiaries. The interviews indicate that the

microcredit recipients could expand business (number of livestock) and buy inputs

for farming (for example, tobacco and spice production) that led to increase in

production. At the same time beneficiaries get good farm gate returns due to

better communication and access to markets. Besides, families are increasing

income from other sources as well as, for example, part-time or seasonal wage

labour outside the villages. Therefore, a combination of factors, where credit is one

element, led to increased income.

48. Other indirect or proxy indicators mentioned in the PCR may be considered to have

contributed to increased income. Through its support to the RCC/RCB networks,

RFSP contributed to an increased access to financial services. According to the

RIMS data, access to RCC loans by poor rural households increased from 36 per

cent in 2007 to 42 per cent in 2009 for Chongqing and from 37 per cent to nearly

60 per cent in Shaanxi in the same period. Additional access to finance usually

leads to expansion of business and good market prices lead to additional income.

This link was corroborated by interviews during the PPA mission.

49. Access to savings services also indicates increase in financial assets. The

percentage of households having a savings accounts increased dramatically from

57 per cent to 90 per cent in Chongqing, and from 49 per cent to 70 per cent in

Shaanxi. This could indicate more net savings. In reality however, the opening of a

savings account is a pre-condition for receipt of government subsidies for small

farmers, and this may be the reason for more savings accounts opened over the

RFSP implementation period. Without knowledge on minimum and average

balances kept in these accounts over the past years, it is more likely that subsidy

receipt rather than genuine savings accumulation was the prime driver for account

openings in RCCs.

50. As also noted during the PPA mission, the two RIMS surveys reported strong and

increasing involvement of farmers in income generating activities related to

marketing, transport and trading. This change is a natural progression in a rapidly

17

developing rural economy where farmers are increasingly producing diversified as

well as high-value products demanded by the markets and venturing into other

non-farm income generating activities. In Chongqing, 83 per cent of farmers

reported such involvement and practice in 2007 and 98 per cent in 2009; similar

trends were observed for Shaanxi, with 84 per cent and 97 per cent of farmers in

2007 in 2009, respectively reporting these positive developments. Yet, all these

changes cannot be attributed only to credit as mentioned above but to a result of

many factors.

51. The PCR tried to indicate increase in income by measuring changes in household

assets. The RIMS surveys show an increase in household assets such as having a

TV and a motorcycle. Positive changes in farm tools can be observed moving from

mainly hand tools in 2007 to animal-driven or machine powered tools in Shaanxi,

and animal-driven tools in Chongqing. The emphasis on animal driven tools in

Chongqing was explained by the small plots in the predominantly mountainous

area. With regard to livestock, the 2009 survey observed that more households

were able to raise larger animals (cattle and pig ownership increased from 22 per

cent to 61 per cent in Chongqing). The PPA mission found that beneficiaries

borrowed loans to repair or build houses, which also reflects improvement in assets

and standard of living. However, there is no evidence that all such changes can be

attributed to credit only.

52. Overall, the changes reported here are typical of microcredit programs elsewhere.

The PPA rates the project impact for household income and net assets as

satisfactory (5).

Human and social capital and empowerment

53. Direct beneficiaries. The project did not undertake any training at client level. All

such training was directed to staff members of the two financial institutions; yet,

clients did indirectly benefit from the credit program that improves social capital

and leads to empowerment in the long run. The PPA mission learned that many of

the clients were first time borrowers, both men and women, - the mission met a

few - who learned about the financial services provided by the RCCU/CRCB and for

the first time interacted with formal financial institutions. Several women opened

savings accounts and also used loans, although men are still ahead in using

financial services. This gradual process leads to long-term empowerment.

54. RFSP has substantially invested in RCC staff training. In Chongqing, 3,260 staff

were reported being trained of which 27 per cent were female participants; in

Shaanxi, 2,600 staff received training of which 15 per cent were female. Training

areas ranged, among others, from integrated operational networking to microcredit

lending, risk management, new loan product piloting, small and medium enterprise

lending, loan categorization, analysis of operational efficiency and financial

sufficiency. Those actions were part of the RCCU/RCB own capacity building but

were strengthened through programme support and they have helped improving

the human assets of the RCC/RCB networks in the two programme provinces.

55. Conceptual introduction and exercise of market segmentation, lending with poverty

and gender sensitivity or with differentiated services have contributed to the

empowerment of the vulnerable groups such as poor, women and returning

migrant workers. According to the Completion Report, their socio-economic status

tends to improve. For example, the surveys indicate an increasing trend of women

signing RCC loan contracts and, especially, becoming savings account holders. For

the latter, an increase of 40 per cent was noted for female savings account holders

in Chongqing.

56. Based on the above analysis, programme impact on human and social capital and

empowerment is rated as moderately satisfactory (4).

18

Agricultural productivity and food security

57. The RIMS survey attempted to capture changes in agricultural productivity

irrespective of sources of interventions. The Completion Report highlights an

increased percentage of households facing increased yields in their agriculture

production. Therefore, the question remains on what are the reasons for such

positive change. The Completion Report explains the improvements as results from

a combined support of government programmes and a more diversified market for

local farmers. Yet, the project may take credit for having enhanced clients’ ability

to grasp opportunities and venture out in agricultural as well as other income

generating activities.

58. The completion report highlights findings of the 2007 and 2009 surveys in terms of

food security, child malnutrition and safe sanitation. According to the survey results

of 2007, approximately 2 per cent of households in Chongqing had faced food

shortages. There was no report of households experiencing food shortages in 2009.

Overall, children’s nutritional status improved between 2007 and 2009. For

example, stunting decreased significantly in Chongqing, from 52 per cent to 35 per

cent. In Shaanxi however, it did decrease only slightly. The presence of chronic

malnutrition in the two provinces seemed to be caused by local food structure,

where nutritive balance may need to be re-assessed. Access to safe water

increased significantly in both provinces, to almost 100 per cent. However, safe

sanitation did decline in Shaanxi from 53 per cent in 2007 to 50 per cent in 2009

because of the long-lasting drought in the northern project counties where water

use was reduced to cover drinking and irrigation needs first.

59. Notably. Food security has improved and malnutrition has been reduced. This

positive result can be rated as satisfactory (5).

Natural resources and the environment

60. Similar to agricultural productivity it is difficult to measure any direct impact of the

project on natural resources and environment. Given the lack of data and evidence,

this criterion has not been rated.

Institutions and policies

61. There were a number of accomplishments in the area of policy and institutional

reform. Reforms of the RCC networks were carried out which led to greater clarity

in the corporate structure and ownership, improved profitability, a significant

decline in non-performing assets, and improved capitalization. However, because

the project was de-linked from CBRC, it is unclear to what extent the project was

responsible for these outcomes. At the micro-level, the project directly assisted the

two financial institutions to improve their policies, practices and systems that

directly benefited the poor, women and small producers. Although small in

resources compared to total assets of the financial institutions, the project

influenced development of financial products suitable for the beneficiaries. The two

financial institutions have shown a higher gender and poverty sensitivity in

selecting clients and giving priorities to households that could not offer any

collateral. All institutions appear to have strengthened their institutional capabilities

and services. Either for the whole provincial network or for the programme county

sub-branches, an improved capacity per outlet and service coverage were reported

which in turn led to an increase in the numbers of customers in the RCC’s savings

and credit business.

62. Based on the above assessment, the PPA rates project impact on institutions and

policies as moderately satisfactory (4).

Overall impact on rural poverty reduction

63. Overall impact. Despite some issues related to attribution, it is clear that the

project showed positive impacts. Based on the above analysis, the overall rural

poverty impact is rated satisfactory (5).

19

C. Other performance criteria

Sustainability 64. Definition. The sustainability of rural financial service program may simply be

defined as the continuation of services without depletion of the credit funds and

with generation of profits. From demand-side, it is important to see clients make

profits from businesses financed by credit from respective financial institution and

that there is continued demand for financial services. From the supply side, it is

important to ensure that the financial service provider (financial institution) is a)

institutionally and financially sustainable. Several indicators such as i) governance,

ii) policies, iii) capacity to offer demand-driven services, iv) strong internal system

and control and v) capacity of human resources, are used to assess institutional

sustainability. On the other hand, financial viability is measured by i) profitability,

ii) level of non-performing loan, iii) return on equity, and iii) return on assets. The

following sections present sustainability of the RFSP, or, rather, the sustainability

of the two implementing financial institutions, using the above definition.

65. Demand side (sustainability of clients). Proxy indicators are used to assess the

sustainability of clients. The RIMS surveys should have analysed profitability of

sample business/income-generating activities financed by the project. In the

absence of such direct measure, the following observations can be made about the

financial sustainability of clients:

RIMS surveys recorded that households in the project villages have

increased household’s assets as well as reduced poverty level, that is, lesser

number of households are now below poverty. Notwithstanding

contributions from exogenous factors, access to financial services increased

their income, which is a reflection of profits from farming and other income

generating activities.

Clients have repaid loans on time as noted from interviewing staff as well as

sample clients, which is also a proxy indicator for profitability as well as

improved family cash-flow. In addition, the non-performing loans are

declining, which indicates good repayment of loans.

Clients are borrowing larger loans in successive years, which reflect their

capacity to manage larger loans as well as interest from borrowing. In

addition, the number of new clients is increasing, which indicates demand

for financial services.

Interviews with sample clients and staff reveal that business opportunities

are expanding in rural areas, more and more profitable farming

opportunities are emerging that will create demand for financial services.

Within the project period, the financial institutions provided loans to more

than 32,000 clients. Both are continuing with their services. CRCBank

reports over 7,000 new or returning clients in the period 2010-2012 (April),

a clear indication of demand.

66. Overall, it can be concluded that clients are making profits from business/farming

and demand for financial services is increasing over the years, which is expected to

increase in future as well.

67. Supply side - Institutional sustainability of the financial institutions. The

following aspects indicate strong institutional sustainability of PRCCU/CRCBank:

Clear legal ownership and governance;

Both institutions have elaborated savings and credit policies and developed

many credit products that have been rolled out in the project as well as in

other counties. Both financial institutions have developed the internal

capacity to design and roll out new products as demonstrated during the

project period;

20

Both institutions have strong internal management and control systems to

oversee overall performance; and

Both institutions benefited from training courses conducted under the

project. More than 5,000 officials were trained. More importantly, training

courses were designed and conducted by internal persons, which

demonstrate strong in-house capacity.

68. Financial viability of the financial institutions. The following indicators show

that both financial institutions are financially viable to offer sustainable financial

services:

Profitability: Both financial institutions for the last several years consistently

posted profits and expanded businesses. Profit margins for CRCBank and

RCCU are given in table 9A and table 9B;

As part of restructuring and improved performance, both financial

institutions have successfully reduced non-performing loans. In 2011, and

using the new and less strict risk categories, the non-performing loans for

CRCBank amounted to 1.44 per cent only; and

In spite of initial under-capitalization in particular in Shaanxi Province, both

institutions are now assessed by this PPA mission to be adequately

capitalized.

Table 9A Selected performance indicators in Shaanxi RCCU

Provincial RCCU Average programme county

Years 2005 2009 2010 2011 2005 2009

Return on equity 16.00% 108.81% n/a n/a 22.60% 41.84%

Return on assets 0.45% 1.45% n/a n/a 1.48% 1.98%

Profit margin 11.17% 32.68% n/a n/a 26.31% 39.64%

Asset utilization 4.01% 4.43% n/a n/a 5.56% 5.09%

NPL 16.96% 12.94% n/a n/a 8.39% 5.72%

Financial self sufficiency 112.78% 152.05% n/a n/a 148.98% 157.67%

Source: Project Completion Report (2011) Table 9B Selected performance indicators in Chongqing RCB

Chongqing RCB Yunyang Yuyang

Years 2005 2009 2010 2011 2005 2009 2005 2009

Return on equity* 17.06% 14.56% 13.65% 15.17% 19.54% -- 14.05% --

Return on assets 0.55% 0.72% 1.26% 1.35% 0.6% 1.56% 0.63% 1.44%

Profit margin 11.15% 13.62% 38.75% 39.8% 12.22% 33.17% 10.72% 34.76%

Asset utilization 5.05% 6.25% n.a n.a 4.98% 4.7% 5.87% 4.13%

NPL 12.01% 3.63% 2.38% 1.44% 17% 3.26% 15.72% 9.77%

Financial self sufficiency

108.14% 148% 1631% 1490% 101.97% 147.44% 117.27% 140.86%

Source: Project Completion Report (2011)

* County level return on equity was no longer recorded in the RCB after its conversion to a single legal person bank for the provincial network.

69. Overall conclusions. Both Shaanxi RCCU and CRCBank are financially viable

institutions and the target population may expect to receive financial services way

21

beyond the project period. Based on the above assessment, sustainability is

considered to be satisfactory (5).

70. Exit strategy. The project had a built-in exit strategy as it was partnering with

two strong financial institutions. After completion of all non-finance type project

activities, the lines of credit are used as revolving funds by the two institutions.

The IFAD loan is expected to be repaid as per agreement. Meanwhile, both

institutions are continuing to provide services to the existing clients and taking in

new clients.

Innovation and scaling-up 71. The main innovation came from the development of loan products by both

CRCBank and Shaanxi provincial RCCU (later Cooperative Bank). CRCBank reported

the development of 35 loan products, of which 17 are related to agricultural

development. Similarly, Shaanxi PRCCU have developed new loan products. A

number of loan products have been reported to be highly popular among clients,

such as the loan for migrant workers and the farm product loan. These products

target agriculture and rural micro enterprise more than poor productive rural

households, but have had an impact in the two project areas regardless.

72. In Shaanxi, a product called “Women Starting Their Business” was developed. This

has led the county RCCUs to increase their lending to the segment of women

entrepreneurs. Shaanxi RCCU developed a special loan product for returning

migrant workers which helped returning migrants with a complementary financing

to support their resettlement in agriculture related income-generating activities, as

well as for the start-up of non-farm business ventures.

73. In Chongqing, programme RCCUs/RCB sub-branches after MTR recommendations

explored efficient ways of extending the credit services to grassroots level through

a mechanism of broker lending. By creating a network of farmer credit agents at

village level who acted as credit brokers between lenders and borrowers,

RCCUs/RCBs were able to lend to farmers in remote villages at a low operating

cost. Meanwhile, an incentive-driven credit screening by the brokers helped

lowering the associated risks and provided additional means of credit monitoring

and collection.

74. It should be recalled that the two financial institutions have in fact replicated and

introduced all these new loan products beyond the project counties. Both reported

that these loan products are available from all rural branches, sub-branches and

outlets. This is a good example of scaling-up. It has also been reported that CBRC

supported and promoted the loan product for natural resource management

(leasing of forest land for developing forestry products) beyond Chongqing

province. Overall, this criterion is considered satisfactory (5).

Gender equality and women’s empowerment

75. There was no explicit target for any gender specific activities. A number of

activities directly benefited women. For example, of the 3,260 trainees of

CRCBank, 27 per cent were women who received training in several areas.

Similarly, in PRCCU 15 per cent of trainees were women. A sample interviewed

during the PPA mission expressed their satisfaction about the contents of the

courses and mentioned that they are applying the gained knowledge in day to day

operations.

76. Of the 36,224 total clients reached by the financial institutions, 28 per cent were

women who in many cases are first time borrowers from any formal financial

institutions. They are currently interacting with the financial institutions, although

male members of the families still play a large role in securing loans. Women

clients were found actively participating in crop production and raising livestock

although marketing still remains a man’s domain. The financial institutions reported

22

large increase in savings accounts by women. Overall, this criterion is rated as

moderately satisfactory (4).

D. Performance of partners

Performance of IFAD

77. IFAD took a strong stance with regard to financial and institutional sustainability as

an important success factor in rural finance. The completion report recognises the

role of IFAD in providing continuous support and training opportunities for PMO

staff. Moreover, IFAD showed flexibility in adjusting its assistance and financial

allocation to cope with the changing environment given the nationwide reform

process of the rural finance system contributes to the positive assessment.

Notably, the RFSP was IFAD’s first sector programme and therefore could not rely

on previous experience. Hence, the project experienced a series of issues, most

notably: (i) the implied programme objective of policy consultation was perceived

as ambitious; (ii) the complexity of the RCC structure was underestimated; and

(iii) implementation of certain activities pre-MTR was idle due to the national

context. At MTR, the Fund addressed the lack of appropriate institutional

arrangements to implement a policy reform project by reallocating resources to

more practical components such as line of credit and guided the development of

innovative and pro-poor financial products. Direct supervision has led to more

timely and effective interventions to rectify problems arising during project

implementation, yet, at the same time, the supervision process is still over

dependent on the MTR as the point at which adjustments in projects are to be

made. Based on the above assessment, the performance of IFAD is considered

overall to be moderately satisfactory (4).

Performance of the Government

78. Ministry of Finance. Overall, management and coordination was handled by the

Ministry of Finance, that is, the IFAD partner ministry on behalf of the Government.

The PCR reports that the office at the Ministry of Finance responsible for IFAD

showed promptness in coordinating communications between IFAD and the two

programme provinces. Despite its frequent change of personnel in charge, most of

the applications and files were processed without delay, once they were sent by the

programme Department of Finance or IFAD. The government established a PMO in

each province, who were the actual coordinators for programme implementation.

Both PMOs were successful in mobilizing and coordinating the counterpart funding

for the programme, which exceeded the original 30 per cent in both provinces. Yet,

implementation progress was at times slow given the absence of full-time PMO

staff, a fact that also led to uneven monitoring and evaluation data and delayed

submission of progress reports. Provincial PMOs successfully assisted county PMOs

by providing implementation support at field level and promoting the RFSP at

various international and domestic events. The final PCR missed a discussion of

interest rate levels specifically for the types of loans that are relevant for the IFAD

target population and has no adequate and recent information on the quality of the

IFAD financed portfolio managed by the two RCC networks in Shaanxi and

Chongqing. Given that most of IFAD funds went for microcredit financing, the

absence of performance information of the IFAD credit funds in the PCR cannot be

justified.

79. Departments of Finance. The Departments of Finance in Shaanxi and Chongqing

managed the Special Account and Withdrawal Applications programme prepared by

PMOs for submission to IFAD. IFAD disbursements were timely transferred to the

PMOs in support of continued implementation. The Departments of Finance

respectively in Chongqing and Shaanxi were challenged by the multitude of donors’

projects they are responsible for, and therefore, the time allocation for the RFSP

was somehow limited. Especially in Chongqing, delays in processing withdrawal

applications before submission to IFAD occurred regularly and PMOs requests were

not always followed-up by the Department of Finance. In Shaanxi, the RFSP was

23

the second IFAD project in the province and the Department of Finance was more

familiar with IFAD proceeds. This had eased the processing of withdrawal

applications, communications and coordination with IFAD.

80. Overall, the performance of the Government is considered to be moderately

satisfactory (4).

E. Overall project achievement

81. The PPA mission rates the project’s overall achievement as satisfactory (5). The

project contributed to strengthening institutional, financial, and operational

capacity of the two implementing financial institutions – Chongqing Rural

Commercial Bank and Shaanxi RCCU network – that led to increased access to

finance by the poor, women and small producers. It also contributed to enhancing

livelihoods of rural households in the project counties. In particular, impact studies

showed increases in household assets and production, improvement in food

security, and strengthening of human and social capital and socio-economic status

of women, although not all impacts on households can be fully attributed to the

project alone. The project partners developed new financial products and rolled

them beyond project counties and encouraged them to continue serving the small

clients. The two financial institutions are institutionally and financially viable to

continue offering financial services to poor smallholders. However, the project

originally designed as a sector project to participate in systemic policy reforms led

by CBRC could not do so due to a lack of proper institutional arrangements, rapid

changing nature of the on-going reform process and inadequate grasp of the

reform issues and institutional complexity of the rural finance sector in China. The

project concept was relevant but design was ambitious and weak in institutional

arrangements.

IV. Conclusions and recommendations

Conclusions

82. The following lessons may be drawn from the implementation experience of RFSP

for future IFAD project in China and elsewhere:

(a) Design process and institutional arrangement. The design team did not do

proper in-depth consultations with various stakeholders, especially at the

national/central level relevant institutions mandated to lead the rural finance

sector. Some discussions were held and contacts were made but those

discussions did not lead to formal agreement with CBRC to be the lead national

implementing agency of the project. Without having the central/national agency

as the host and implementer of the project, it is hard to imagine how the

project could have contributed to systemic change in the sector. There was no

formal feedback system to pass on the knowledge generated by the project to

the policy makers. CBRC independently made all policy decisions to implement

the nation-wide RCCU networks including the two RFSP partner financial

institutions. Three critical lessons are: a) for a policy reform project to drive

systemic changes it has to be hosted and managed by the central body

responsible for taking policy reforms; and b) all consultations and dialogue

during design process must be held at appropriate level and lead to formal

agreement about scope of works as well as responsibilities of the parties

involved so that binding commitment to the project is ensured; and c) involving

a banking supervisory agency as an implementing agency for a pro-poor

financial sector reform programme is counterproductive since this design

introduces conflict-of-interest issues between the same agency as regulator and

supervisor of financial institutions on one hand, and as implementer of reform

on the other.

(b) Understanding implementing institution’s work process. It is also critical

that the design process and design documents reflect the core business, scope

24

and mandate, strengths and weaknesses, and internal work process of the

implementing agencies. The two financial institutions performed well in

disbursement and recovery of loans, developing new products and activities

related to reaching the clients, which were within their mandate and expertise.

On the other hand, it was difficult for them to handle the procurements of

several advisory services. At the same time, none of the two local financial

institutions was in a position to influence central policies. Their job was to

implement in letter and spirit the central regulations and policies by CBRC. It

was an undue expectation on the part of the project to ask the two provincial

RCCs to experiment with new ideas and influence national policy. It is critical

that the design team/process understands the partner institutions adequately

and that their concerns are addressed at the design stage even if that may

mean a longer design process.

(c) IFAD financed portfolio performance needs continued and priority

attention. The absence of performance data on the IFAD financed loan

portfolio in Shaanxi and Chongqing needs to be noted. Comparative information

in the PCR date back to a single static figure for end FY 2007 without the

possibility to assess portfolio performance over time and in comparison between

the two project locations. This discussion should also have included the criteria

used to determine impaired portfolios and have provided a critical discussion on

the adequacy of the same.

(d) Partnership with viable financial institutions. One more lesson to be

learned from the RFSP is that, to ensure sustainable financial service delivery to

the poor and small producers, IFAD must select strong and financially viable

financial institutions as partners. The positive achievements by the project were

largely due to selection of two strong and resourceful institutions with financial

services as their core business.

Recommendations

83. Future direction of rural finance reform. There is a need and scope for future

reform in the rural finance sector in China. A number of issues, such as lack of

strong competition in the rural areas (RCCs monopolize rural areas), potential

effectiveness of village and township banks as they are initiated by RCCs or RCBs,

ownership structure of various institutions and prudential regulatory issues,

strengthening of governance are all areas where further research and reforms are

needed. However, it is unclear when, if at all, further reforms in these areas will be

embarked upon by CBRC. IFAD should maintain regular interaction with CBRC and

the Ministry of Finance to explore possibility of participating in future rural finance

sector reforms. Concrete proposals as to how to structure a policy type of loan

from IFAD’s perspective were included in the MTR and are still valid.

84. With fast economic growth the need for small loans is expected to decline in

prospering rural areas, but the demand of loans for farming is expected to

continue. Financial services in remote and isolated areas as well as in central and

western provinces are needed for poverty reduction. Therefore, expansion of

financial services as well as developing appropriate institutional arrangements for

these areas are important in an environment where RCCs and RCBs are

increasingly driven by profit motives and inevitably are moving away from less

profitable areas and businesses.

85. IFAD has three on-going projects with rural finance components in China, which

were designed after RFSP. All these projects are implemented through either RCCs

or women federations in central and western provinces, which are poorer,

compared to eastern provinces. IFAD’s assistance in Chongqing and Shaanxi will no

longer be needed as the rural economy in both provinces is growing fast and both

financial institutions are institutionally and financially capable of offering financial

services to the target population. In this context, future IFAD interventions should

25

continue developing and expanding financial services in poorer provinces and

remote communities. Lastly, the RIMS and other impact studies should include

profitability analysis of sample income generating activities financed by IFAD

projects. This would be useful to document increases in income from income-

generating activities and the contribution of such income to total family annual

income. Such analysis will provide learned answers while assessing the direct

impact of financial services to the poor.

Annex I

26

Rating comparison

Criteria IFAD-PMD ratinga PPA rating

a Rating disconnect

Project performance

Relevance 4 4

Effectiveness 4 5 +1

Efficiency 4 4

Project performanceb 4 4

Rural poverty impact

Household income and assets 5 5

Human and social capital and empowerment 4 4

Food security and agricultural productivity 5 5

Natural resources environment and climate change n.a. n.a

Institutions and policies 4 4

Rural poverty impactc 5 5

Other performance criteria

Sustainability 4 5 +1

Innovation and scaling up 4 5 +1

Gender equality and women’s empowerment 4 4

Overall project achievementd 4 5 +1

Performance of partnerse

IFAD 5 4

Government 4 4

Average net disconnect

a Rating scale: 1 = highly unsatisfactory; 2 = unsatisfactory; 3 = moderately unsatisfactory; 4 = moderately satisfactory; 5 =

satisfactory; 6 = highly satisfactory; n.p. = not provided; n.a. = not applicable. b Arithmetic average of ratings for relevance, effectiveness and efficiency. c This is not an average of ratings of individual impact domains.

d This is not an average of ratings of individual evaluation criteria but an overarching assessment of the project, drawing

upon the rating for relevance, effectiveness, efficiency, rural poverty impact, sustainability, innovation and scaling up, and gender. e The rating for partners’ performance is not a component of the overall assessment ratings.

Annex II

27

Basic project data

Approval (US$ m) Actual (US$ m)

Region Asia and the Pacific Total project costs 21.3 30.4

Country People’s Republic

of China IFAD loan and percentage of total 14.7 69% 15.9 52%

Loan number 634 Borrower 0.406 1.9%

Type of project (subsector) CREDI

Rural Credit Cooperatives (loan) 6.1 28.6%

Financing type E

Rural Credit Cooperatives Unions (grant) 0.097 0.46%

Lending termsa HC Cofinancier 3

Date of approval 21 April 2004 Cofinancier 4

Date of loan signature 27 May 2004

China Banking Regulatory Comission (grant) 0.024 0.12%

Date of effectiveness 13 September 2005 Other sources:

Loan amendments 1

Number of beneficiaries

(if appropriate, specify if direct or indirect)

120,000 rural households

Direct: 36,224 households

Indirect:

830,584 households

Loan closure extensions 1

Country programme managers

E. Martens

T. Rath

S. Jatta Loan closing date 31 March 2010 30 September

2010

Regional director(s) T. Elhaut Mid-term review April 2007

Project completion report reviewer L. Kellens

IFAD loan disbursement at project completion (%) 95.74%

Project completion report quality control panel

A. Brubaker

A. Lambert Date of project completion report February 2011

Source: Project Completion report a There are four types of lending terms: (i) special loans on highly concessional terms, free of interest but bearing a service

charge of three fourths of one per cent (0.75%) per annum and having a maturity period of 40 years, including a grace period of 10 years; (ii) loans on hardened terms, bearing a service charge of three fourths of one per cent (0.75%) per annum and having a maturity period of 20 years, including a grace period of 10 years; (iii) loans on intermediate terms, with a rate of interest per annum equivalent to 50% of the variable reference interest rate and a maturity period of 20 years, including a grace period of 5 years; (iv) loans on ordinary terms, with a rate of interest per annum equivalent to one hundred per cent (100%) of the variable reference interest rate, and a maturity period of 15-18 18 years, including a grace period of three years.

Annex III

28

Terms of reference

I. Background 1. The Independent Office of Evaluation of IFAD (IOE) will conduct a project

performance assessment (PPA) of the Rural Finance Sector Programme (RFSP) in

China The PPA is a project-level evaluation aiming at: (i) providing an independent

assessment of the results and impact of the project / programme under

consideration; and (ii) generating findings and recommendations for the design and

implementation of on-going and future operations in the country.

2. PPAs are conducted on a sample of projects for which a project completion report

(PCR) has been validated by IOE and taking into consideration the following

criteria: (i) synergies with forthcoming or on-going IOE evaluations; (ii) major

information gaps in the PCR; (iii) novel approaches; and (iv) geographic balance.

In the case of RFSP, a project completion report validation was undertaken in 2011

and forms the basis for this PPA exercise.

3. The PPA applies the evaluation criteria outlined in the IFAD Evaluation Manual. In

view of the time and resources available, the PPA is generally not expected to

undertake quantitative surveys; rather, it adds analysis based on interviews at

IFAD headquarters, interactions with stakeholders in the country including project

beneficiaries, and direct observations in the field.

4. Country context. Over the past 30 years, China has emerged as one of the

world’s leading powers, and it boasts the world’s second largest economy. The

government has stepped up investments in rural areas to meet the growing market

demand for agricultural products and to improve the livelihoods of rural people.

One result has been a dramatic decrease over the last 30 years in the number of

people living in absolute poverty. China’s GDP was nearly US$5 trillion in 2009

(almost double compared to 2006), and annual GDP per capita growth reached

8.5% in 2009. This positive economic climate, in collaboration with the on-going

rural finance reform, challenges the attribution of the rural poverty impact

measured in the programme area.

5. Project description. The Rural Finance Sector Programme (RFSP) was designed

with the overall objective to ensure that rural financial services contribute

effectively and sustainably to reducing poverty. Its specific objectives were to

ensure that: (i) rural households, including poor households, have better access to

financial services and effectively make use of them to improve their living

standards; (ii) Rural Credit Cooperatives (RCC) policy reforms have been

successfully tested in the programme area and are being implemented in IFAD-

financed interventions elsewhere; (iii) improved institutional and operational

management capacity in programme RCCs is applied on a larger scale and

contributes to improving cost-effectiveness and profitability; and (iv) modalities to

resolve the problem of non-performing loans have been tested and applied on a

wider scale.

6. The programme was to be implemented at the national level to support the China

Banking Regulatory Commission (CBRC) and the Rural Credit Cooperatives Unions

(RCCUs) in testing a number of policy and institutional reforms. Those tests were

to be undertaken in selected geographical areas before applying the reforms at a

national scale. Design provided for pilot activities in the project areas of on-going

IFAD-funded interventions in order to ensure that microfinance policies are

complemented by both agricultural and social support services and infrastructure

development. The programme would initially be implemented in Chongqing and

Shaanxi Provinces.

Annex III

29

7. The programme was expected to benefit about 120,000 poor rural households.

Several of the above-mentioned policy reforms were expected to improve the

access of poor households to loans, especially in terms of micro and group lending

and loans for women. Other policies were to help to transform the RCCs into

sustainable financing institutions, which was in the interest of all their clients but

especially the poor for whom RCCs constitute the only source of formal deposits

and loans. The programme would support those policy reforms and assist in on-

going or planned policy adjustments with a view to help refining implementation

modalities and taking care that new policies have adequate poverty and equal

access/gender dimensions.

8. The programme comprised five components:

(a) Policy development (11 per cent of total base costs) which focuses on as-

sisting on-going or planned policy adjustments, refining implementation mo-

dalities, and ensuring that new policies have clear gender and poverty dimen-

sions;

(b) Institutional development (19 per cent of base cost) focusing on infra-

structure and equipment for RCCs, staff training and the legal structure;

(c) Operational development (4 per cent of base cost) to support the account-

ing system and policies and the staff incentive system;

(d) Financing (60 per cent of base cost) through the provision of credit lines to

help catalyse innovative microfinance practices and to demonstrate the opera-

tional viability of well-designed credit programmes to the RCCs; and

(e) Programme management (6 per cent of base cost).

9. The principal programme partners were the networks of RCCs that were federated

up to the province level, but do not have a national apex. The CBRC and provincial

RCCUs in the two programme provinces of Chongqing and Shaanxi were the

principal implementation partners of the programme. Programme Management

Offices (PMOs) were established within provincial RCCUs and were expected to

receive strategic and policy guidance related to RFSP from the CBRC, and to

manage and coordinate operations undertaken by county RCCUs at field level. A

National coordination and monitoring office (NCMO), responsible for providing

policy guidance, was planned to be established and located at the CBRC head

office.

II. Methodology 10. Objectives. The main objectives of the PPA are to: (i) assess the results of the

programme; and (ii) generate findings and recommendations for the design and

implementation of on-going and future operations in China.

11. Scope. The PPA will take account of the preliminary findings of the PCRV and

further desk review, issues emerging from interviews at IFAD headquarters, and a

focused mission to the country for the purpose of generating a comprehensive,

evidence-based evaluation. However, the PPA will not need to examine or re-

examine the full spectrum of programme activities, achievements and drawbacks,

but will focus on selected key issues. Furthermore, subject to the availability of

time and budgetary resources, due attention will be paid to filling in the major

evaluative information gaps of the PCR and other programme documents.

12. Evaluation criteria. In line with the evaluation criteria outlined in IOE’s Evaluation

Manual (2009), added evaluation criteria (2010)6 and the IOE Guidelines for PCRV

and PPA (January 2012), the key evaluation criteria applied in this PPA will include:

i. Relevance, which is assessed both in terms of alignment of project objec-

tives with country and IFAD policies for agriculture and rural development

6 Gender, climate change, and scaling up

Annex III

30

and the needs of the rural poor, as well as project design features geared to

the achievement of project objectives;

ii. Effectiveness, which measures the extent to which the project’s immediate

objectives were achieved, or are expected to be achieved, taking into account

their relative importance;

iii. Efficiency, which indicates how economically resources/inputs are converted

into results;

iv. Rural poverty impact, which is defined as the changes that have occurred

or are expected to occur in the lives of the rural poor (whether positive or

negative, direct or indirect, intended or unintended) as a results of develop-

ment interventions. Five impact domains are employed to generate a compo-

site indication of rural poverty impact: household income and assets; human

and social capital and empowerment; food security and agricultural produc-

tivity; natural resources, environment and climate change; and institutions

and policies;

v. Sustainability, indicating the likely continuation of net benefits from a de-

velopment intervention beyond the phase of external funding support. It also

includes an assessment of the likelihood that actual and anticipated results

will be resilient to risks beyond the project’s life;

vi. Innovation and scaling up, assessing the extent to which IFAD develop-

ment interventions have introduced innovative approaches to rural poverty

reduction and the extent to which these interventions have been (or are likely

to be) replicated and scaled up by government, private sector and other

agencies;

vii. Gender equality and women’s empowerment, This criterion is related to

the relevance of design in terms of gender equality and women’s empower-

ment, the level of resources committed, and changes promoted by the pro-

ject; and

viii. Performance of partners, including the performance of IFAD and the Gov-

ernment, will be assessed on an individual basis, with a view to the partners’

expected role and responsibility in the project life cycle.

13. Data collection. The PPA will be built on the initial findings of the PCRV. For

further information, interviews will be conducted both at IFAD headquarters and in

China. In the course of the in-country mission, additional primary and secondary

data will be collected in order to reach an independent assessment of performance

and results. Data collection methods will mostly include qualitative participatory

techniques. The methods deployed will consist of individual and group interviews,

focus group discussions with beneficiaries, and direct observations. The PPA will

also make use – where applicable – of additional data available through the

programme’s monitoring and evaluation system. Triangulation will be applied to

verify findings emerging from different information sources.

14. Stakeholders’ participation. In compliance with the Evaluation Policy of 2011,

the main programme stakeholders will be involved throughout the PPA. This will

ensure that the key concerns of the stakeholders are taken into account, that the

evaluators fully understand the context in which the programme was implemented,

and that opportunities and constraints faced by the implementing institutions are

identified. Regular interaction and communication will be established with the Asia

and the Pacific Division (APR) of IFAD and with the Government of China. Formal

and informal opportunities will be explored during the process for the purpose of

discussing findings, lessons and recommendations.

Annex III

31

III. Evaluation process 15. In all, the PPA will involve five phases: desk work; in-country work; report drafting

and peer review; receipt of comments from APR and the Government of China; and

the final phase of communication and dissemination.

16. Desk work phase. The related PCRV for RFSP and further desk review based on

official project documentation and other evaluative material as appropriate will

provide initial findings and identify key issues to be investigated by the PPA.

17. Country work phase. The PPA mission is scheduled for 31 May – 12 June 2012,

subject to the agreement by the Government of China. Mission members will

interact with the Government, local authorities, local partners, NGOs, programme

staff and clients (beneficiaries), and collect information from the programme’s

monitoring and evaluation system and other sources. At the end of the mission, a

brief will be provided to the IFAD partner ministry, followed by a wrap-up meeting

in Beijing, to summarize the preliminary findings and discuss key strategic and

operational issues.

18. Report drafting and peer review. At the conclusion of the field visit, a draft PPA

report will be prepared and submitted to IOE internal peer review for quality

assurance. Konstantin Atanesyan, Senior Evaluation Officer, together with another

evaluation officer to be designated, will be the peer reviewers for the PPA.

19. Comments by APR and the Government. The PPA report will be shared with APR

and thereafter with the Government for comments. IOE will finalize the report

following receipt of the Government’s comments.

20. Communication and dissemination. The final report will be disseminated among key

stakeholders and the evaluation report published by IOE, both online and in print.

IV. Key issues for investigation

21. According to the PCRV’s findings, programme support to reforming the rural

finance networks in Chongqing and Shaanxi has contributed to strengthened

institutional, financial, operational and technical sustainability of those institutions.

Improved access to financial services has impacted on the livelihoods of rural

households in the programme counties, although to a less extent that would have

been expected from a programme of such scope. Rural household assets and

production improved, as well as food security, human and social capital was

strengthened and the socio-economic status of women improved. RFSP did bring

innovative approaches in segmenting the lending market and adapting lending

products for the poor and women. However, the programme main weakness can be

found in the visionary but ambitious design, the weak poverty focus at the lowest

level, and the absence of clear linkages with other IFAD-funded projects.

22. In reaffirming the good overall quality of the PCR in presenting and analysing the

programme results, IOE found that, while rightly assessing project performance

against the evaluation criteria adopted by PMD and IOE, the report concentrated on

the final results, giving very little information about the changes that took place

during implementation and problems that had to be addressed and resolved. Also,

the report did not mention the reason for the 16 months gap between project

approval and loan signing. The PPA will therefore collect additional data in these

respects.

23. A review of the PCRV has led to the identification of a number of important issues,

some relevant to present challenges faced by IFAD. As such, the following issues,

among others, will be further investigated by the PPA.

24. Household income and net assets. Despite the implicit results chain of the

programme (improved access to financial services allowing rural households to

engage in income generating activities to improve their income), no specific data

Annex III

32

was collected regarding household income and income generating activities. Using

the RIMS data, the PCR suggest that income increased in the programme villages;

for example, in Chongqing, poor or low-income decreased from 16% in 2007 to

less than 5% in 2009; in Shaanxi, surveys recorded 11% of poor and low-income

in 2007, which was reduced to about 8% in 2009. However, these improvements

seem to correspond with the overall poverty reduction in the country (see

paragraph 4). The PPA will attempt to clarify this issue through data collection and

other evaluative methods.

25. Institutions and policies. The programme did not conduct the official policy

consultation as to the central regulatory agency as suggested by the Appraisal

report and the Mid-Term Review. The latter proposed an alternative policy

consultative mechanism after the retreat of CBRC to ensure that any systematic

innovations or operational breakthroughs obtained by the programme would be

forwarded to policy makers of sound practices in the national RCC reform.

However, the completion report does not provide more information or analysis on

why this was not done. The PPA mission will enquire on this aspect and report

accordingly.

26. Innovation and scaling up. The completion report indicates that the innovations

introduced by RFSP were not captured by PMOs and IFAD for replication and

scaling up in future project. The project design did not include an explicit

mechanism for further innovation, replication and scaling up, and this has not

occurred. Moreover, given the changed role of the CBRC, there was no apparent

mechanism to transfer knowledge and experience gained in provincial and country

level reform from this project to subsequent programme activities related to RCCs.

The PPA will analyse this specific aspect and provide recommendations for future

interventions.

27. Gender equality and women’s empowerment. Programme design targeted

poor households in the pilot counties, and provided for a Women Window so that

women could contract individual or group loans in their own names. However, this

activity did not deliver the expected results pre-MTR, mainly because of the lack of

adjusted loan products and the insufficient coverage of remote poor villages. The

post-MTR period focused on product development and promoted the introduction of

village credit models (which would further increase access for women). Despite

sporadic reports on this, there seemed to be no application of this village credit

model. The PPA mission will review this aspect and report accordingly.

V. Evaluation team 28. Under the guidance of Konstantin Atanesyan, Senior Evaluation Officer, Mr Mark

Keating, Evaluation Officer, has been appointed as Lead Evaluator for this PPA and

will be responsible for delivering the final report. He will be assisted by a senior

consultant as rural finance expert who will lead the mission and prepare the draft

report.

Annex IV

33

Methodological note on project performance assessments

A. What is a project performance assessment?1

1. The project performance assessment (PPA) conducted by the Independent Office of

Evaluation of IFAD (IOE) entails one mission of 7-10 days2 and two mission

members.3 PPAs are conducted on a sample of projects for which project

completion reports have been validated by IOE, and take account of the following

criteria (not mutually exclusive): (i) synergies with forthcoming or on-going IOE

evaluations (e.g. country programme or corporate-level evaluations); (ii) major

information gaps in project completion reports (PCRs); (iii) novel approaches; and

(iv) geographic balance.

2. The objectives of the PPA are to: assess the results and impact of the project under

consideration; and (ii) generate findings and recommendations for the design and

implementation of on-going and future operations in the country involved. When

the PPA is to be used as an input for a country programme evaluation, this should

be reflected at the beginning of the report. The PPA is based on the project

completion report validation (PCRV) results, further desk review, interviews at IFAD

headquarters, and a dedicated mission to the country, to include meetings in the

capital city and field visits. The scope of the PPA is set out in the respective terms

of reference.

B. Preparing a PPA

3. Based on the results of the PCRV, IOE prepares brief terms of reference (ToR) for

the PPA in order to sharpen the focus of the exercise.4 As in the case of PCRVs,

PPAs do not attempt to respond to each and every question contained in the

Evaluation Manual. Instead, they concentrate on the most salient facets of the

criteria calling for PPA analysis, especially those not adequately explained in the

PCRV.

4. When preparing a PPA, the emphasis placed on each evaluation criterion will

depend both on the PCRV assessment and on findings that emerge during the PPA

process. When a criterion or issue is not identified as problematic or in need of

further investigation, and no additional information or evidence emerges during the

PPA process, the PPA report will re-elaborate the PCRV findings.

Scope of the PPA

1 Extract from the PCRV and PPA Guidelines.

2 PPAs are to be conducted within a budget ceiling of US$25,000.

3 Typically, a PPA mission would be conducted by an IOE staff member with the support of a consultant (international

or national). An additional (national) consultant may be recruited if required and feasible within the evaluation budget. 4 Rather than an approach paper, IOE prepares terms of reference for PPAs. These terms of reference ensure

coverage of information gaps, areas of focus identified through PCRVs and comments by the country programme manager, and will concentrate the PPA on those areas. The terms of reference will be included as an annex to the PPA.

PCRV

assessment

PPA process

PPA ToR: Emphasis on selected criteria

and issues are defined

PPA report considers all criteria but

emphasizes selected criteria and issues

Annex IV

34

C. Evaluation criteria

5. The PPA is well suited to provide an informed summary assessment of project

relevance. This includes assessing the relevance of project objectives and of

design. While, at the design stage, project logical frameworks are sometimes

succinct and sketchy, they do contain a number of (tacit) assumptions on

mechanisms and processes expected to generate the final results. At the post-

completion phase, and with the benefit of hindsight, it will be clearer to the

evaluators which of these assumptions have proved to be realistic, and which did

not hold up during implementation and why.

6. For example, the PPA of a project with a major agricultural marketing component

may consider whether the project framework incorporated key information on the

value chain. Did it investigate issues relating to input and output markets

(distance, information, monopolistic power)? Did it make realistic assumptions on

post-harvest conservation and losses? In such cases, staff responsible for the PPA

will not be expected to conduct extensive market analyses, but might consider the

different steps (e.g. production, processing, transportation, distribution, retail)

involved and conduct interviews with selected actors along the value chain.

7. An assessment of effectiveness, the extent to which a project’s overall objectives

have been achieved, should be preferably made at project completion, when the

components are expected to have been executed and all resources fully utilized.

The PPA considers the overall objectives5 set out in the final project design

document and as modified during implementation. At the same time, it should be

flexible enough to capture good performance or under-performance in areas that

were not defined as an objective in the initial design but emerged during the

course of implementation.

8. The PPA mission may interview farmers regarding an extension component, the

objective of which was to diffuse a certain agricultural practice (say, adoption of a

soil nutrient conservation technique). The purpose here would be to understand

whether the farmers found it useful, to what extent they applied it and their

perception of the results obtained. The PPA may look into reasons for the farmers’

interest in new techniques, and into adoption rates. For example, was the

extension message delivered through lectures? Did extension agents use audio-

visual tools? Did extension agents engage farmers in interactive and participatory

modules? These type of questions help illustrate why certain initiatives have been

conducive (or not conducive) to obtaining the desired results.

9. The Evaluation Manual suggests methods for assessing efficiency, such as

calculating the economic internal rate of return,6 estimating unit costs and

comparing them with standards (cost-effectiveness approach), or addressing

managerial aspects of efficiency (timely delivery of activities, respect of budget

provisions). The documentation used in preparing the PCRV should normally

provide sufficient evidence of delays and cost overruns and make it possible to

explain why they happened.

10. As far as rural poverty impact is concerned, the following domains are

contemplated in the Evaluation Manual: (a) household income and assets;

(b) human and social capital and empowerment; (c) food security and agricultural

5 Overall objectives will be considered as a reference for assessing effectiveness. However, these are not always

stated clearly or consistent throughout the documentation. The assessment may be made by component if objectives are defined by components; however the evaluation will try to establish a correspondence between the overall objectives and outputs. 6 Calculating an EIRR may be challenging for a PPA as it is time consuming and the required high quality data are often

not available. The PPA may help verify whether some of the crucial assumptions for EIRR calculation are consistent with field observations. The mission may also help shed light on the cost-effectiveness aspects of efficiency, for example whether, in an irrigation project, a simple upgrade of traditional seasonal flood water canalization systems might have been an option, rather than investing on a complex irrigation system, when access to markets is seriously constrained.

Annex IV

35

productivity; (d) natural resources, the environment and climate change;7 and

(e) institutions and policies. As shown in past evaluations, IFAD-funded projects

generally collect very little data on household or community-level impact

indicators. Even when impact data are available, both their quality and the

methodological rigour of impact assessments are still questionable. For example,

although data report significant increases in household assets, these may be due to

exogenous factors (e.g. falling prices of certain commodities; a general economic

upturn; households receiving remittances), and not to the project.

11. PPAs may help address the “attribution issue” (i.e. establishing to what extent

certain results are due to a development intervention rather than to exogenous

factors) by:

(i) following the logical chain of the project, identifying key hypotheses and

reassessing the plausibility chain; and

(ii) conducting interviews with non-beneficiaries sharing key characteristics (e.g.

socio-economic status, livelihood, farming system), which would give the

mission an idea of what would have happened without the project

(counterfactual).8

12. When sufficient resources are available, simple data collection exercises (mini-

surveys) may be conducted by a local consultant prior to the PPA mission.9 Another

non-mutually exclusive option is to spot-check typical data ranges or patterns

described in the PCR by means of case studies (e.g. do PCR claims regarding

increases in average food-secure months fall within the typical ranges recorded in

the field?). It is to be noted that, while data collected by a PPA mission may not be

representative in a statistical sense, such data often provide useful reference points

and insights. It is important to exercise care in selecting sites for interviews in

order to avoid blatant cases of non-beneficiaries profiting from the project.). Sites

for field visits are selected by IOE in consultation with the government concerned.

Government staff may also accompany the PPA mission on these visits.

13. The typical timing of the PPA (1-2 years after project closure) may be useful for

identifying factors that enhance or threaten the sustainability of benefits. By that

stage, the project management unit may have been disbanded and some of the

support activities (technical, financial, organizational) terminated, unless a second

phase is going forward or other funding has become available. Typical factors of

sustainability (political support, availability of budgetary resources for

maintenance, technical capacity, commitment, ownership by the beneficiaries,

environmental resilience) can be better understood at the ex post stage...

14. The PPA also concentrates on IFAD’s role with regard to the promotion of

innovations and scaling up. For example, it might be observed that some

innovations are easily scaled up at low cost (e.g. simple but improved cattle-

rearing practices that can be disseminated with limited funding). In other cases,

scaling up may involve risks: consider the case of a high-yield crop variety for

which market demand is static. Broad adoption of the variety may be beneficial in

terms of ensuring food security, but may also depress market prices and thereby

reduce sale revenues for many households unless there are other, complementary

activities for the processing of raw products.

15. The PPA addresses gender equality and women’s empowerment, a criterion

recently introduced into IFAD’s evaluation methodology. This relates to the

emphasis placed on gender issues: whether it has been followed up during

7 Climate change criterion will be addressed if and when pertinent in the context of the project, as most completed

projects evaluated did not integrate this issue into the project design. 8 See also the discussion of attribution issues in the section on PCRVs.

9 If the PPA is conducted in the context of a country programme evaluation, then the PPA can piggy-back on the CPE

and dedicate more resources to primary data collection.

Annex IV

36

implementation, including the monitoring of gender-related indicators; and the

results achieve.

16. Information from the PCRV may be often sufficient to assess the performance of

partners, namely, IFAD and the government. The PPA mission may provide further

insights, such as on IFAD’s responsiveness, if relevant, to implementation issues or

problems of coordination among the project implementation unit and local and

central governments. The PPA does not assess the performance of cooperating

institutions, which now has little or no learning value for IFAD.

17. Having completed the analysis, the PPA provides its own ratings in accordance with

the evaluation criteria and compares them with PMD’s ratings. PPA ratings are final

for evaluation reporting purposes. The PPA also rates the quality of the PCR

document.

18. The PPA formulates short conclusions: a storyline of the main findings. Thereafter,

a few key recommendations are presented with a view to following up projects, or

other interventions with a similar focus or components in different areas of the

country.10

10

Practices differ among multilateral development banks, including recommendations in PPAs. At the World Bank, there are no recommendations but “lessons learned” are presented in a typical PPA. On the other hand, PPAs prepared by Asian Development Bank include “issues and lessons” as well as “follow-up actions” although the latter tend to take the form of either generic technical guidelines for a future (hypothetical) intervention in the same sector or for an ongoing follow-up project (at Asian Development Bank, PPAs are undertaken at least three years after project closure).

Annex V

37

Definition of the evaluation criteria used by IOE

Criteria Definitiona

Project performance

Relevance The extent to which the objectives of a development intervention are consistent with beneficiaries’ requirements, country needs, institutional priorities and partner and donor policies. It also entails an assessment of project design in achieving its objectives.

Effectiveness The extent to which the development intervention’s objectives were achieved, or are expected to be achieved, taking into account their relative importance.

Efficiency A measure of how economically resources/inputs (funds, expertise, time, etc.) are converted into results.

Rural poverty impactb Impact is defined as the changes that have occurred or are expected to occur in

the lives of the rural poor (whether positive or negative, direct or indirect, intended or unintended) as a result of development interventions.

Household income and assets

Household income provides a means of assessing the flow of economic benefits accruing to an individual or group, whereas assets relate to a stock of accumulated items of economic value.

Human and social capital and empowerment

Human and social capital and empowerment include an assessment of the changes that have occurred in the empowerment of individuals, the quality of grassroots organizations and institutions, and the poor’s individual and collective capacity.

Food security and agricultural productivity

Changes in food security relate to availability, access to food and stability of access, whereas changes in agricultural productivity are measured in terms of yields.

Natural resources, the environment and climate change

The focus on natural resources and the environment involves assessing the extent to which a project contributes to changes in the protection, rehabilitation or depletion of natural resources and the environment as well as in mitigating the negative impact of climate change or promoting adaptation measures.

Institutions and policies The criterion relating to institutions and policies is designed to assess changes in the quality and performance of institutions, policies and the regulatory framework that influence the lives of the poor.

Other performance criteria

Sustainability

The likely continuation of net benefits from a development intervention beyond the phase of external funding support. It also includes an assessment of the likelihood that actual and anticipated results will be resilient to risks beyond the project’s life.

Innovation and scaling up The extent to which IFAD development interventions have: (i) introduced innovative approaches to rural poverty reduction; and (ii) the extent to which these interventions have been (or are likely to be) replicated and scaled up by government authorities, donor organizations, the private sector and others agencies.

Gender equality and women’s empowerment

The criterion assesses the efforts made to promote gender equality and women’s empowerment in the design, implementation, supervision and implementation support, and evaluation of IFAD-assisted projects.

Overall project achievement This provides an overarching assessment of the project, drawing upon the analysis made under the various evaluation criteria cited above.

Performance of partners

IFAD

Government

This criterion assesses the contribution of partners to project design, execution, monitoring and reporting, supervision and implementation support, and evaluation. It also assesses the performance of individual partners against their expected role and responsibilities in the project life cycle.

a These definitions have been taken from the OECD/DAC Glossary of Key Terms in Evaluation and Results-Based Management

and from the IFAD Evaluation Manual (2009). b

The IFAD Evaluation Manual also deals with the “lack of intervention”, that is, no specific intervention may have been foreseen or

intended with respect to one or more of the five impact domains. In spite of this, if positive or negative changes are detected and can be attributed in whole or in part to the project, a rating should be assigned to the particular impact domain. On the other hand, if no changes are detected and no intervention was foreseen or intended, then no rating (or the mention “not applicable”) is assigned.

Annex VI

38

List of key persons met

Beijing

Mr Sun Yinhong, IFAD China Presence Officer

Ms Weijing Wang, IFAD Programme Officer

Mr Peter Situ, Rural Finance Specialist and author of PCR

Mr Zheng Kangbing, Head, Private Sector Coordination Unit, ADB, PRC

Mr Wang Jun, Lead Financial Specialist, The World Bank, China

Ms Wang Ying, Financial Sector Specialist, The World Bank, China Representative, Robbo

Bank

Dr Guangwen He, Professor and Director, Center for Rural Finance and Investment

Research, China Agricultural University

Ministry of Finance

Mr Zhang Lee, International Financial Institutional Div III, International Department

Re Chongqing Province

Mr Liu Zunming, Principal Staff, Chongqing Municipal Finance Bureau, External Finance

Division

Ms Jing Shu, General Manager, Head Office Company Business Office, Chongqing Rural

Commercial Bank

Ms Chen Yan, Vice General Manager, Corporate Banking Dept., Chongqing Rural

Commercial Bank

Ms Xin Gao, Board Office, Investor Relations Management

Ms Chen Xi, PMO

Branch Manager, CRCB, Younyang County

Credit Officers, CRCB, Younyang County

Project beneficiaries, Younyang County

Shaanxi Province

Mr Peng Hui, PMO, Shaanxi, PRCCU

Director, Shaanxi, PRCCU

Mr Wang, Chief of Xixian RCC

Chairman, Xixian RCC

Credit Officers, Xixian RCC

Beneficiaries, Xixian RCC

Annex VII

39

Bibliography

International Fund for Agricultural Development (IFAD) (1999). Country Strategic

Opportunities Paper.

____ (2001). Rural Financial Services in China, Thematic Study.

____ (2003). Appraisal Report.

____ (2004). Report and Recommendation of the President. Rural Finance Sector

Programme.

____ (2007). Mid Term Review.

____ (2008). Aide-Mémoire. Supervision mission May 2008.

____ (2009). Aide-Mémoire. Supervision mission April 2009.

____ (2009). Rural Finance Policy.

____ (2011). Project Completion Report. Main Report.

____ (2011). Project Completion Report. Appendixes 1 and 2.

____ (2011). Project Completion Report.

____ (2011). Country Programme Review.

____ (various years). Project Status Reports and Country Programme Issues Sheets.

IFAD/Independent Office of Evaluation (2010). Interim Evaluation Qinling Mountain Area

Poverty-Alleviation Project, People’s Republic of China.

____ (2010). Completion Evaluation West Guangxi Poverty-Alleviation Project, People’s

Republic of China.

____ (2011). Project Completion Report Validation.

United Nations Office for Project Services (UNOPS) (2006). Supervision Report.

____ (2007). Supervision Report.

P R O J E C T P E R F O R M A N C E A S S E S S M E N T

December 2013

IFAD Internal Printing Services

People's Republic of China

Rural Finance Sector Programme

International Fund for Agricultural Development

Via Paolo di Dono, 44 - 00142 Rome, ItalyTel: +39 06 54591 - Fax: +39 06 5043463E-mail: evaluation@ifad.orgwww.ifad.org/evaluationwww.ruralpovertyportal.org

ifad-un.blogspot.com

www.facebook.com/ifad

www.twitter.com/ifadnews

www.youtube.com/user/ifadTV

Independent Offi ce of Evaluation