document of the world bank...02 19-dec-2011 satisfactory satisfactory 1.43 03 07-jul-2012...

The World Bank Higher Education Quality and Capacity Improvement Project ( P106605 )

of 71

Document of

The World Bank FOR OFFICIAL USE ONLY

Report No: ICR00004320

IMPLEMENTATION COMPLETION AND RESULTS REPORT Credit Number 4796-KH and Grant Number 607-KH

ON A

CREDIT

IN THE AMOUNT OF SDR 7.85 MILLION

(US$11.50 MILLION EQUIVALENT)

AND A

GRANT

IN THE AMOUNT OF SDR 7.85 MILLION (US$11.50 MILLION EQUIVALENT)

TO THE

KINGDOM OF CAMBODIA

FOR THE

HIGHER EDUCATION QUALITY AND CAPACITY IMPROVEMENT PROJECT ( P106605 )

March 22, 2018

Education Global Practice East Asia and the Pacific Region

This document has a restricted distribution and may be used by recipients only in the performance of their official duties. I ts contents may not otherwise be disclosed without World Bank authorization.

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized


of 71

CURRENCY EQUIVALENTS

(Exchange Rate Effective August 24, 2017)

Currency Unit = Cambodian Riel (KHR)

KHR 4,041= US$1

US$1.358 = SDR 1

FISCAL YEAR July 1–June 30

ABBREVIATIONS AND ACRONYMS

ACC Accreditation Committee of Cambodia ACI ASEAN Citation Index ASEAN Association of Southeast Asian Nations BBU Build Bright University CAS Country Assistance Strategy CBA Cost-Benefit Analysis CCI Cambodia Citation Index CDRI Cambodia Development Research Institute CDU Charles Darwin University CSRD Cambodia Society for Research and Development DGHE Director General of Higher Education DHE Department of Higher Education DSR Department of Scientific Research ESP Education Strategic Plan ESSP Education Sector Support Project FM Financial Management GDP Gross Domestic Product HE Higher Education HEI Higher Education Institution HEIP Higher Education Improvement Project HEMIS Higher Education Management Information System HEQCIP Higher Education Quality and Capacity Improvement Project HR Human Resource ICR Implementation Completion and Results Report IDP Industrial Development Policy IFL Institute of Foreign Languages IFR Interim Financial Report IQA Internal Quality Assurance IRR Internal Rate of Return


of 71

ISR Implementation Status and Results Report ITC Institute of Technology of Cambodia M&E Monitoring and Evaluation MEF Ministry of Economic and Finance MoEYS Ministry of Education, Youth, and Sports MOU Memorandum of Understanding MTR Midterm Review NPV Net Present Value NTG Northern Territories Government PCC Project Coordination Committee PDO Project Development Objective PMC Program Management Committee POC Priority Operating Costs PUC Pannasatra University of Cambodia RGMC Research Grant Management Committee RUPP Royal University of Phnom Penh SIL Specific Investment Loan SPS Special Priority Scholarship SRU Svay Reang University STEAM Science, Technology, Engineering, Arts, and Mathematics STEM Science, Technology, Engineering, and Mathematics UC University of Canberra UHR University of Human Resources UHS University of Health Sciences

Regional Vice President: Victoria Kwakwa

Country Director: Ellen Goldstein

Senior Global Practice Director: Jaime Saavedra Chanduvi

Practice Manager: Harry Anthony Patrinos

Task Team Leader(s): Tsuyoshi Fukao

ICR Main Contributor: Sandra Beemer


of 71

TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 2

I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 9

A. CONTEXT AT APPRAISAL................................................................................................................... 9

B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ......................................... 15

II. OUTCOME .................................................................................................................... 18

A. RELEVANCE OF PDOs ...................................................................................................................... 18

B. ACHIEVEMENT OF PDOs (EFFICACY) .............................................................................................. 18

C. EFFICIENCY ...................................................................................................................................... 23

D. JUSTIFICATION OF OVERALL OUTCOME RATING .......................................................................... 25

E. OTHER OUTCOMES AND IMPACTS (IF ANY) ................................................................................... 25

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 27

A. KEY FACTORS DURING PREPARATION ........................................................................................... 27

B. KEY FACTORS DURING IMPLEMENTATION .................................................................................... 28

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME ... 30

A. QUALITY OF MONITORING AND EVALUATION (M&E) .................................................................. 30

B. ENVIRONMENTAL SAFEGUARD, SOCIAL SAFEGUARD, AND FIDUCIARY COMPLIANCE ............... 31

C. BANK PERFORMANCE ..................................................................................................................... 32

D. RISK TO DEVELOPMENT OUTCOME ............................................................................................... 33

V. LESSONS LEARNED AND RECOMMENDATIONS .............................................................. 34

ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 36

ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 46

ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 48

ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 49

ANNEX 5. BORROWER, CO-FINANCIERS, AND OTHER PARTNERS/STAKEHOLDERS COMMENTS..................................................................................................................................... 52

ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) ..................................................................... 62

ANNEX 7. DETAIL OF MODIFICATIONS TO RESULTS FRAMEWORK DURING OCTOBER 2014 RESTRUCTURING........................................................................................................... 63

ANNEX 8. DETAILS OF PDO RELATED TO ACC ACCREDITATION CRITERIA ............................... 65

ANNEX 9. LIST OF COMPLETED SUB-PROJECTS ...................................................................... 67


of 71

DATA SHEET

BASIC INFORMATION Product Information Project ID Project Name

P106605 Higher Education Quality and Capacity Improvement Project

Country Financing Instrument

Cambodia Specific Investment Loan

Original EA Category Revised EA Category

Not Required (C) Not Required (C)

Organizations

Borrower Implementing Agency

Ministry of Economy and Finance Ministry of Education, Youth and Sport

Project Development Objective (PDO)

Original PDO The development objective of this higher education project is to: (a) improve the quality of teaching, management, and research in project-supported entities; and (b) pilot the targeting of disadvantaged students for enhanced retention.


of 71

FINANCING

Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing IDA-H6070 11,500,000 11,299,640 11,607,358

IDA-47960 11,500,000 11,272,984 11,242,498

Total 23,000,000 22,572,624 22,849,856

Non-World Bank Financing Borrower 0 0 0

Total 0 0 0

Total Project Cost 23,000,000 22,572,623 22,849,856

KEY DATES

Approval Effectiveness MTR Review Original Closing Actual Closing 05-Aug-2010 30-Dec-2010 12-Jul-2013 31-Dec-2015 30-Sep-2017

RESTRUCTURING AND/OR ADDITIONAL FINANCING

Date(s) Amount Disbursed (US$M) Key Revisions 15-Oct-2014 15.12 Change in Results Framework

Change in Components and Cost Reallocation between Disbursement Categories

08-Dec-2015 19.46 Change in Results Framework Change in Loan Closing Date(s) Change in Implementation Schedule

KEY RATINGS

Outcome Bank Performance M&E Quality

Satisfactory Satisfactory Substantial


of 71

RATINGS OF PROJECT PERFORMANCE IN ISRs

No. Date ISR Archived DO Rating IP Rating Actual

Disbursements (US$M)

01 05-Feb-2011 Satisfactory Satisfactory 0

02 19-Dec-2011 Satisfactory Satisfactory 1.43

03 07-Jul-2012 Satisfactory Satisfactory 2.35

04 26-Dec-2012 Satisfactory Satisfactory 4.33

05 04-Jul-2013 Satisfactory Moderately Satisfactory 5.67

06 14-Oct-2013 Moderately Satisfactory Moderately Satisfactory 7.66

07 21-Jun-2014 Moderately Satisfactory Moderately Satisfactory 13.55

08 22-Nov-2014 Satisfactory Moderately Satisfactory 15.12

09 11-May-2015 Moderately Satisfactory Moderately Satisfactory 17.65

10 03-Sep-2015 Moderately Satisfactory Moderately Satisfactory 18.67

11 16-Mar-2016 Moderately Satisfactory Moderately Satisfactory 19.46

12 18-Nov-2016 Moderately Satisfactory Moderately Satisfactory 22.15

13 11-May-2017 Satisfactory Moderately Satisfactory 23.28

SECTORS AND THEMES

Sectors Major Sector/Sector (%)

Education 100

Public Administration - Education 52 Tertiary Education 48

Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%)


of 71

Human Development and Gender 100 Education 100

Access to Education 25

Science and Technology 25

Teachers 25

Standards, Curriculum and Textbooks 25

ADM STAFF

Role At Approval At ICR

Regional Vice President: James W. Adams Victoria Kwakwa

Country Director: Annette Dixon Ellen A. Goldstein

Senior Global Practice Director: Emmanuel Y. Jimenez Jaime Saavedra Chanduvi

Practice Manager: Eduardo Velez Bustillo Harry Anthony Patrinos

Task Team Leader(s): Omporn Regel Tsuyoshi Fukao

ICR Contributing Author: Sandra F. Beemer


of 71

I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES

A. CONTEXT AT APPRAISAL

Context

1. Country context. At the time of appraisal, Cambodia had experienced remarkable economic growth and macroeconomic stability. Its growth rate was 9.5 percent per year on average, and per capita income had doubled during the past decade. In 2007, real gross domestic product (GDP) increased by around 10 percent. Cambodia had managed its transition while ending its three-decade-long conflict and shifting from a command-and-control economy toward greater market orientation. Economic growth had been mainly driven by four sectors: garments, tourism, construction, and agriculture. Overall progress had also been underpinned by an open trade regime with exports increasing by 23 percent per year since 1998. There was a favorable investment regime for foreign direct investment, which had been greater than 6 percent of GDP at the time. Cambodia joined the Association of Southeast Asian Nations (ASEAN) in 1999 and the World Trade Organization in October 2004 and had become increasingly integrated into the world economy. This exposure, however, had increased the impact of the 2008 global economic slowdown on the country. Growth slowed to 5.5 percent in 2008, and initial estimates showed a contraction of 2.2 percent in 2009. In the second half of 2008, the financial crisis originating in the United States affected foreign demand. International prices of commodities exported by Cambodia were on a declining trend, export orders for garments and other industrial products collapsed in the fourth quarter of 2008, and a slowdown in manufacturing became noticeable. According to official statistics, job losses were modest, though the labor market remained tight for the skilled and semiskilled.

2. Despite the decade of robust economic growth, Cambodia was still one of the poorest countries in Southeast Asia. More than 30 percent of its population earned less than US$1 per day, and daily life was characterized by vulnerability due to poverty, the existence of landmines, disability, and a high incidence of mortality and morbidity, the effects of which were exacerbated by the lack of effective social safety nets. One quarter of all households were headed by females, and 59 percent of the population relied on agriculture for their livelihood. The Government of Cambodia and the donor community were working together to address many of these issues, and some success had been realized. During this time, poverty had been reduced by more than 60 percent in Phnom Penh and 44 percent in other urban areas. However, poverty reductions were significantly less in rural areas—about 22 percent.

3. Sector context. During the Khmer Rouge period (1975–1979), Cambodia lost almost three quarters of its educated population. It has been reported by Ayers (2000) that during the People’s Republic of Kampuchea, 75 percent of teachers, 96 percent of higher education (HE) students, and 65 percent of primary and secondary students disappeared. This does not include the other 300,000 individuals that fled the country. Following these years, the Government of Cambodia, in its efforts to address the above needs, recognized the key importance of education in general and HE and research in particular. In the basic education sub-sectors, the Education Strategic Plan (ESP) 2006–2010 focused on two key issues: the achievement of universal access to nine years of high-quality basic education and the promotion of equity in educational opportunities to increase income and employment. With the declining trend for export orders for garments and other industrial products, the government began to focus on the


of 71

need to spur growth by providing more relevant and better skills needed to move the country up the value chain and produce more sophisticated knowledge-intensive products.

4. The Cambodian HE sub-sector had made significant progress and expanded in both the public and private sectors. The Education Law was in place, which provided direction for the future development of the HE sub-sector. Enrollment had grown rapidly, from a little over 10,000 in 1992 to 137,253 (including 12,802 scholarship students) in 2008, of which approximately 60 percent were enrolled in private higher education institutions (HEIs). There were 32 public and 45 private HEIs spread over more than 111 campuses (including branches, which were mostly in the provinces). Public HEIs had been granted partial autonomy, which allowed them to establish admissions criteria, design academic curricula, and charge tuition fees. The Accreditation Committee of Cambodia (ACC) was established in 2003 and had successfully completed assessments of the Foundation Year—a prerequisite for the Full Accreditation Certificate—in 41 HEIs.

5. The Ministry of Education, Youth, and Sports (MoEYS), in consultation with the ACC, was working toward finalizing a vision and strategy for the HE sub-sector. Within the government’s overall development strategy, HE was expected to play a major role. HEIs were seen as having the potential to contribute to better quality education, health, governance, environmental outcomes, and the ongoing debate on major social and cultural issues.

6. Although significant progress had been made in the HE sub-sector, much remained to be done. There was a need to

(a) Make institutional management more systematic and transparent, for example, by providing a framework for strategic planning, revenue generation, fund allocation, and reporting that supports the institution as a whole rather than individual departments;

(b) Establish procedures and policies for staff development that recognize that most HE staff need not be fully trained as PhDs. These actions would draw from international experience, which had shown that most courses at the university level could be effectively taught by people with master’s degrees (although it was found that at the department level, supervision of these instructors is often best carried out by specialized university staff with PhDs);

(c) Bring the ACC to a level of good international practice and further integrate it into the emerging global and regional community of practitioners in quality assurance;

(d) Establish basic procedures and policies that recognize the interrelationship of teaching and research and place an emphasis on the strengthening of teaching to improve research capacity—that is, basic skills of faculty, curricula, syllabi, libraries, and connectivity; and

(e) Establish a scholarship program that balanced rewards for high-achieving students with support for the poor and incentives to study in fields of strategic importance.

7. Bank involvement in the HE sub-sector. The International Development Association (IDA) started its financial support for the HE sub-sector in 2005 through the Cambodia Education Sector Support Project (ESSP) (2005–2010). The ESSP included a small component that focused on strengthening the capacity of


of 71

the Department of Higher Education (DHE) and the ACC. In 2010, the government requested an additional US$23 million (IDA grant and credit) for the Higher Education Quality and Capacity Improvement Project (HEQCIP) to respond to the growing concerns that the rapid increase in HE enrollments had not been accompanied by improvements in educational or institutional quality. HEQCIP was designed to build a foundation of sectoral capacity, create a policy environment, and improve the quality of HEIs that allowed for increased investment in HEIs. The success of the early IDA investments has led to a government request for a new US$90 million IDA investment (plan: 2018–2023) in HE that will advance the World Bank’s previous work by focusing on research; labor market skills development particularly related to agriculture and science, technology, engineering, and mathematics (STEM); and sectoral governance reform to move HEIs toward more autonomy.

Theory of Change (Results Chain) 8. As indicated, the university system in Cambodia had been decimated during the Khmer Rouge period, and the Government of Cambodia wanted to rebuild the sub-sector. It was widely acknowledged that after years of neglect, there had been virtually no opportunities (a) to participate in training for management capacity development for HE faculty and staff, (b) for teaching faculty to obtain the advanced degrees needed to increase their discipline knowledge that could be transferred to improving the quality of their lectures and teaching, and (c) to do research—applied or basic. It was also widely acknowledged that improvements in all of these areas would be incremental and long-term. With support of HEQCIP, MoEYS began to rebuild the HE system by focusing on improving the quality of institutional teaching, research, and management. The project theory of change was based on the HE vision and strategy that MoEYS, in consultation with the ACC, had developed. The vision and strategy outlined the long-term plan for improving the HE system that would ultimately contribute to increases in a better trained workforce as well as Cambodia’s national development strategy thereby preparing them for 21st century challenges.

9. The project development objective (PDO) and supporting project activities were selected based on international best practice, which indicated that advanced degree training, establishment of a research culture through increased research capacity, and training of university management staff were all components that contribute to improving the quality of HEIs. It was also recognized that institutional accreditation was, and is, a mechanism to ensure the quality of HE (Ryan 2015) and that measuring quality outcomes, based on an externally monitored accreditation system of standards, was a basic component for quality assurance. The monitoring of the quality improvements of the project were targeted directly at the 36 HEIs that were awarded research grants, fellowships, and capacity development training. Moreover, the theory of change recognized that the retention of students, particularly disadvantaged students, was a critical component for achieving the government’s long-term goals. (See Figure 1 for project theory of change diagram.)

10. Project beneficiaries. As indicated, there were 36 HEIs that benefited from the awarding of research grants, faculty fellowships for advanced degree training, and capacity improvement training. There were 72 faculty and staff members from the DHE, ACC, Department of Scientific Research (DSR), and public and private HEIs that received fellowships for advanced degree training. Additionally, there were

(a) 45 applied research projects (32 public HEIs and 13 private HEIs) with faculty and students participating in the projects;


of 71

(b) Capacity development training for approximately 7,395 participants1 from the DHE, ACC, DSR, and public and private HEIs related to research, assessment, procurement, and financial management (FM); and

(c) An initial 1,000 disadvantaged students that received scholarships to attend domestic HEIs.

Figure 1. Theory of Change

1 The total participants are defined by persons potentially attending one or several training activities.

Component 1 Strengthening the Capacity of Higher Education System

Component 2: Provision of Competitive Development and Innovation Grants

Component 3: Provision of Scholarships to Disadvantaged Students

Component 4: Project Management and Monitoring and Evaluation

Components Activities Outputs PDO outcomes Long-term outcomes

C1 Conduct workshop to Send MoEYS/HEI leaders and staff to overseas/domestic training programs C1 Send fellowship students to overseas graduate programs C1 Send MoEYS/HEI leaders and staff to overseas/domestic training programs C1 Support ACC to build implementation capacity of accreditation

C2 Conduct a series of workshops for search skills C2 Implement pilot research funds

C3 Conduct a pilot special priority scholarship C3 Monitor support the scholarship students C3 Conduct data collection for impact evaluation

C4 Develo9pment of a vision and strategy C4 Conduct policy analysis for HE governance C4 M&E capacity development C4 Quality assurance support C4 RUPP governance reform pilot

Increase the availability of well-trained public and private sector workforce to Cambodia’s national development objectives.

C1.1 Number of HEIs shows increased aggregate score against a modified subset of ACC accreditation C1.2 Number of staff of Ministry and HEIs who complete graduate programs C1.3 Number of MoEYS/HEI staff who participated in training courses C1.4 Number of ACC staff who participated in training courses C1.5 Number of HEIs staff in participating research related

Improve quality of teaching, management and research in project supported entities

Pilot the targeting of disadvantaged students for enhanced retention.

C3.1 Number of SPS students retain in the degree course C3.2 Number of SPS students receive the scholarships C3.3 Conduct baseline and end-line data collection for impact evaluation

C2.1 Number of submitted/complete research projects

C4. 1 Vision and strategy paper/HE roadmap C4.2 HE governance paper, draft HEI autonomy policy paper completed C4.3 Results based monitoring implemented C4.3 Development of HEMIS and pilot at 5 HEIs

PDO: The development objective is to (a) improve the quality of teaching, management, and research in project supported entities; and (b) pilot the targeting of disadvantaged students for enhanced retention.


of 71

Project Development Objectives (PDOs) 11. The PDO was to (a) improve the quality of teaching, management, and research in project-supported entities; and (b) pilot the targeting of disadvantaged students for enhanced retention.

Key Expected Outcomes and Outcome Indicators 12. The key project outcome indicators were (a) 50 percent of sub-projects2 approved under the first round of proposals completed by the closing date, and 70 percent of sub-projects approved under the second round of proposals completed by the closing date; and (b) an annual retention rate of 95 percent of project-supported scholarship recipients.

Components 13. Component 1: Strengthening the Capacity of the Higher Education System (estimated cost US$7.46 million and actual costs US$10.36 million). This component focused primarily on capacity building to improve the overall development, management, and governance of the HE sub-sector. It was to improve the overall development and management of the HE sub-sector by focusing on staff development at the DHE, DSR, and ACC, and therefore improving their capacity to offer practical guidance and support to HEIs. It was to concurrently strengthen the organizational capacity of individual HEIs. A strategic staff development plan covering general training, technical skill, and leadership development was to be developed, reflecting the specific needs of each government agency and the overall strategic needs of the sub-sector. Specific activities under this component included

(a) For DHE (with participation from DSR and ACC). Workshops and an annual conference on selected HE topics, study tours and internships, short courses, postgraduate fellowships, participation at regional and international HE events, English language training, training in areas related to institutional performance, and upgrading and expanding information technology facilities and competencies;

(b) For DSR. Upgrading facilities and training events on selected research topics to strengthen existing research capacity at HEIs, improving its own capacity to monitor and support research activities, and disseminate information on the research competitions in Component 2;

(c) For ACC. Regional conferences related to academic quality and accreditation, workshops on assessor training, and visiting assessors from other countries who will offer suggestions and guidance to the ACC in key areas; and

(d) For eligible HEIs. Approximately 68 postgraduate fellowships for existing HEI faculty seeking master’s degrees, particularly in national priority areas such as agriculture, science and technology, and mathematics; participation at regional and international academic conferences; workshops related to HE administration and on research methods; and support for visiting lecturers.

2 Sub-projects were defined as those research projects that were awarded a research grant under Component 2 of the project.


of 71

14. The fellowship activities were to target selected subjects so that fellowship recipients could later become policy makers working in the government sectors related to the subject of their fellowship.

15. Component 2: Provision of Competitive Development and Innovation Grants (estimated cost US$4.58 million and actual costs US$3.66 million). Component 2 aimed to (a) strengthen the capacity of participating public and private HEIs and provide the enabling conditions for improved quality in research, teaching, and learning; and (b) introduce an efficient and sustainable mechanism, which emphasized innovation and accountability, in the DHE for the allocation of public funds for research in HEIs. A Research Grant Management Committee (RGMC) would be established in the DHE to manage the competitive grant selection and implementation process. The committee comprised representatives from relevant government agencies and HEIs. Funds were to be disbursed through two windows: the first—Support to Teaching and Learning—would be directed to improving the quality of teaching and learning, particularly with respect to academic disciplines prioritized in the draft Vision and Strategy of Higher Education in Cambodia3 (Vision 2030). Research study to be supported would include activities such as restructuring of courses or programs, modernizing/upgrading curricula, use of technology to improve pedagogy, and development of curriculum materials including electronic materials. The second window—Solutions to Local Problems—would enhance the capacity of HEIs to seek innovative solutions to address national/local development issues. Prioritized research areas would include agricultural research and extension; food science and related technology; sciences including medical, health, and nutritional sciences; pharmacy; engineering and technology; social sciences; and law. In all cases, HEIs would be encouraged to work with functional partners such as nongovernmental organizations, private sector organizations, and national/international universities.

16. Component 3: Provision of Scholarships to Disadvantaged Students (estimated cost US$5.85 million and actual costs US$4.24 million). Component 3 was to complement the government’s existing scholarship program and increase the retention of poor students in HE through the provision of 1,000 ‘special priority scholarships’ (SPSs), based on pro-poor targeting and educational criteria.4 The component financed (a) tuition and stipends to beneficiary students, (b) guidance and awareness-raising activities related to the program, (c) monitoring and evaluation (M&E) activities at the project unit and participating HEIs, and (d) a rigorous impact evaluation that assessed the scholarships’ effectiveness and outcomes. The monitoring indicators associated with this component were (a) the number of eligible poor students receiving the scholarships; (b) the number of females, ethnic minorities, and students from rural areas in the scholarship program; and (c) retention and completion rates for these scholarship students.

17. Component 4: Project Management and Monitoring and Evaluation (estimated cost US$5.11 million and actual costs US$4.59 million). Component 4 supported a project management structure that was based on a system that already existed. The minister of education had already established a Program Management Committee (PMC) responsible for the oversight of the ESP to streamline policy 3 The Vision and Strategy of Higher Education in Cambodia is an official document describing the long-term vision and strategy of HE development of the country. 4 The scholarship program was a pilot to provide tuition and stipends to approximately 1,000 poor students who met both educational as well as pro-poor selection criteria. The ‘special-priority’ scholarship program was implemented by the DHE within the existing framework of the MoEYS national scholarship scheme, which only provided tuition waivers and suffered from low retention rates. The application processes and schedules for the pilot scholarship program essentially followed the regular scholarship scheme; however, the application form added an additional set of socioeconomic criteria to facilitate the improved selection of students from poor households, based on Proxy Means Testing methodology. The selection of beneficiary students for the pilot scholarship program coincided with that for the regular existing scholarship program. The pilot funded only one cohort for the entire four-year university cycle provided the recipient students remained academically eligible.


of 71

development, strategic planning, and decision-making processes. The PMC also had the mandate to provide oversight responsibility of all donor-financed projects. Given the existing structure, under the project, the PMC was responsible for (a) approving annual activities and budget allocations of the project and (b) overseeing progress and compliance with agreed guidelines. The Secretary of State responsible for the Directorate General of HE had the overall responsibility for management and acted as the key point of liaison with the World Bank. Implementation arrangements were based on the existing MoEYS structure with detailed project responsibilities clarified from project inception. The Project Management Team led by the Secretary of State responsible for the Directorate General for HE would have the primary responsibility for sub-sector coordination between MoEYS, ACC, and HEIs and overall management; the Project Coordination Committee (PCC) within the DHE was responsible for the daily management, implementation, administration, project coordination, and M&E of the project. The PCC led by the Deputy Director General for HE responsible for the DHE assumed the role of Project Manager.

B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE)

Revised PDOs and Outcome Targets 18. The PDO was not revised during the project period.

Revised PDO Indicators 19. On October 15, 2014, the key indicators were modified in a Level 2 restructuring. The indicator modifications were made following the midterm review (MTR) because the ACC had been able to establish standards that could provide a more in-depth measure of quality related to teaching, management, and research. During the December 8, 2015 Level 2 restructuring, the team reintroduced the PDO indicator to measure improved quality of research that had been mistakenly removed during the October 2014 restructuring. Although the indicator had been mistakenly removed, data related to the indicator were monitored and being reported to the World Bank. The modifications were as follows:

(a) PDO indicator 1 was added and was ‘40 percent of HEIs supported under Components 1 and 2 showing an increased aggregate score against a modified subset of ACC accreditation criteria’,5 introduced in 2014.

5 The ACC accreditation criteria used to measure PDO 1 are as follows:

Standard 1: Governance, Planning, and Evaluation—(i) HEI results-based plan developed and submitted to MoEYS; (ii) HEI annual operations plan develop and submitted to MoEYS; and (iii) HEI M&E framework developed and submitted to MoEYS.

Standard 2: Academic Program—(i) committee for curriculum development established, implementation plan developed, funds allocated, and progress report submitted to MoEYS; (ii) each HEI has a handbook on curriculum publicly available; (iii) research promotion policy with allocation of HR, financial resources, and facilities to support research developed; (iv) IQA function established, physical office set up with implementation plan developed, funds allocated, and progress report submitted to MoEYS; and (v) IQA framework with standard guidelines and key performance indicators developed and introduced within HEI.

Standard 3: Academic and Administrative Staff—(i) HEI capacity development plan created and funds allocated; and (ii) HEI policy on incentives for capacity building developed.

Standard 4: Financial Management—(i) strategic financial plan in place; and (ii) annual financial report reflecting the annual operational plan in place.

Standard 5: Dissemination of Information—(i) information disseminated publicly and the information shared publicly; and HEMIS established.


of 71

(b) PDO indicator 2 was modified to ‘an average annual retention rate of 95 percent of project supported scholarship recipients’ rather than just the annual retention rate. The newly stated indicator would then measure the average retention rate over the five-year project period, introduced in 2014.

(c) PDO indicator 3 was reintroduced in 2015 and was stated as ‘50 percent of sub-projects approved under the first round of proposals completed by the closing date and 70 percent of sub-projects approved under the second round completed by the closing date’.

Revised Components 20. The project components were not revised. However, at the time of the December 2015 restructuring, an additional activity was introduced under Component 1 to support the government’s request to work with the Royal University of Phnom Penh (RUPP) to create a model HEI that exemplified good practice in governance and financial and human resource (HR) management. Related to this was the support of international technical assistance to provide analytical support on FM and HR management and to develop an FM Manual and HR Manual. This was also the time when MoEYS initiated its request for the project to provide support for STEM-related disciplines through the inclusion of an additional 15 fellowships specifically focused on STEM subjects.

Other Changes 21. In March 2011, there was a Level 2 restructuring to modify disbursement categories to accommodate a new government policy toward the merit-based performance incentive scheme in the project design. In January 2010, the government cancelled all salary supplement and incentive schemes regardless of the source of financing and replaced it with the establishment of the Priority Operating Costs (POC) scheme.6 Based on this, the restructuring included a POC category in the original budget and allocated SDR 107,200 equal to 100 percent of the scheme from the unallocated expenditure category.

22. In October 2014, there was a Level 2 restructuring that (a) modified the PDO indicators (as described above) as well as some project intermediate indicators and (b) reallocated project funds. The intermediate indicators were modified as follows (see annex 7 for a complete table on indicator revisions):

(a) Reduced and streamlined Component 1 indicators from four to two indicators;

(b) Measured both absolute numbers and percentages related to Component 2 indicators to provide clearer information and eliminated Year 4 and Year 5 targets for intermediate Indicator 4 (‘Proportion of proposals reviewed, evaluated and awarded’);

(c) Under Component 3, dropped one intermediate indicator; and

6 The POC was a new overarching regulatory framework for salary supplements and incentive schemes that had been agreed between Cambodia and the development partners. The POC outlined meritocratic equity and transparency principles that applied to the business with all development partners.


of 71

(d) Revised both Component 4 indicators and added one new indicator to be more in line with the component’s activities.

23. Reallocation. Project proceeds were reallocated as follows: (a) IDA Credit: SDR 722,800 from the Category 5 ‘Unallocated’ and SDR 17,200 from Category 6 ‘POC’7 to Category 1 ‘Goods, Works, Consultant Services, Training and Incremental Operating Costs’ and (b) IDA Grant: SDR 900,000 from Category 3 ‘Scholarships’ to Category 2 ‘Fellowships’. The reallocation financed additional fellowships as well as additional capacity-building activities in responding to higher unit costs of postgraduate programs (all fellowship programs are organized with universities in Australia and New Zealand) as well as higher demands of training opportunities from HEIs.

24. On December 8, 2015, the project was restructured to

(a) Reintroduce the PDO indicator that had been mistakenly removed as indicated above in paragraph 19;

(b) Extend the project closing date by 21 months from December 31, 2015, to September 30, 2017; and

(c) Assist RUPP and the government in building a model HEI as indicated in paragraph 20.

Rationale for Changes and Their Implication on the Original Theory of Change 25. The modifications did not affect the original theory of change but rather enhanced the ability to measure progress toward achievement of the theory of change. HEQCIP was the first World Bank-supported HE project in Cambodia. When the project was designed, the data available through the ACC and the education information systems were limited. At the time of the MTR, the new ACC standards had been developed and approved and provided an opportunity for a more robust system for measuring the quality of HEIs. Moreover, accreditation ensured that the university was continuously reviewed and met nationally endorsed standards. Based on this, the PDO indicators were enhanced with the inclusion of ACC standards that could measure the quality aspects of the PDO with more rigor. The extension of the closing date and inclusion of RUPP allowed MoEYS to (a) provide 15 additional fellowships specifically focusing on STEM that supported the overall long-term goals of the results chain as well as the government’s newly outlined Industrial Development Policy (IDP) 2015–2025 goals and (b) further improve governance and management by looking more deeply into the financial and HR management of an HEI with an eye toward increased HEI autonomy. These are all important components of improving quality as outlined by ACC standards.

7 In March 2011, the project introduced the POC scheme in accordance with the government’s Sub-Decree No. 66 on the establishment and implementation of the POC within the framework of development cooperation. In mid-2012, this was discontinued because it was viewed as a transitional scheme eventually linking to the broader civil service reform under way at the time.


of 71

II. OUTCOME

A. RELEVANCE OF PDOs

Assessment of Relevance of PDOs and Rating 26. The PDO was highly relevant to the country’s sectoral needs when the project was developed. The objectives fit into the government’s ESP 2006–2010 that focused on the promotion of the quality of education and equity in educational opportunities to increase income and employment. It also fit into the Country Assistance Strategy (CAS) updated in May 2008 that focused on building strong foundations for sustainable development and poverty reduction by investing in good governance, physical infrastructure, human capital, and the private sector. Moreover, the CAS explicitly indicated that one of its major objectives was the development of HE capacity to a level where it could effectively train professionals for public service and the private sector. To meet this objective, there was a need to strengthen HEIs through the development of quality standards, organizational improvements, staff training, and building the planning and support skills of ministry staff. The project objectives continue to fit into the World Bank’s Country Partnership Framework, which has HE as one of the three human development pillars as well as the Cambodia Country Engagement Note. The project also continues to fit directly into the government’s IDP 2015–2015 that specifically calls for increasing the human capital for diversifying the Cambodian economy and moving the country toward a more ‘technology-driven’ and knowledge-based modern industrial country by 2025.

27. The design focused on quality through the improvement of teaching and research and equity by targeting disadvantaged students with the inclusion of targeted scholarships. The PDO was focused and highly appropriate given the early stage of the sub-sector’s development and the need to improve teaching, research, and equitable access. The original Results Framework was developed to measure both quality and equity using measurable indicators available at the time of project design. The revisions in the restructured Results Framework, following the MTR, provided the opportunity for a more nuanced measure of quality based on the improved ACC criteria. This was a highly relevant modification. The theory of change behind the project was sound and the selection of key targets was appropriate. The objective is outcome-oriented and was appropriately pitched for the government capacity and development status of Cambodia.

28. The government also used the results focus to justify its increasing financial commitment to HE research and to shape its own scholarship program so that it targets resources to disadvantaged areas that need additional support to ensure equitable access to HE and to meet the country’s 21st century demands for a technology-driven knowledge-based economy.

B. ACHIEVEMENT OF PDOs (EFFICACY)

Assessment of Achievement of Each Objective/Outcome 29. The efficacy of the PDOs is rated Substantial for the prerestructuring period and Substantial for the postrestructuring period. HEQCIP had the objective of improving the quality of teaching, management, and research in project-supported entities and piloting the targeting of disadvantaged students for enhanced retention that was to be measured using two original PDO indicators and one


of 71

added PDO indicator during restructuring. All PDO indicators were met or exceeded. The 12 intermediate indicators were also all met or exceeded.

Prerestructuring of October 2014

30. PDO Indicator: 50 percent of sub-projects approved under the first round of proposals completed by the closing date, and 70 percent of sub-projects approved under second round completed by the closing date. As indicated, this indicator was mistakenly removed during the October 2014 restructuring. The original end target was for 75 percent of all the sub-projects to be completed by project closing. When the indicator was reintroduced during the December 2015 restructuring, the end target was for 70 percent of all sub-projects to be completed by project closing. There were 45 applied research projects,8 and 100 percent of all the projects in the first and second rounds were completed thereby exceeding the indicated targets. The sub-project grants were given to both public and private HEIs—32 (71 percent) and 13 (27 percent), respectively, and the distribution of grants for the two funding windows was as follows:

Table 1. Sub-project Grants Distribution

Window Number of Projects Percentage Window 1 - Support to Teaching and Learning 25 56 Window 2 - Solutions to Local Problems 20 44 45 100

31. Moreover, the grants were awarded through a competitive process and were given to HEIs under several line ministries responsible for the HEIs as follows: 73 percent to MOEYS, 16 percent to the Ministry of Agriculture, 7 percent to Ministry of Health, and 4 percent to the Ministry of Arts. It should also be noted that 31 percent of the grants were awarded for curriculum development that would directly contribute to improving the quality of teaching in the classroom, 40 percent for experimental research, and 29 percent for pure research. (See annex 8 for a list of the research projects approved through the competitive process.) While these projects were largely applied research projects, these are all considerable achievements given the fact that there was virtually no research culture in Cambodia before HEQCIP. In addition, there were five articles published in peer-reviewed national research journals based on the project-supported research projects, which is an indication of the quality of the research conducted.9 (See annex 8).

32. PDO Indicator: Annual retention rate of 95 percent of project supported scholarship recipients (indicator as stated pre-2014 restructuring); an average annual retention rate of 95 percent of project-supported scholarship recipients (indicator as stated post-2014 restructuring). The modification in the wording was to show the average retention rate over the five-year project period (rather than just yearly) to measure the impact of the scholarship program over five years. By project closing, the annual average retention rate was 96 percent thereby exceeding the target of 95 percent. As indicated, the project also

8 Disciplines of the research projects funded were agriculture, health environment, multidisciplinary, business management, social sciences, language, education, and physical sciences. 9 In addition to the articles published, researchers from RUPP attended an international conference at Nagoya University: Forum of International Development Studies, No. 46 (2016.03) where the results of their research was presented—“The Learning and Sharing of Rice Growing Knowledge among Model Farmers Case Study: Rice Farmers in Kirivong District, Takeo Province, Cambodia.”


of 71

supported an impact evaluation of the SPS program to gain additional insight into the HEQCIP pilot program. The results from supporting 1,000 students with additional stipend support showed that enrolled students continued their study into the second year by 13.7 percentage points.10 Moreover, the impact of the financial aid on retention rate became higher with each additional year of enrollment, and by the fourth year, the difference in retention rate between the project-supported students and the control group with tuition waivers was approximately 34 percentage points. It also showed that students without the stipend were more likely to drop out if they fail to advance to the next academic year. The impact evaluation showed that there was a strongly positive and significant impact on the on-time completion11 of the project-supported scholarship students. Overall, this was an impressive achievement. The full impact evaluation report has been attached for review.

Postrestructuring of October 2014

33. PDO Indicator: 40 percent of HEIs supported under Components 1 and 2 showing an increased aggregate score against a modified subset of ACC accreditation criteria. (See footnote 5 for the subset of the 13 ACC criteria used in measuring achievement within the 36 project-supported HEIs.) As indicated, this indicator was added at the time of the 2014 restructuring to provide an additional measure for the quality of teaching, management, and research. It was also an important indicator to add because studies suggest that external quality audits through an accreditation process, together with internal university processes, have stimulated change and improvement of teaching and learning in universities (Greatbatch and Holland 2016). At the time of project closing, 46 percent of the 36 HEIs being measured had shown an increased aggregate score against the subset of accreditation standards surpassing the target of 40 percent by 6 percent. This is an impressive achievement given the low base from which all of the HEIs started and the fact that the accreditation process using ACC standards only began implementation in 2014. There were two substandards12 that all 36 HEIs had achieved when the subset was decided. If these two are removed from the mix, the achievement is greater at 53.8 percent. It should also be noted that the progress for the HEIs on each one of the 13 substandards has been Substantial. (See annex 8 for complete details.)

34. All 12 Intermediate Indicators were met:

(a) Number of staff (MoEYS and HEI) who participate in training courses which improved their professional competencies; fields related to: HE quality, management, assessment, and accreditation. During the project period, for all components, there were a total 841 training programs13 (83 overseas and 758 local) with a total of 7,395 staff. A total of 2,224 of the beneficiaries came from the MoEYS departments (DSR, DHE, and ACC,) and 5,171 came from public and private HEIs. These achievements far surpassed the original target of 1,500 (500

10 Most HEIs in Cambodia require students to repeat a year if they cannot pass the required credits or subjects. In the impact study, repeaters were also treated as retained as long as they did not drop out from their programs. 11 On-time completion in the study referred to those students who were promoted to the next academic year for four years consecutively without having to repeat any year. 12 The two ACC subset indicators were (a) each HEI has a handbook on curriculum publicly available and (b) information disseminated publicly and the information shared publicly. 13 The training programs include local and overseas training workshops, seminars, conferences, study tours, assessment visits, follow-up visits, monitoring visits, and retreats.


of 71

MoEYS and 1,000 HEIs) because of higher demands of training and increased budget allocation based on the 2014 project restructuring.

(b) Number of staff (MoEYS and HEI) who were conferred with a Master or Doctor degree in academic specialty, educational management or related fields. The total number of individuals that received masters or doctoral degrees was 79, exceeding the target of 54 for an increase of approximately 46 percent. There were 64 students that went to universities in Australia,14 and of the 64, 61 received master’s degrees and 3 received PhDs of which 22 percent were female. These students graduated and returned to Cambodia with degrees in the fields of economics, education, development studies, commerce, public administration, and environmental management. In 2016, 15 students went to Malaysia on STEM-related fellowships to study for a one-year master’s degree program. They all graduated and returned to Cambodia with degrees related to computer science, building technology, applied geophysics, health toxicology, and electronic system design.

(c) Number and percentage of HEI academic staff in participating institutions completing training in research methodologies and design. There were 579 HEI academic staff that participated in research methodologies and design training, surpassing the target of 180 by 221 percent because of higher demands of academic training.

(d) Number and percentage of eligible HEIs submitting proposals to RGMC competing for funds. During the life of the project, there were two rounds of funding available for research grants. In 2012, 22 out of 30 eligible HEIs submitted proposals, approximately 73 percent against an overall target of 70 percent. In 2013, 26 out of 35 eligible HEIs submitted proposals for consideration, 72 percent against an overall target of 70 percent. Each year the target was exceeded.

(e) Number and percentage of complete research projects (competitively selected) funded by the project. There were 45 research proposals funded by the project, and 100 percent of the projects were completed during the project period thereby meeting the target.

(f) Within RGMC/DHE, number and percentage of proposals awarded after review and evaluation in accordance with project processing timeline. There were 45 out of 62 research project proposals submitted to the RGMC (approximately 72 percent) that were awarded a grant after review and evaluation in accordance with the project timeline. This exceeds the target of 47 percent.

(g) Pass rate of retained scholarship recipients at each grade categorized by: female, rural, poor (these are annual pass rates for retained recipients, not cumulative rate). For each year, the target was 80 percent, which was surpassed each year.

14 Universities in Australia were: Charles Darwin, Melbourne, James Cook, Canberra, Sydney, New South Wales, and Flinders.


of 71

Table 2. Annual Pass Rates for Retained Recipients

Grade/Year 2013 (%) 2014 (%) 2015 (%) 2016 (%) 2017 (%) Female 91 95 98 97 100 Poor 92 96 98 96 100 Rural 91 98 99 97 100

(h) Dropout rate of scholarship recipients categorized by female, rural, poor (the dropout

rates are cumulative dropouts for recipients). The end target for each category was 19 percent. In each category, the dropout rates were considerably less than the target. As indicated above, and based on the impact evaluation, the SPS played an important role in ensuring the scholarship recipients did not drop out of their academic programs.

Table 3. Cumulative Dropout Rates for Recipients

Year 2013 (%) 2014 (%) 2015 (%) 2016 (%) 2017 (%) Female 7 10 12 16 16 Poor 7 10 11 16 16 Rural 9 11 12 15 15

(i) Completion rate of scholarship recipients, categorized by female, rural, poor. The end

target for each category was 67 percent. In each category, the target was exceeded.

Table 4. Completion Rates for Recipients

Year 2016 (%) 2017 (%) Female 83 85 Poor 83 84 Rural 85 86

(j) Higher Education Governance & Finance policy paper adopted by the MoEYS. The DHE,

with support from the project, developed a Higher Education Governance and Finance policy paper that was adopted by MoEYS in August of 2017, thereby meeting the target. This paper, along with the Higher Education Road Map (developed with project support) and the ‘Vision 2030’ paper (see below) have contributed to the sectoral governance framework to improve the HE system.

(k) Higher Education Vision and Strategy 2030 adopted by the MoEYS. The target for this indicator was met. The Higher Education Vision and Strategy 2030 was completed in 2013 and was adopted in 2014 by MoEYS. The ‘Vision 2030’ has been used by MoEYS to build on the HEQCIP achievements and embark upon an eight-point education reform agenda to address (i) concerns over lack of relevant skills and knowledge of HE graduates; (ii) insufficient capacity, skills, and facilities for research; (iii) a mismatch between HE provision and the needs of the labor market; (iv) an oversupply of graduates in management and skills shortage in STEM; and (v) improved governance mechanisms at the HEI level.

(l) HEMIS developed and piloted with at least 4 HEIs. During the project period, the Higher Education Management Information System (HEMIS) was developed and piloted in 6 HEIs thereby surpassing the target. In addition to the 6 HEIs utilizing the HEMIS, there were 24


of 71

HEIs (including provincial branches) that uploaded their enrollment data into the HEMIS. MoEYS will now work to roll out the HEMIS and make it the standard system across the sector.

35. Contribution to higher level objectives. The medium-term objective of the project was to increase the availability of well-trained public and private sector workforce to support Cambodia’s national development objectives of creating a competitive economy through knowledge and innovation. An important contributing outcome of the project was the Higher Education Road Map that provided MoEYS with a clear strategy for HE that connects directly to the government’s new IDP 2015–2025 that aims to diversify the economy and move the country toward a more ‘technology-driven and knowledge-based modern industry’ as well as the project-supported Vision 2030. The road map is a holistic strategy that aims to equip Cambodian graduates with the skills to meet the rapidly changing economic demands. The road map focuses on three key policy areas: (a) quality and relevance, (b) access and equity, and (c) governance. The project support for 79 master’s degree and PhD students, all of whom returned to Cambodia with degrees; the 844 disadvantaged students that had completed their study programs by August 2017; and the large number of educators, staff, and researchers that received training will have a considerable impact on the availability of a well-trained workforce.

Justification of Overall Efficacy Rating 36. The overall efficacy rating is Substantial. The rating prerestructuring is Substantial and is justified because the two original PDO indicators were on track to being met and were being measured based on progress within the overall project implementation. Activities related to PDO 1 were being implemented. Once the research teams received training on procurement and FM, they began to implement the research projects with the end result of achieving the PDO indicator. Moreover, these activities improved curricula and introduced new research findings into the classroom thereby improving the quality and relevance of teaching as well as the courses being taught. Component 1 training activities focused on improving management through the 841 training activities, overseas and local, that gave administrators, researchers, and academics the opportunity to learn about research methodologies, curriculum renewal, university management, HE system management, accreditation, and so on. PDO 2 was also being measured based on implementation progress. By project closing, PDO 2 had been achieved and as the SPS impact evaluation showed, the project’s SPS program had a direct impact on the retention of the treatment group with direct attribution to the project. Efficacy for the postrestructuring period is judged to be Substantial. This is justified because by project closing, all PDO indicators and intermediate indicators were met or exceeded. The inclusion of the third PDO indicator (40 percent of HEIs supported under Components 1 and 2 showing an increased aggregate score against a modified subset of ACC accreditation criteria) provided an opportunity to better assess quality improvements within project-supported universities by using the ACC accreditation criteria.

C. EFFICIENCY

Assessment of Efficiency and Rating 37. The overall rating for efficiency is Substantial. The economic benefit for HEQCIP is evaluated by aggregating the net benefits arising from three project components. Under Component 1, the capacity of the HE system has been enhanced through improving the overall development, management, and governance of the HE sub-sector. Concurrently, the organizational capacity of individual HEIs was


of 71

strengthened through a strategic staff development plan covering general training, technical skill, and leadership development. Component 2, on the other hand, has contributed to (a) strengthening the capacity of participating public and private HEIs and providing enabling conditions for improved quality in research, teaching, and learning and (b) introducing an efficient and sustainable mechanism, which emphasizes innovation and accountability, in the DHE for the allocation of public funds for research in HEIs. Finally, the benefit from Component 3—a ‘special-priority’ scholarship pilot program to provide tuition and stipends to 1,000 poor students who met both educational as well as pro-poor selection criteria—is evaluated. From an impact evaluation study carried out, it is estimated that the dropout rate for this group of disadvantaged students has declined significantly from 48.8 percent to 15.5 percent as a result of the intervention.

38. The beneficiary students in the participating HEIs benefit from these interventions through improved teaching and learning environment, better research capacity of the universities, and reduction in dropout. As a result, these graduates would possess a higher level of skills and enjoy higher earnings streams throughout their working lives. Regarding the cost side, actual total project costs incurred over five years, amounting to US$22.85 million, were taken into account in the cost-benefit analysis (CBA). Also taken into account were both the additional private and public recurrent cost incurred for those who would have otherwise dropped out without the interventions.

39. Table 5 presents the net present values (NPVs) and internal rates of return (IRRs) of the project, analyzed through the quantity and quality perspectives as discussed above. In the baseline scenario, the resulting NPV is estimated at US$283.1 million, using a real cost of capital (or real discount rate) of 5 percent, with an IRR of 22 percent. Two other scenarios are also evaluated in the sensitivity analysis. In the ‘Low’ scenario, the NPV declines to US$15.3 million, with an IRR of 1.5 percent. Finally, in the ‘High’ scenario, the NPV reaches US$587 million, with an IRR of 29.6 percent. The results from the economic analysis, therefore, suggest that the project yielded significant net economic benefits. More detailed analysis is provided in annex 4.

Table 5. CBA of HEQCIP

Baseline Low High US$ US$ US$

Present Value Benefits - HE Quality Enhancement 298,346,625 0 602,282,285 Present Value Benefits - Fellowship + Development and Innovation Grant

8,249,857 8,249,857 8,249,857

Present Value Benefits - Scholarships to Disadvantaged Student

3,029,202 3,029,202 3,029,202

Present Value Recurrent Cost - Fellowship + Development and Innovation Grant

2,700,137 2,700,137 2,700,137

Present Value Recurrent Cost - Scholarships to Disadvantaged Student

991,443 991,443 991,443

Total project spending by component: Capacity Building 10,366,293 10,366,293 10,366,293 Development and Innovation Grants 3,663,724 3,663,724 3,663,724 Scholarships to Disadvantaged Student 4,238,060 4,238,060 4,238,060 Program Management 4,581,836 4,581,836 4,581,836

Total project spending 22,849,913 22,849,913 22,849,913


of 71


NPV (5% discount rate) 283,084,191 (15,262,434) 587,019,851 IRR 22.0% 1.5% 29.6%

D. JUSTIFICATION OF OVERALL OUTCOME RATING

40. The overall outcome is Satisfactory based on High relevance, Substantial efficacy, and Substantial efficiency.

Table 6. Overall Outcome Rating

Rating Dimension Original Objective Objectives after 2014 Revisions Relevance of Objectives High Efficacy Objective 1: Improve the quality of teaching, management, and research in project-supported entities

Substantial Substantial

Objective 2: Pilot the targeting of disadvantaged students for enhanced retention

Substantial Substantial

Overall Efficacy Substantial Efficiency Substantial Outcome Rating Satisfactory Satisfactory Outcome Rating Value 5 5 Amount Disbursed (US$, millions) 15.12 7.73 Disbursement (%) 66.17 33.83 Total Weights 5.0 Overall Outcome Rating Satisfactory

E. OTHER OUTCOMES AND IMPACTS (IF ANY)

Gender

41. HEQCIP was a gender-informed project. The project indicators and outputs, where appropriate, were all gender-disaggregated thereby ensuring that the DHE and MoEYS focused attention on improving gender equity throughout the project. The female participation rate for Component 1 training activities was approximately 17 percent. The participation rate of women in the first group of overseas fellowships to Australia was 22 percent, and for the second group that went to Malaysia, it was 47 percent. This shows a positive trend toward gender equity. Women accounted for 44 percent of the 1,000 SPSs awarded. The SPS impact evaluation also showed that the female students had a higher chance of remaining in school and completing the program on time. The impact on retention by the fourth year was 4.8 percentage points higher than male students and 6.0 percentage points for completion. The new World Bank-supported Higher Education Improvement Project (HEIP) (P162971) being developed will continue the focus on increasing female participation through the construction of student dormitories for female and disadvantaged students, promotion of female faculty participation in research projects, and a tracer study that tracks employability with gender-disaggregated analysis.


of 71

Institutional Strengthening 42. The ACC was strengthened, and ACC standards were institutionalized during the project period. This was evidenced by the fact that at the time of the project midterm, the ACC standards PDO indicator was added so as to include a more robust measure of the quality of teaching, management, and research at project-supported HEIs. By project closing, the ACC had used the revised ACC standards instrument to conduct the pilot of full accreditation evaluations for 12 out of 36 target HEIs. In 2017, the ACC conducted an additional five evaluations. Moreover, MoEYS and the World Bank have agreed to continue strengthening the ACC in the HEIP to continue improvement in the overall quality of HEIs through the accreditation process. The competitive research proposal process has also been institutionalized. In 2015, the government provided US$1,000,000 equivalent to support the competitive research process. In addition, funding for the new HEIP project will be awarded through a competitive process that uses a research grant committee/council—a process that was first established through HEQCIP. The students that received master’s and PhD degrees have developed long-term professional and institutional relationships with the Australian universities that will lead to the Cambodian HEIs being a part of the wider international academic network. This is evidenced by HEIs having (a) signed memoranda of understanding (MOUs) with Australian universities that will support continued capacity development in teaching and research, (b) been awarded 15 Australian fellows for a seven-week program to strengthen quality assurance in HE curricula, and (c) presented proposals to Charles Darwin University (CDU) in Australia to establish a joint postgraduate program with RUPP, Cambodia Development Research Institute (CDRI), and the DHE to help develop a STEM curriculum teacher education program. Finally, MoEYS has taken the impressive results from the SPS impact evaluation into consideration as it restructures its national K-12 scholarship program and begins to formalize the inclusion of HE into the system, which includes stipends and tuition waivers. Overall, these are extremely important institutional achievements.

Mobilizing Private Sector Financing 43. Not applicable.

Poverty Reduction and Shared Prosperity 44. As indicated, the SPS program was a pilot to provide tuition and stipends to 1,000 students, which was merit- and needs-based. For the merit criteria, the project looked at examination data. For the needs-based criteria, the project used a World Bank-developed poverty questionnaire15 to assess the level of applicant’s poverty. Based on all of these criteria, the poorest 1,000 students eligible for the SPS were selected. The results of the impact evaluation showed that the SPS program had the strongest effects among students from remotes areas. The results showed that the SPS impact on enrollment among students of remote areas was 20.3 percentage points compared to an impact of 18.1 percentage points and 11.2 percentage points for students from rural and urban areas that did not participate in the SPS, respectively. The results also suggested the impact among the remote group on retention and completion was as high as 49.7 percentage points. The long-term benefits of this are definitely related to the project

15 The questionnaire was based on an analysis that was done for the World Bank-supported Cambodia Socio-Economic Survey 2009 so as to be able to better assess the poverty levels of students. There were 29 additional poverty-related questions added to the MoEYS application that the World Bank team used to assess poverty levels and to make the final selection of the 1,000 project-supported scholarship recipients. Examples are number of parents alive, employment of father and mother, household size, number of children under 14 in the family, availability and source of drinking water, and so on.


of 71

results chain as well as poverty reduction efforts. Moreover, the results are directly attributable to the project intervention.

Other Unintended Outcomes and Impacts 45. The project supported the completion of the Vision 2030 and the HE road map and developed the HEMIS. All three accomplishments fit within the internationally accepted criteria for good governance, which include (a) ensuring there is a strategy-setting guide, (b) supportive legislation, (c) development of a quality assurance mechanism and framework, and (d) an information system used for policy planning and evaluation. The impact of these outputs has been their complete institutionalization as guides for policy reform and guidance in the HE sector. The road map took the Vision 2030 and provided a detailed framework for implementing the HE policy, which MoEYS has fully adopted. Moreover, MoEYS is using the two documents along with the findings from the RUPP pilot to conduct a legislative review of their current HEI policies, particularly as they related to financial and HR management, to develop legislation on increased university autonomy. The inclusion of support for RUPP during the 2014 restructuring contributed to the knowledge base needed to move the university autonomy reforms forward. In addition, the HEQCIP work on university autonomy led MoEYS to further discussions with the World Bank on support for HE and ultimately to preparation of the second World Bank-supported HE project. The project research activities also led to the establishment of the Cambodia Society for Research and Development (CSRD)16 and the Cambodia Citation Index (CCI)17 that will sustain the culture of research support and collaboration among HEIs. The project work led to the establishment of a HE working group that functions as a small discussion group composed of staff from MoEYS, Supreme National Economic Council selected HEIs, and development partners. This is a substantial sector achievement.

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME

A. KEY FACTORS DURING PREPARATION 46. HEQCIP was envisioned to support quality reforms within the HE sub-sector. The project built off the modest funding of US$3 million that had been provided to the HE sub-sector through the Cambodia ESSP, which focused on improving facilities, equipment, and staff capacity. The government’s interest in World Bank involvement in HE was the fact that HEQCIP would focus on improving the quality of the system, increasing the availability of well-trained public and private sector manpower as well as piloting a new approach to decreasing poverty levels through the SPS. The design and preparation activities provided a sound foundation for the project. In addition, the HEQCIP design was focused on the priorities of the World Bank as outlined in the CAS that was updated in May 2008 that focused on building strong foundations for sustainable development and poverty reduction by investing in good governance and human capital.

16 The CSRD has become a key forum/network for research faculty and staff of all HEIs to connect on common research interest and to share research findings. 17 The ASEAN Citation Index (ACI) is a regional data base set up to index all bibliographic records and citations of all quality ASEAN research outputs appearing in scholarly journals from ASEAN member countries. Each ASEAN member state is required to set up their own national citation index. In the case of Cambodia, it is called the CCI. The CCI team reviews and recommends quality journals or journal articles to be indexed to the ACI. Indexing of quality journals or journal articles by Cambodian researchers to the ACI increases visibility of Cambodian research and helps to build capacity of researchers. In 2018 the Cambodian Journal of Natural History by RUPP has become the first Cambodian journal to be indexed into the ACI.


of 71

47. Lending instrument. The Specific Investment Loan (SIL), rather than Development Policy Lending, was appropriate when the project was developed given that strengthening central and institutional staff capacity to understand and implement policy was a higher priority than further policy reform. The SIL allowed for provision of the needed technical assistance and institutional strengthening schemes necessary to build the capacity to support policy implementation.

48. Evidence-based design. The technical design of the project drew from globally recognized approaches for developing effective HE interventions that are aligned with international best practice. These design elements were well-suited to institutions participating in the project.

49. Appropriate selection of beneficiary groups to target. The targeted project beneficiaries for capacity development were project-supported HEI education managers, researchers, accreditation staff, and academics. In addition, disadvantaged students, selected based on merit and poverty criteria, were targeted. The targeting criteria ensured that the poorest students were selected for the SPS program.

50. Appropriate plan for monitoring. Through the Results Framework, improvements in the HEMIS, and development of the accreditation standards, the DHE was able to measure progress toward achievement of the PDO. The inclusion of the SPS impact evaluation was highly appropriate and provided the ability to further assess the overall impact of the project interventions.

51. Adequacy of risk and mitigation measures identification. Key risks and appropriate mitigation measures were identified during preparation. Challenges in the form of capacity, governance, and implementation structure were anticipated. This was particularly true for the implementation structure given that the DHE was responsible for day-to-day project management but had no actual direct line of authority or responsibility over the HEIs. The project activities were directly related to risk mitigation through the (a) inclusion of capacity development training within MoEYS and HEIs, (b) inclusion of a strong poverty targeting mechanism for the SPS, and (c) inclusion of the RGMC to ensure transparency in the selection of research proposals.

52. Sustainability. During project design, there were already discussions about mainstreaming the competitive innovation and development grants and the SPS into the MoEYS operation after the project closed. At this time, the SPS was already to be included into the ESP as part of its emphasis on access and equity, and the project’s Component 3 design fed into the implementation of the MoEYS vision for their program.

B. KEY FACTORS DURING IMPLEMENTATION 53. There were various factors that played a role in the challenges and successes of the project during implementation.

54. Capacity building. The capacity-building activities began more slowly than initially anticipated because a needs assessment needed to be conducted. The project also had difficulties in finding suitably qualified and experienced trainers to support the activities given that the procurement procedures made it difficult to hire local consultants because many qualified candidates were public servants and procedures for overseas professions were more suited for competitive selection for longer-term commitments. In 2014, a system of recruiting key resource persons was introduced for the project that followed a quick procedure for recruiting both local and international short-term trainers. During this


of 71

period, the capacity development activities increased from 34 in 2010–2013 to 114 for 2014–2015, a 235 percent increase. The process was very successful in enabling the project to benefit from using local PhDs who understood the academic context in which the participants were working and were able to share their skills and knowledge directly in Khmer without using translation from English. However, this process was replaced again in 2015 when judged to be out-of-line with standard operating procedures. Once replaced, an additional 73 activities were completed.

55. Fellowships began slowly due to the need to make appropriate overseas university connections for fellowship participants due to limited knowledge/connections of international programs within HEIs as well as among students. There were also challenges related to (a) obtaining visas, (b) school admissions processes were complicated and time-consuming for the Cambodian students, and (c) demand for a high level of English proficiency. The project hired an international consultant to assist the DHE in making the needed connections with universities in Australia, supporting students with the application process and working with the DHE on ensuring students received the needed language courses.

56. The ACC activities were slowed due to exogenous factors. As the project began, the ACC guidelines were approved and assessments were to begin. However, in 2013, the legal status of ACC was transferred from the Council of Ministers to MoEYS. During the process of transfer, ACC was without a definitive status and the accreditation process stalled. By the end of 2014, the transfer had been concluded. A set of revised ACC standards were officially approved and then used for the new PDO indicator introduced after the MTR. Once officially approved, there was an additional consultative process, which took place within the selected HEIs of RUPP, University of Health Science, Pannasatra University of Cambodia (PUC), Build Bright University (BBU), and University of Human Resources (UHR). By September 2016, ACC had made some revisions to the instrument and guidelines (based on the consultations) and the ACC began to conduct full pilot evaluations in 12 out of 36 target HEIs.

57. Research grants were awarded based on the project timeline, but implementation was delayed due to limited experience by project managers on implementing research projects because there had been little to no research at the HEIs before HEQCIP. In addition, the universities and project managers had no previous experience with procurement and FM processes, which also delayed research project implementation. The project provided international technical assistance to support project managers in the implementation of research projects as well as considerable support and training by World Bank procurement and FM specialists on how to procure research materials as well as monitoring and reporting on project expenditures.

58. Scholarships. Stipends for scholarships were delayed due to HEI challenges in reporting attendance three times a year. To mitigate against possible dropout due to late payment, the project reduced the number of payments from three to two.

59. MTR and restructurings. The MTR was an important opportunity for a midcourse modification to the Results Framework through the 2014 restructuring. With the ACC responsibilities returned to MoEYS and the new HEI accreditation standards, the project was able to improve the project’s ability to measure achievement of the PDO. At the time of the 2015 restructuring, MoEYS was interested in increasing HEI autonomy and wanted an opportunity to build an HEI that exemplified good practice in governance through improved financial and HR management. There were project funds available to work with RUPP, and MoEYS sent a request to the World Bank to extend the project for this purpose. The project worked with RUPP to do an analysis of their financial and HR systems and was able to produce draft regulations


of 71

on HE governance and finance that present a strong framework for future development. Moreover, MoEYS was able to focus on increasing the number of people trained in STEM-related subjects by providing the opportunity for 15 students from 6 HEIs to receive master’s degrees in overseas STEM programs18 and return to Cambodia.

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME

A. QUALITY OF MONITORING AND EVALUATION (M&E)

M&E Design 60. The project M&E design was simple. The M&E design was developed to monitor and report on progress toward meeting the targets of the PDO and intermediate indicators in the Results Framework as well as monitoring project implementation progress. A project team was established for each component, and the team members were responsible for gathering the needed data for semiannual project reporting including for PDO and intermediate indicators. Each project HEI also appointed an M&E person to report on progress to the DHE on its own university project progress. The project also included project activities to enhance the project M&E with the inclusion of the SPS impact evaluation as well as development of the HEMIS. In addition, support for the ACC led to the development of an internal quality assurance (IQA) program for HEIs.

M&E Implementation 61. Reporting of data for the Results Framework indicators was systematically done throughout the life of the project. The DHE analyzed and reported on Results Framework indicator achievement on a semiannual basis, maintained databases for all the components on a monthly basis, and provided cumulative data on all the capacity building. At the project level, the design and development of the HEMIS was completed and piloted in 6 HEIs. The HEMIS was designed to collect and analyze data in terms of the ESP 2014–2018 and Higher Education Road Map indicators. At the sector level, the DHE also worked on the development of results-based planning and M&E framework for the HE sector based on the policy areas of access, equity, and institutions related to the ESP 2014–2018 and the results-based M&E Operational Manual that had been developed. The DHE M&E team collaborated with the HEMIS team in the development of the HEMIS to link with a results-based planning and M&E system. There were results-based planning workshops conducted for 71 HEIs, and during the project period, 24 HEIs submitted data based on the model. Moreover, an IQA Operations Manual and guidelines at the HEI program level were developed based on the ASEAN University Network Model that incorporated relevant criteria from the ACC institutional accreditation model, Cambodian Qualification Framework, and Cambodian Education Law. Workshops were held with HEIs on the IQA materials. Finally, the SPS impact evaluation was conducted over the life of the project by comparing the impact of the SPS on the 1,000 SPS recipients (receiving tuition waivers and stipends) to a control group that only received the tuition waiver.

18 The students received degrees related to computer science, building technology, applied geophysics, health toxicology, and electronic system design/engineering.


of 71

M&E Utilization 62. As indicated, MoEYS and HEIs have piloted the HEMIS and they are using the information from piloting to roll out the system to other HEIs. In addition, the first report using the system to generate enrollment numbers for 2016/17 in HEIs (9 public HEIs out of 13 and 58 private HEIs out of 91) has been produced. The annual MoEYS scholarships selection was completed using the HEMIS for the first time in August 2017. In the follow-on World Bank-supported HE project, use of the HEMIS will be expanded to additional universities and populated with their data. In addition, they will begin to build capacity within the additional universities on using the system and analyzing data as well as learning how to link the HEMIS to results-based planning, M&E frameworks, and international quality standards. The ACC team has already conducted accreditation assessments within the project-supported HEIs. These institutions met some of the standards and are still working on full accreditation—some will be fully accredited in 2018. Going forward, again, the next World Bank-supported project will build on the progress made in HEQCIP and work with the DHE, ACC, and HEIs on improving the ACC system and HEI internal quality assurance. Finally, MoEYS has already begun to reassess its scholarship program in light of the findings of the SPS impact evaluation. These are all substantial M&E achievements that helped lay a strong foundation for M&E within the sub-sector.

Justification of Overall Rating of Quality of M&E 63. The overall rating for M&E is substantial.

B. ENVIRONMENTAL SAFEGUARD, SOCIAL SAFEGUARD, AND FIDUCIARY COMPLIANCE 64. Safeguards. The project was rated a Category ‘C’ operation, not requiring a separate environment assessment. The project also did not trigger any additional World Bank safeguards policies, again not requiring separate assessments. Safeguards were monitored consistently during project implementation and were consistently rated Satisfactory.

65. Financial management. FM performance rating remained Moderately Satisfactory and Satisfactory throughout the life of the project. The FM rating was downgraded in late-2016 due to challenges related to FM internal controls at the HEIs. The updated project FM Manual was approved by the World Bank to include greater use of the country systems and to strengthen the internal controls. The project conducted a dissemination workshop on the new procedures, and upon completion, there was noticeable improvement in compliance with internal controls, and the workshop was facilitated by the international finance advisor and the World Bank FM specialist. The DHE/MoEYS was in compliance with the financial covenants of the Financing Agreement. The quarterly unaudited interim financial reports (IFRs) were submitted to the World Bank on time, and there were no outstanding IFRs. The required audits were submitted to the World Bank on time in accordance with the Financing Agreement, and the auditors’ opinion was unmodified (clean). The audit field work for 2017 audit was completed in November 2017. The final audit report was submitted on March 14, 2018.

66. Procurement. Procurement was consistently rated Moderately Satisfactory. This was primarily due to two factors. First, the DHE staff initially had no experience with the World Bank procurement procedures. The project implementation structure assigned HEQCIP procurement duties to the DHE staff rather than the experienced MoEYS procurement unit. This meant that the DHE staff needed considerable capacity development on the World Bank procurement procedures. In retrospect, the project design


of 71

should have relied on the MoEYS procurement unit. Second, the HEIs had no experience with the World Bank procurement procedures, which meant that directors of research projects needed extensive training in the World Bank procedures before being able to purchase materials for their research projects. These two factors played a role in slowing down disbursements in the early stages of project implementation. After the procurement training and support from the World Bank procurement staff, procurement performance improved as well as disbursements. It should also be noted that some of the DHE staff trained in procurement have now moved to the MoEYS unit because of their increased knowledge of procurement procedures.

C. BANK PERFORMANCE

Quality at Entry 67. The project preparation team ensured that the project design was closely aligned with the ESP 2006–2010 and the World Bank’s CAS updated in May 2008. The original design was simple, the PDO was precise, and key indicators were appropriate for measuring progress toward achieving the PDO and the medium-term goals outlined in the results chain. The design took into consideration the recommendations from the Quality Enhancement Review, peer reviewers, and lessons learned from other World Bank-supported HE projects. The preparation team identified the appropriate risks, incorporated design features to mitigate them, and included the relevant technical specialists to develop the project. The quality at entry for HEQCIP was Satisfactory.

Quality of Supervision 68. There were 10 World Bank review and implementation support missions, which included an MTR. The project team was actively engaged in supporting the government in its efforts to implement the project. Whenever implementation challenges arose, the World Bank team worked with the government to find appropriate solutions that would not compromise the integrity of the design. The supervision teams consistently reported on FM and procurement progress during supervision missions and worked with the project management unit team to build their capacity in these areas. They also systematically documented project progress in aide memoires, back-to-office reports, and Implementation Status and Results Reports, all of which kept the World Bank management informed of progress and provided the foundation for the Implementation Completion and Results Report (ICR) analysis. Key to the project’s achievements was the consistent supervision by the World Bank team, both in-country and from headquarters, that had the needed technical expertise to support the DHE with implementation. The MTR also played an important role in making a midcourse adjustment to the Results Framework by including an additional PDO indicator that could provide a more robust analysis of improvements in the quality of teaching, management, and research. Moreover, the World Bank’s willingness to respond to the government’s request to extend the project closing date and to pilot an effort that looked at improving HEI governance, with an eye toward HEI autonomy, was an important addition given Cambodia’s increased emphasis on HE and STEM education. These new activities provide valuable information to MoEYS and the World Bank for the development of the new World Bank-supported HE project. Overall, there was a high level of supervision for this project. Excellent collaboration on project implementation and coordinated work with MoEYS contributed to the many project successes. The quality of supervision of HEQCIP was Highly Satisfactory.


of 71

Justification of Overall Rating of Bank Performance 69. Based on the analysis above, overall World Bank performance is rated Satisfactory.

D. RISK TO DEVELOPMENT OUTCOME 70. The risk to sustaining the development outcome is low based on the following:

(a) Sustainability and continuation of the outcomes have been demonstrated through key policies. The project supported the development of key documents (Vision 2030 and Higher Education Road Map) that have contributed to HE policy reform that are now directly attached to the government’s IDP 2015–2025. Moreover, experience from the RUPP pilot is being used to develop legislation for university autonomy.

(b) Key activities have been institutionalized. The HEMIS and results-based planning have been fully developed and piloted. These will be further expanded under the new World Bank-supported HE project. The SPS program has proven to be highly successful in keeping poor students from dropping out and increasing completion rates. MoEYS is incorporating aspects of the SPS into the government scholarship program that includes stipends and tuition waivers.

(c) The capacity development activities have played a substantial role in increasing the levels of capacity related to accreditation, research, and HE management within MoEYS, DHE, and HEIs. The fellowship program not only increased the number of students holding international master’s and PhD degrees (supporting the medium-term results chain goal) but also led to increased professional connections and long-term partnership agreements with universities in Australia. These include, but are not limited to, continuing collaboration on the establishment of joint postgraduate programs particularly on STEM-related courses, MOUs with Cambodian universities to support capacity development in teaching and research, and grants to fund 15 students to a seven-week program to strengthen quality assurance in HE curricula.

(d) A research environment has been fostered to the extent that researchers and academics decided to form the CSRD and CCI, which can be used to identify areas of research collaboration and publish research findings. In addition, the use of a competitive research grant mechanism was introduced and will continue to be used in the new World Bank-supported project for awarding research grants.

71. The project provided the foundation for further development of the HE sub-sector. As indicated, the new World Bank-supported HEIP will build on the HEQCIP foundation and continue to support research, capacity development, HEMIS, quality of teaching, and improved quality of HEIs through accreditation and the internal quality assurance mechanism. This all contributes to the strong likelihood of HEQCIP interventions being sustained.


of 71

V. LESSONS LEARNED AND RECOMMENDATIONS 72. Lesson 1. Peer partnerships and networks (with international institutions, specialist professors, specialized consultants, and HEI administrators) are effective in improving the quality of professional performance.

73. Lesson 2. Forging long-term relations with HEIs overseas can reduce training and capacity development costs as well as offering sustainable ways of providing consistent capacity development opportunities.

74. Lesson 3. Research is essential if Cambodia is to achieve success in engaging with global knowledge networks and training its own postgraduate research students. HEIs are more successful in their research when they

(a) Set a clear vision of research and development and all members of the HEI community share this vision;

(b) Develop and implement their own research and development strategic plan;

(c) Actively engage with and support the procurement and financial tasks of research projects at the institutional level;

(d) Ensure that there are no financial or management tensions between research and teaching; and

(e) Actively involve students in research.

75. Lesson 4. Research grants

(a) Encourage dialogue between government and university leaders on national priorities and institutional goals;

(b) Combine autonomy with added accountability for results;

(c) Stimulate institutional strategic planning and improvement processes;

(d) Encourage quality research; and

(e) Work best when designed with a flexible funding mechanism with transparent procedures.

76. Lesson 5. Open dialogue, flexibility, and a willingness of all parties to solve problems together is a key to achieving successful outcomes in complex and innovative projects.

77. Lesson 6. To ensure that targets are met in a timely and effective manner, the project management or coordinating entity must commit sufficient resources, especially manpower, to regularly monitor project activities as a whole and take an active part in supporting successful implementation.


of 71

78. Recommendation 1. Going forward, close coordination for quality assurance is recommended between the Director General of Higher Education (DGHE) and ACC. Effective quality assurance of HEIs requires that ACC, as an external quality assurance body, conducts assessments to determine the accreditation status based on national standards while the DGHE supports HEIs to undertake continuous internal quality assurance based on the recommendations outlined by ACC. Smooth coordination and close collaboration between the DGHE and ACC will strengthen the quality assurance mechanism.

79. Recommendation 2. It is recommended that research activities support the implementation of the IDP. Linking to the IDP will increase the relevance of research activities. Building off the nascent research culture supported by HEQCIP, new research activities should focus on STEM and agriculture, two priority areas in the IDP. Close collaboration with private industries will further strengthen the relevance of research activities.

80. Recommendation 3. Expanding the national scholarship program to HE is recommended because the SPS impact evaluation found a significant impact from the pilot scholarship program. The completion rate for the pilot scholarship students was 86 percent versus 57 percent for the government tuition waiver students.

81. Recommendation 4. Now that the foundation for increased capacity has been established within HEIs and the DGHE, it is recommended that future projects include student tracer studies as an additional measure for the quality of HEI programs.

.


of 71

ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS

A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Improve the quality of teaching management, and research in project-supported entities.

Indicator Name Unit of Measure Baseline Original Target Formally Revised Target

Actual Achieved at Completion

40 percent of HEIs supported under Components 1 and 2 showing an increased aggregate score against a modified subset of ACC accreditation criteria.

Percentage 0.00 40.00 40.00 46.00

03-Jan-2011 31-Dec-2015 30-Sep-2017 30-Sep-2017

Comments (achievements against targets): This indicator was added at the project restructuring.



50 percent of Sub-projects approved under the first round of proposals completed by the Closing Date, and 70 percent of Sub-projects approved under the second round completed by

Percentage 0.00 70.00 70.00 100.00

03-Jan-2011 31-Dec-2015 30-Sep-2017 30-Sep-2017


of 71

the Closing Date.

Comments (achievements against targets): Objective/Outcome: Pilot the targeting of disadvantaged students for enhanced retention.



An average annual retention rate of 95 percent of project-supported scholarship recipients

Percentage 0.00 95.00 95.00 96.00

03-Jan-2011 31-Dec-2015 30-Sep-2017 30-Sep-2017

Comments (achievements against targets): Baseline is 0 as this is a pilot scholarship scheme.

A.2 Intermediate Results Indicators

Component: 1: Strengthening the Capacity of the Higher Education System



Number of staff (MoEYS and HEI) who participate in training courses which improved their professional competencies; fields related to: HE quality, management, assessment, and accreditation; English; o

Number 0.00 1500.00 1500.00 7395.00

03-Jan-2011 31-Dec-2015 30-Sep-2017 30-Sep-2017


of 71

Comments (achievements against targets):



Number of staff of (MoEYS and HEI) who were conferred with a Master or Doctor Degree in academic specialty, educational management or related fields

Number 0.00 54.00 54.00 79.00

03-Jan-2011 31-Dec-2015 30-Sep-2017 30-Sep-2017

Comments (achievements against targets): Component: 2: Provision of Competitive Development and Innovation Grants



Number and percentage of HEI academic staff in participating institutions completing training in research methodologies and design

Percentage 0.00 180.00 180.00 579.00

31-Dec-2011 31-Dec-2015 30-Sep-2017 30-Sep-2017



of 71



Number and percentage of eligible HEIs submitting proposals to RGMC competing for funds

Percentage 0.00 70.00 70.00 72.00

03-Jan-2011 31-Dec-2015 30-Sep-2017 30-Sep-2017




Number and percentage of complete research projects (competitively selected) funded by the project

Percentage 0.00 100.00 100.00 100.00

03-Jan-2011 31-Dec-2015 30-Sep-2017 30-Sep-2017




Within RGMC/DHE, number and percentage of proposals awarded after review and evaluation in accordance with project processing timeline

Percentage 0.00 100.00 47.00 47.00

03-Jan-2011 31-Dec-2015 30-Sep-2017 30-Sep-2017


of 71

Comments (achievements against targets): Component: 3: Provision of Scholarship to Disadvantaged Students



Pass rate of retained scholarship recipients at each grade categorized by: female; rural; poor (not cumulative rate)

Percentage 0.00 80.00 80.00 100.00

03-Jan-2011 31-Dec-2015 30-Sep-2017 30-Sep-2017




Drop out rate of scholarship recipients categorized by: female; rural; poor

Percentage 0.00 19.00 19.00 16.00

03-Jan-2011 31-Dec-2015 30-Sep-2017 30-Sep-2017




Completion rate of scholarship recipients, categorized by: female; rural;

Percentage 0.00 67.00 67.00 85.00

03-Jan-2011 31-Dec-2015 30-Sep-2017 30-Sep-2017


of 71

poor

Comments (achievements against targets): Component: 4: Project Management and Monitoring and Evaluation



Higher Education Governance & Finance Policy Paper adopted by the MoEYS

Yes/No N Y Y Y

31-Dec-2010 30-Sep-2017 30-Sep-2017 30-Sep-2017




Higher Education Vision and Strategy 2030 adopted by the MoEYS

Yes/No N Y Y Y

31-Dec-2010 31-Dec-2015 30-Sep-2017 30-Sep-2017




HEMIS developed and piloted with at-least 4 HEIs

Yes/No N Y Y Y

31-Dec-2010 30-Sep-2017 30-Sep-2017 30-Sep-2017


of 71



of 71

B. KEY OUTPUTS BY COMPONENT

Objective/Outcome 1 Improve the quality of teaching, management, and research in project –supported entities

Outcome Indicators

1. 50 percent of sub-projects approved under the first round of proposals completed by the closing date and 70 percent of sub-projects approved under the second round completed by the closing date 2. 40 percent of HEIs supported under Components 1 and 2 showing an increased aggregate score against a modified subset of ACC accreditation criteria

Intermediate Results Indicators

1. Number of staff (MoEYS and HEIs) who participate in training courses which improved their professional competencies 2. Number of staff (MoEYS and HEIs) who were conferred Master or Doctor degree in academic specialty, educational management or related fields 3. Number and percentage of HEI academic staff in participating institutions completing training in research methodologies and design 4. Number and percentage of eligible HEIs submitting proposals to RGMC competing for funds 5. Number and percentage of complete research projects (competitively selected) funded by the project 6. Within RGMC/DHE, number and percentage of proposals awarded after review and evaluation in accordance with project processing timeline 7. Higher Education Governance & Finance Policy Paper adopted by the MoEYS 8. Higher Education Vision and Strategy 2030 adopted by the MoEYS 9. HEMIS developed and piloted


of 71

Key Outputs by Component (linked to the achievement of the Objective/Outcome 1)

Components 1, 3, and 4 1. 841 training programs for 7,395 participants from MoEYS and HEIs related to research, assessment, procurement, FM, and so on. 2. 79 individuals received master’s and PhD degrees in academic specialties exceeding the target. 3. 579 HEI academic staff received training in research methodologies 4. 73 percent of eligible HEIs submitted research proposal to RGMC exceeding the target. 5. 100 percent of research projects funded (45 total) were completed exceeding the target. 6. Higher Education Governance & Finance policy paper was adopted by MoEYS. 7. Higher Education Vision and Strategy 2030 were completed and adopted by MoEYS. 8. The HEMIS was developed and piloted in 6 HEIs. 24 HEIs (including provincial branches) uploaded enrollment data into the HEMIS. MoEYS continues to roll out the HEMIS to other HEIs. 9. Additional outputs were Draft Higher Education Action Plan 2018–2022; Draft Regulation on HE Governance and Finance; Functional Review of the ACC; and Functional Review of the DHE and DSR.

Objective/Outcome 2 Pilot the targeting of disadvantaged students for enhanced retention

Outcome Indicators 1. An average annual retention rate of 95 percent of project-supported scholarship recipients

Intermediate Results Indicators

1. Pass rate of retained scholarship recipients at each grade categorized by female, rural, poor (not cumulative rate) 2. Dropout rate of scholarship recipients categorized by female, rural, poor 3. Completion rate of scholarship recipients categorized by female, rural, poor


of 71

Key Outputs by Component (linked to the achievement of the Objective/Outcome 2)

Component 2 1. Pass rates exceeded the target of 80 percent in 2014, 2015, 2016, and 2017 for all three categories (female, rural, and poor). 2. Dropout rates were less than the target of 19 percent in 2014, 2015, 2016, and 2017 for all three categories (female, rural, and poor) exceeding the target. 3. Completion rates exceeded the target of 67 percent for both 2016 and 2017. 4. Additional outputs included an Operations Manual on implementation of SPSs; a draft policy on scholarships, subsidies, and student loans; data to inform the scholarships impact evaluation has been collected; a booklet to showcase SPS student success stories has been completed; a University Guidebook has been developed to promote interest in university study among Grade 12 students; and a video showcasing examples of SPS student achievement has been completed and it is published on the World Bank website.


of 71

ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION

A. TASK TEAM MEMBERS

Name Role

Preparation Omporn Regel Task Team Leader

Simeth Beng Team Member

Tsuyoshi Fukao Team Member

Dilip Parajuli Education Economist

Prateek Tandon Education Economist

Hena Mukherjee HE Specialist

Chandra Chakravarti Program Assistant

Ravan Chieap Program Assistant

Supervision/ICR

Tsuyoshi Fukao Task Team Leader

Latharo Lor Procurement Specialist

Reaksmey Keo Sok FM Specialist

Chandra Chakravarthi Team Member

Simeth Beng Team Member

Vanna Nil Social Safeguards Specialist

Bunlong Leng Social Safeguards Specialist

Hena Mukherjee Consultant, HE

Juan Prawda Consultant, M&E

Mokhlesure Rahman Operations Officer

Sophear Khiev FM Specialist

Ravan Chieap Program Assistant

Prateek Tandon Education Economist

Sandra Beemer Consultant


of 71

B. STAFF TIME AND COST

Stage of Project Cycle Staff Time and Cost

No. of staff weeks US$ (including travel and consultant costs)

Preparation FY08 5.350 46,966.34

FY09 56.360 319,765.09

FY10 62.278 327,742.83

FY11 14.675 45,093.68

Total 138.66 739,567.94

Supervision/ICR

FY11 19.788 116,594.64

FY12 24.405 83,621.73

FY13 33.010 70,551.79

FY14 24.525 92,628.73

FY15 28.283 93,224.29

FY16 22.260 94,219.35

FY17 11.800 62,493.63

FY18 15.414 88,070.17

Total 179.49 701,404.33


of 71

ANNEX 3. PROJECT COST BY COMPONENT

Components Amount at Approval (US$, millions)

Actual at Project Closing (US$, millions)

Percentage of Approval (%)

Strengthening the Capacity of the Higher Education System

6.39 10.36a 162

Provision of Competitive Development and Innovation Grants

4.40 3.66 83

Provision of Scholarships to Disadvantaged Students 5.75 4.24 74

Project Management and Monitoring and Evaluation 4.40 4.59 104

Unallocated 2.06

Total 23.00 22.85 99.35

Note: a. During the 2014 restructuring, unallocated funds and underspent funds were reallocated to Component 1 to finance additional fellowships and capacity-building activities due to higher unit costs of postgraduate programs and higher demand for training opportunities.


of 71

ANNEX 4. EFFICIENCY ANALYSIS 1. The economic analysis for this project is performed both in terms of ‘Quality’ and ‘Quantity’ dimensions. Under Component 1, the capacity of the HE system has been enhanced through improving the overall development, management, and governance of the HE sub-sector. Concurrently, the organizational capacity of individual HEIs was strengthened through a strategic staff development plan covering general training, technical skill, and leadership development.

2. Further, Component 2 has contributed to

(a) Strengthening the capacity of participating public and private HEIs and providing enabling conditions for improved quality in research, teaching, and learning; and

(b) Introducing an efficient and sustainable mechanism, which emphasizes innovation and accountability, in the DHE for the allocation of public funds for research in HEIs.

3. It is assumed that the beneficiary students in the participating HEIs benefit from these interventions through improved teaching and learning environment and better research capacity of the universities. As a result, these graduates would possess higher level of skills and benefit from higher earnings streams throughout their working lives. Three scenarios are explored for the CBA. In the baseline scenario, it is assumed that the earnings ‘premium’ will be equivalent to 0.2 year of tertiary education. In the ‘Low’ and ‘High’ scenarios, it is assumed that the premiums will be equivalent to 0 and 0.4 year of tertiary education, respectively.

4. The number of student beneficiaries used for the CBA is based on actual enrolment figures between 2011/2012 through 2015/2016 academic years (see Table 4.1). It is assumed that 10 cohorts of tertiary students will benefit from the interventions. It is assumed that enrolment will remain constant at 44,659 for student cohorts enrolling after 2015/2016.

Table 4.1. Enrollment in the Participating HEIs

Year Enrolment in Year 1

Total Public Private

Total Female Total Female Total Female 2011–2012 5,255 2,460 47,517 21,856 52,772 24,316 2012–2013 9,366 4,518 58,245 24,438 67,611 28,956 2013–2014 10,116 4,598 50,893 21,498 61,009 26,096 2014–2015 7,108 3,458 34,244 15,985 41,352 19,443 2015–2016 7,910 3,765 36,749 17,174 44,659 20,939

Source: DHE.

5. From the ‘Quantity’ perspective, 72 HEI lecturers were provided with fellowships to study at the postgraduate level under Component 1. It is assumed that each lecturer was responsible for the generation of an additional 10 graduates who would otherwise have dropped out without this program. Under Component 2, 45 research grants were awarded in the first year. It is assumed that each grant


of 71

program generated an additional 5 graduates who would otherwise have dropped out. Only a single cohort of students benefited from these activities.

6. The impact of Component 3 is also viewed under the ‘Quantity’ perspective. This ‘special-priority’ scholarship program is a pilot to provide tuition and stipends to 1,000 poor students who met both educational as well as pro-poor selection criteria. From an impact evaluation study carried out, it is estimated that the dropout rate for this group of disadvantaged students has declined from 48.8 percent to 15.5 percent as a result of the program (only one cohort of student beneficiaries benefited from the pilot program). Therefore, the number of dropouts is estimated to have declined by 332 as a result of the intervention.

7. Regarding the cost side, taken into account were both the additional private and public recurrent cost incurred for those who would have otherwise dropped out without the interventions. The public per student cost of US$1,000 per year is used in this CBA exercise, while the private household per student spending of US$769 per year is estimated from the 2014 round of the Cambodian Household Socioeconomic Surveys. Both figures are expressed in constant U.S. dollar throughout the analysis. Actual total project costs incurred over five years, amounting to US$22.85 million, are also taken into account in the CBA.

8. Table 4.2 presents the NPVs and IRRs of the project, analyzed through the quantity and quality perspectives as discussed above. In the baseline scenario, the resulting NPV is estimated at US$283.1 million, using a real cost of capital (or real discount rate) of 5 percent, with an IRR of 22 percent.

9. In the sensitivity analysis, two other scenarios are also evaluated. In the ‘Low’ scenario, the NPV declines to US$15.0 million, with an IRR of 1.6 percent. Finally, in the ‘High’ scenario, the NPV reaches US$587 million, with an IRR of 29.6 percent. The results from the economic analysis, therefore, suggest that the project yielded significant net economic benefits.

Table 4.2. CBA of HEQCIP


Present Value Benefits - HE Quality Enhancement 298,346,625 0 602,282,285 Present Value Benefits - Fellowship + Development and Innovation Grant 8,614,491 8,614,491 8,614,491 Present Value Benefits - Scholarships to Disadvantaged Student 3,029,202 3,029,202 3,029,202 Present Value Recurrent Cost - Fellowship + Development and Innovation Grant 2,819,480 2,819,480 2,819,480 Present Value Recurrent Cost - Scholarships to Disadvantaged Student 991,443 991,443 991,443 Total project spending by component:

Capacity Building 10,366,293 10,366,293 10,366,293 Development and Innovation Grants 3,663,724 3,663,724 3,663,724 Scholarships to Disadvantaged Student 4,238,060 4,238,060 4,238,060 Program Management 4,581,836 4,581,836 4,581,836

Total project spending 22,849,913 22,849,913 22,849,913 NPV (5% discount rate) 283,329,482 (15,017,143) 587,265,143


of 71


IRR 22.0% 1.6% 29.6%


of 71

ANNEX 5. BORROWER, CO-FINANCIERS, AND OTHER PARTNERS/STAKEHOLDERS COMMENTS

Higher Education Quality and Capacity Improvement Project (Credit No: 4796-KH Grant No: 607-KH)

Borrowers Portion of the ICR

Main Outputs by Component

1. Component 1 overall objective. This component aimed to improve the overall development, management, and governance of the HE sub-sector by strengthening institutional capacity of the DHE and DSR as well as the ACC and HEIs and overall coordination of the HE system.

2. Capacity-Building objective. To improve the overall development and management of the HE sub-sector by focusing on professional development at central as well as HEI levels to strengthen the capacity of individual HEIs.

3. Key outputs achieved. Training and workshops - A grand total of 7,395 (13 percent female) staff from MoEYS, other ministries, and HEIs participated in 841 HEQCIP capacity development programs. Of these, 2,224 (19 percent female) participants were from MoEYS and other ministries while 5,171 (10 percent female) were from HEIs. There were 758 local trainings/ workshops/seminars/conferences/ assessment and monitoring visits, and so on and 83 overseas training, workshops, and conferences.

4. Fellowships. A total of 79 (31 female) HEI and MoEYS staff benefited from HEQCIP by being awarded fellowships. In the first round, 64 MoEYS and HEI staff benefited from fellowships to Australia. Of the 62 staff who graduated, 59 gained master’s degrees and 3 were awarded PhDs. A further 18 staff graduated with diplomas in teaching English as a second language (TESOL) from the University of Canberra (UC). The project extension from December 2015 to September 2017 offered additional time for one more round of fellowships. This final round of fellowships was designed to strengthen skills in STEM subjects only and took place at the University of Sains Malaysia. A total of 16 fellowships were secured but one person dropped out before the course started in early September 2016. Therefore, 15 staff (7 females, 47 percent) from 6 public HEIs went to Malaysia. All 15 completed the course.

5. ACC. The Guidelines for Higher Education Institutional Accreditation were approved in 2012; new board members were selected in 2014; legal documents on ACC’s mandate were finalized in 2014; revised ACC standards and guidelines were officially approved during 2015/16; and 36 evaluations were conducted in all 36 target HEIs.

6. DSR. Early in the project, the whole DSR building was renovated, and new office furniture and equipment were procured. This raised the standard of facilities and created a more conducive working environment. DSR initiated no specific activities, but the management team and staff still joined relevant training programs/workshops organized by the DHE.

7. RUPP. An FM Manual and an HR Management Manual transferable to other HEIs were produced.

8. Other outputs. Curriculum Framework for English for Specific Academic Purposes prepared; HEI research and innovation priority report finalized; Strategic Plan for STEM Promotion Policy developed; the CSRD established; the CCI established.


of 71

9. Key factors affecting successful achievement of objectives. Capacity building: Apart from the fellowships program, capacity-building activities were initiated through short courses. These required short inputs from highly qualified and experienced trainers sought through overseas networks and local consultants returning home having gained their PhD from abroad. The Ministry of Economic and Finance (MEF) standard operating procedures for procurement made it difficult to hire short-term trainers because the procedures are founded on competitive selection, which is more suited to longer-term commitments. It was also difficult to benefit fully from the majority of local PhD holders because most of them were public servants. A system of recruiting key resource persons was introduced in 2014. This enabled the project to follow an effective procedure for the recruitment of both local and international short-term trainers. The process was very successful in enabling the project to benefit from using local PhDs who understood the academic context in which the participants were working and were able to share their skills and knowledge directly in Khmer without using translation from English.

10. Fellowships. Procedures for students to obtain visas and school admissions were complicated and time-consuming. The demand for a high level of English proficiency was a challenge for all students studying abroad.

11. Support for ACC. Institutional accreditation standards and guidelines were approved early in the project. ACC planned to conduct the first institutional assessments for 22 HEIs in 2012 and complete institutional assessments for all HEIs by 2014. However, in 2013, the legal status of ACC was transferred from the Council of Ministers to MoEYS. By the end of 2014, the transfer had been concluded. A set of revised ACC standards were officially approved and guidelines developed. These were the standards used to develop the new project PDO indicators. Once approved, there was an additional 2015 consultative process, which involved selected HEIs including RUPP, University of Health Sciences (UHS), PUC, BBU, and UHR. By September 2016, the ACC revised instrument and guidelines had been officially approved and the ACC team had conducted pilot evaluations in 12 out of 36 target HEIs by project closing. The results of this will enable the ACC to begin the full institutional accreditation process in 2018.

12. Support to RUPP. FM - Based on the project support after the 2015 restructuring, RUPP has made several improvements to its financial practices. Finance staff have improved their skills; however, further support will be necessary for further skills upgrading given that many staff still do not have a financial background and have limited theoretical accounting knowledge and skills.

13. HR management. RUPP has just started the reform of HR management, but more support is needed to build the relevant skills and knowledge of staff in the Personnel Office.

14. Sustainability after HEQCIP has finished. The skills and knowledge gained by participants through the capacity development and fellowships programs will continue to strengthen the sector as a whole. In general, the long-term partnership agreements with universities in Australia, first established as part of the fellowships program, will help support strategic relationships so that Cambodian HE can be part of the wider international academic network and ensure possible future technical collaboration. More particularly, fellowship graduates have formed strong Alumni relations with CDU, and this has resulted in significant benefits to Cambodia. CDU has signed MOUs with RUPP, Svay Reang University (SRU), and CDRI as well as the Cambodia International Academy (schools) to support capacity development particularly in teaching and research. CDU was recently awarded a prestigious Australia Awards Fellowship Grant to build capacity and integrate international standards into teacher education programs at RUPP. Part of this grant will be used to fund 15 Australia Award Fellows from RUPP to follow a seven-week program at CDU to


of 71

strengthen capacity in Quality Assurance in Higher Education Curricula. The project also has a training and research component. Upon their return, the fellows will train staff at RUPP and SRU in quality assurance. Reflections and experiences from the training and implementation phase will be explored in a book co-published by researchers from RUPP, SRU, and CDU. In addition to this, one staff member from CDU will work with RUPP, CDRI, and DGHE to help develop a STEM curriculum and teacher education program for Cambodia. An emerging new plan with CDU envisages CDU students undertaking placements in Cambodian schools to share skills and knowledge. This strong link with CDU would not have been possible without the links made through the HEQCIP fellowship program. In addition to this, a proposal is being developed to establish joint postgraduate courses integrating Cambodian STEM scholars into CDU STEM teams and STEM teacher training. Because STEM-related and Science, Technology, Engineering, Arts, and Mathematics (STEAM)-related subjects require proficiency in English Language, teaching English with the help of technology will be part of a STEAM/STEM development with research engaging technology, neuroscience, arts, and language. Development will be supported by the CDU STEM specialist. Development of an online database to support STEM education is also being discussed, and CDU is currently preparing an MOU with the Institute of Technology of Cambodia (ITC) to initiate this. Most importantly, these connections and future plans offer opportunities for Cambodia to raise its profile with the Northern Territories Government (NTG). The NTG offers scholarships at undergraduate and postgraduate levels to many Southeast Asian countries, which may include Cambodia in the future.

15. Support to ACC. The capacity developed through the project has enabled ACC to start conducting formal institutional accreditation from 2018 onward.

16. Support to RUPP. Reforming FM at RUPP has been strongly supported by the Rector, the Board, the management team, and development partners. Decisions to centralize more of the finance function (for example, moving the Institute of Foreign Languages (IFL) finance staff to the central finance office) have helped have more consistent financial processes and more complete financial information. HR management reform has also supported by the Rector and management team. Moving IFL personnel officers to the central personnel office has made HR management work more effective.

17. Key enabling factors. Good team work and communications with the project management, procurement, and finance teams at the World Bank and central levels; strong appreciation from the HEIs on the quality of training as demonstrated by the increase in the number of HEI participants and demand for training from HEIs over the life of the project; strong participation and support from HEI faculty and researchers to join the capacity-building activities as presenters and technical facilitators; establishment of effective overseas contacts and academic networks; and strong support from the MEF.

18. Conclusion. The creation of overseas networks through the fellowships program and the establishment of the CSRD and CCI have strengthened the sub-sector as a whole at technical, management, and leadership levels. The accreditation process and the role of ACC have seen many fluctuations, but an agreed set of guidelines and pool of trained assessors have now been established, and full institutional accreditation will begin in 2018. RUPP has improved its management in many areas during HEQCIP including management of finance, staff, research, and information. Participation in the project has enabled RUPP to lay the foundations for becoming a flagship HEI.


of 71

Component 2 overall objective: To strengthen the capacity of participating HEIs and provide enabling conditions for improving quality in research, teaching, and management

19. Key outputs achieved. A total of 45 grants were awarded to 24 HEIs, and all were successfully completed. Under Window 1, which was to provide support to teaching and learning, 25 grants were awarded. Under Window 2, which was to provide solutions to local problems, 20 grants were awarded. Science-based subjects were awarded 26 grants and non-science subjects were awarded 19 grants. Of these grants, 14 were on curriculum development, 16 were on experimental research projects, 14 were on pure research, and 1 was on applied research. The project funded small works and procured technical resources as well as office equipment and furniture to assist HEIS to improve research and STEAM-related education provision in line with the requirements of the ESP, HE Reform Agenda, and the IDP. A total of 6 new buildings were constructed and 9 were renovated using project research grant funds. The HEIs that received the new and renovated buildings will benefit from these for generations to come in terms of having more relevant teaching and learning resources.

20. A ground-breaking research culture in Cambodian HE is emerging, and at project closing, 16 HEIs had prepared research promotion policies. As a direct result of the capacity developed in HEQCIP, some HEIs have succeeded in gaining research grants from other donors. Other outputs include stocktaking study prepared and results disseminated to all stakeholders; edited volume prepared and disseminated to stakeholders; Independent Evaluation Report prepared and disseminated to stakeholders; the CSRD established; recognition of research as an essential activity for HEIs introduced; the CCI established; and MEF has recognized education research as a valid activity for HEIs.

21. Key factors affecting successful achievement of objectives. Research as a new concept - The majority of HEIs had no experience of conducting research. Sub-project teams needed basic training and technical on-the-job support before they could begin. Research project management experience - The majority of HEIs had no experience of managing research projects. Some HEIs suffered a lack of strong support from their management level in implementing the research projects.

22. Sustainability after HEQCIP has finished. Some of the sub-grant activities have led directly to sustainable improvement through, for example, curriculum development, improvement of farmer and community livelihoods, and supporting development of enquiry-based learning in some HEIs. The skills and knowledge generated through the implementation of the sub-projects has generated a new culture where research has gained considerable intellectual capital. The grants have provided an opportunity for HEIs to develop their own research projects and prepare research proposals. They have also introduced a model for supporting and sustaining academic research in a Cambodian context. Many public HEIs are now actively involved in joint research projects, for example, Erasmus+, SEA-EU-NET19, and Re-inventing Japan. The establishment of the CSRD becomes a key forum to build networks as well as capacity and sustainably promote a research culture in Cambodia. Links with CDU through the fellowships alumni are leading to further opportunities for research as part of a STEAM/STEM development with research engaging technology, neuroscience, arts, and language.

23. Key lessons learned. If research is to develop sustainably in HEIs, there is a need to find ways to support both teaching and research. Involving students in the research projects has been very successful in enriching the outcomes of the sub-projects and increasing the relevance of student experience. Strong

19 SEA-EU-NET is a cooperation network deepening science and technology cooperation between Europe and Southeast Asia.


of 71

support and encouragement from the management level at HEIs is vital to ensure successful research project management and implementation.

24. Key enabling factors. Two key factors secured a successful outcome for this component. The first was strong support from the project manager who worked with the Rectors of participating HEIs to gain their cooperation. Equally important was the willingness of the World Bank to provide substantial technical and management support to the HEI research teams.

25. Conclusion. Despite initial delays, the HEIs successfully completed their research sub-projects within the original project completion date, and many have now gained research funding from other donors. The MEF has recognized research as a valid activity for HEIs, and a research culture has been firmly established.

Component 3 overall objectives: To increase access and retention of disadvantaged students in HE through the provision of the tuition and stipend based on pro-poor targeting and educational criteria and to assess the effectiveness of this pilot pro-poor scholarship program through an appropriate impact evaluation

26. Key outputs. From an original total of 1,000 selected HEQCIP scholarship recipients, 555 (56 percent) were male and 445 (44 percent) were female. A total of 196 (20 percent) were from rural areas and 804 (80 percent) were from urban areas. Of the 1,000 selected students, 1 withdrew immediately leaving 999. In the final year, 844 (377 female) students were retained. This is significantly higher than that of HE students receiving other scholarships. 100 percent of all retained SPS Students successfully completed their courses. Other outputs include an Operations Manual on the implementation of SPSs; a draft policy on scholarships, subsidies, and student loans; data to inform the scholarships impact evaluation has been collected; a booklet to showcase SPS student success stories has been completed; a University Guidebook has been developed to promote interest in university study among Grade 12 students; and a video showcasing examples of SPS student achievement has been completed and it is published on the World Bank website.

27. Key factors affecting successful achievement of objectives. Delays by HEIs in sending reports on attendance as well as achievement and other information on SPS students often delayed the disbursement of stipends. To militate against possible dropout due to late payment, the project reduced the number of payments from three to two per year in 2015. There was a delay in preparing the draft policy on student scholarships and loans due to pressure of work. However, the project hired a short-term consultant in 2016, and the draft policy is now under consideration by MoEYS senior leaders.

28. Sustainability after HEQCIP. The SPSs will not be continued exactly as piloted in the project. However, there are four main outcomes that contribute to the sustainability of the skills, knowledge, and lessons learned as a result of the program, as follows:

(a) Results of the project-supported impact evaluation have supplied key information to contribute to national policy decisions on the use of scholarships to increase access to and retention of disadvantaged students in HE.

(b) The skills and knowledge gained by the scholarship recipients themselves can be seen as a positive investment in the development of the national economy.


of 71

(c) The Component 3 Scholarships team members have gained knowledge on scholarship management and student loans, which will be invaluable to implement future programs.

(d) The Operations Manual developed for the implementation of the program can be used as the basis for the implementation of future scholarships programs.

29. Key lessons learned. Provision of realistic stipends, tuition fees, and counseling lead to increased completion rates. The cumulative retention rate for SPS students was 86 percent in the final year. An analysis of the subject matter selected by the students shows that 83.6 percent selected business- or administration-related courses leaving only 16.4 percent who studied STEM-related courses to meet the needs of the developing economy in line with the ESP, HE Reform Agenda, and IDP. More opportunities to study STEM-related subjects are needed at lower secondary level or earlier to increase the number of STEM students at HE level.

30. Key enabling factors. The reliability of scholarship payments gave students confidence to continue studying. A realistic amount of scholarship payment enabled students to concentrate on their studies. Regular monitoring and on-site visits to students as well as group visits by the DHE staff helped solve problems on time and encouraged students to continue with their studies

31. Conclusion. The SPS program was very successful in terms of attracting and retaining students from poor and disadvantaged backgrounds. The retention figures were significantly higher than those receiving MoEYS scholarships. The scholarship stipend was US$70 per month for students in Phnom Penh and US$50 per month for students in the provinces, and it also included payment of tuition fees. This is significantly higher than current MoEYS scholarships. Feedback from scholarship recipients shows that the extra attention given to them by tutors at HEI level and regular follow-up visits by the DHE staff were highly appreciated. This encouragement and attention balanced with realistic financial support contributed to a very successful outcome. A full impact evaluation will be conducted by the World Bank using baseline and end line data collected during the project. This will reveal the true effect of the scholarships program and serve as a basis for future development.

Component 4. Subcomponents 4.1: Monitoring and Evaluation, 4.2: HEMIS, 4.3: Development of an HE Governance Policy Paper with Operational Manuals on Finance and HR for Autonomous HEIs, and 4.4: Project Management

4.1: Monitoring and Evaluation

32. Objective. To assist the DHE, ACC, and HEIs to systematically collect, organize, analyze, and report data to better manage HEQCIP implementation in the DHE and also HE sub-sector.

33. Key outputs achieved. The M&E team was established, and members were fully trained; M&E Operations Manual produced; databases created for systematic analysis and reporting; Results-based Planning and M&E System for the HE sector matrix was finalized and is in use by HEIs; baseline and end line data from the 36 target HEIs were collected to inform the project on level of achievement against PDO 1 indicators; a comprehensive set of project-related quantitative data covering the whole duration of the project has been compiled; and IQA guidelines and manuals have been drafted.


of 71

34. Key factors affecting successful achievement of objectives. The M&E system was established after the MTR in 2013 and the international technical assistance was contracted from 2014. This reduced the time available to establish the system and institutionalize it.

35. Sustainability after HEQCIP has finished. Staff within the DHE have greater capacity to train and support the HEIs in results-based planning, results-based M&E, and results-based management. Staff within the DHE also have greater capacity to train and support HEIs in implementing the IQA system. As at September 2017, of the 36 HEIs covered by HEQCIP, 24 are actively using the results-based management planning and M&E system. The system was linked with the HEMIS in September 2017. This will make it possible in the future to obtain real-time data from all the HEIs as a basis for results-based planning.

36. Key lessons learned. Establishment of a team of fully trained, core M&E staff at the DHE and HEI levels increased the effectiveness of the system.

37. Key enabling factors. These included the establishment of committed implementation teams at the DHE and HEI levels as well as consistent on-the-job and online support from the international technical assistance and strong support from the ministry senior leaders.

4.2: HEMIS

38. Objective. Initiate, develop, and facilitate the supply, installation, and configuration of a HEMIS, which effectively enables staff to access information to inform the HE section of the ESP, the DHE - Annual Operations Plan, other sub-sector M&E outputs, and the annual sub-sector review

39. Key outputs achieved. Established a basic HEMIS capability using software that can be integrated with all other MoEYS management information systems; 6 HEIs fully piloted the HEMIS against a target of 4, and 24 HEIs (including provincial branches) uploaded enrollment data into the HEMIS. The system stores all Grade 12 student details regarding results of pass and fail from the year 2002 up to present based on the annual upgrading options. The first report using the system to generate enrollment numbers for 2016/17 in HEIs (9 public HEIs out of 13 and 58 private HEIs out of 91) has been produced. The annual MoEYS scholarships selection was completed using the HEMIS for the first time in August 2017.

40. Key factors affecting successful achievement of objectives. The small team charged with the establishment of the HEMIS had no background experience or training before embarking on the development of the system. Recommendations from independent consultant reports were beyond the financial and technical resources of the project. Some HEIs were reluctant to share their data. The team has operated with minimal numbers of staff. More skilled staff are needed in the future to scale up the development of the system.

41. Sustainability after HEQCIP has finished. The HEQCIP HEMIS team has been trained by a firm in developing and managing the HEMIS, and the team now has the capacity to maintain and further develop the HEMIS in the future. The activity is operational and mainstreamed into MoEYS. Therefore, it is completely sustainable.


of 71

42. Key lessons learned. Support from senior management at HEI level is critical to successful operation of a nation-wide HEMIS. The HEMIS team learned to accept risk and manage it as best as possible.

43. Key enabling factors. The key enabling factor in establishing the basic HEMIS was the dedication of the HEMIS team at the DHE and support from the project manager. This was a very complex subcomponent calling for new approaches to data collection and intensive technical training over a short period, but the team members did everything they could to share their skills and knowledge and encourage HEIs to share their data.

44. Conclusion. Progress was slow, but the team has gained a great deal of knowledge and has shared it with all HEI staff at a range of workshops during the last three years of the project. The basic technology is in place, but further expansion of the system will now largely depend on support from the senior management of the HEIs.

4.3: Development of an HE Governance Policy Paper with Operational Manuals on Finance and HR for Autonomous HEIs

45. Objectives. To prepare a Policy on Higher Education Governance and Finance, a Cambodian Higher Education Road Map, and its first-phase Higher Education Action Plan and to prepare a draft regulation on Public Autonomous HEIs.

46. Key outputs achieved. Policy on Higher Education Governance and Finance for Cambodia; Cambodian Higher Education Road Map 2030 and Beyond; Higher Education Reform Action Plan 2015–2018; Draft Higher Education Action Plan 2018–2022; Draft Regulation on HE Governance and Finance; Functional Review of the ACC; and Functional Review of the DHE and DSR.

47. Key factors affecting successful timely achievements of objectives. Difficulties in finding suitably qualified and experienced international technical assistance led to delays in finalizing the Policy on Higher Education Governance and Finance, but it was finished and adopted before the end of the project.

48. Sustainability after HEQCIP has finished. Approval of the policies and the long-term road map with its action plan provides a clear view of future direction and can be used to lobby for more funding from the government and development partners. The consultation workshops and broader consultative processes initiated as part of policy preparation have increased cooperation and collaboration within the DHE as well as between the DHE, HEIs, and other agencies responsible for HE. This has initiated a climate of increasing trust and transparency within the sub-sector, which strengthens the foundation for future development.

49. Key enabling factors. Full support from the project management and willingness of HEIs to actively participate in meetings and workshops to share their ideas and inputs.

4.4: Project Management

50. Objective. Maintain oversight of all project activities to ensure transparency, good governance, accountability, and adherence to World Bank as well as MEF and MoEYS regulations. Provide timely,


of 71

effective overall direction and management to the HEQCIP implementation teams to achieve project targets and objectives.

51. Key outputs achieved. Results Framework successfully restructured; project extension successfully negotiated; budgets monitored and expenditure managed to ensure maximum efficient use of project funds in line with project priorities; procurement of goods, works, and services planned and completed in line with project needs; semiannual progress reports submitted to the World Bank on time; project management team meetings held as required to monitor general project progress; Good Governance Framework complied with and updated every six months; a Sub-Sector Working Group for Higher Education composed of staff from MoEYS, selected HEIs, and development partners established; and a number of key fundamental policy documents to support the development of a cohesive direction for the Sector produced. These are as follows:

(a) Higher Education Vision 2030. This document was approved at the highest levels of MoEYS and covers the entire HE Sector. It is aligned with Cambodia’s IDP and its National Strategic Development Plan and has become the guide for public policy intervention in HE to support Cambodia’s ambition to become a middle-income country and knowledge-based society by 2030.

(b) A Policy on HE Governance and Finance with Draft Regulations on HE Governance and Finance

(c) A Road Map for HE and accompanying Action Plan

(d) Long-term plan for HE

(e) Governance and Finance Operations Manual for Autonomous HEIs prepared for the legislative process

52. Lessons learned. The project’s attention to strengthening structures and capacity at central and HEI levels had a positive impact and was valued by all stakeholders. Difficulties arose where problems were not addressed on time. A key lesson learned is to always address problems as they arise in an open and transparent manner and always discuss problems with the Task Team Leader in the spirit of solving problems together.

53. Enabling factors. Excellent relations between the World Bank, other development partners, MEF and MoEYS. At the project level, the main enabling factor to achieve success has been the dedication of the Project Manager. He has the personal strength of character to be an excellent leader and manager, able to deal with problems and be fully in control of what needed to be done. Productive and supportive relations with the MEF and support from the World Bank, particularly the Task Team Leader and Finance and Procurement Specialists, were also a crucial enabling factor for the success of the project. Timely provision of technical support from the World Bank-funded consultants ensured successful implementation of Component 2. The HEI and DHE staff learned a great deal from them. The project extension enabled the project to further strengthen the sub-sector with an increased focus on response to the ESP, HE Sector Reform Initiative, and new IDP. Well-organized, committed administrative technical teams understood their roles and ensured effective, high-quality implementation of their assigned duties.


of 71

Support from United Nations Educational, Scientific and Cultural Organization, Swedish International Development Cooperation Agency, and other development partners were also key factors.

54. Sustainability. The Higher Education Governance and Finance Policy and Draft Regulations on HE Governance and Finance present a strong framework for future development.

55. Production of an HE Governance, Finance, and HR Manual to support autonomous HEIs will be a key sustainable operations guideline for the sector in the future. The M&E Operations Manual provides a sustainable guide to results-based management. The HEMIS, while still in basic format, is a sustainable mechanism to support future detailed monitoring of the HE sector as a whole. The Higher Education Road Map and Action Plan offer clear guidance for future development. Provision of technical resources to HEIs participating in Component 2 enable them to improve the relevance and quality of teaching and learning.

56. Conclusion. A total of 4,954 (31 percent) staff from MoEYS and HEIs participated in Component 4-related training programs, which included education planning/strategic planning; procurement; FM, budgeting, and audit; M&E and results-based management; HEMIS; and production of the HE Road Map and action plan. The capacity developed by those who participated in the training and other component activities leaves a lasting legacy for the individuals who benefited personally and also contributes to strengthening skills, knowledge, and experience within the sector. Similarly, local and international networks established through Component 1 will continue to sustain professional development for many years to come. The emergence of a research culture initiated through Component 2 has already resulted in the establishment of research policies and centers in some of the HEIs that benefited from Component 2, and this, together with the founding of the CCI, the Research Society, and the new international networks formed through both Component 1 and Component 2 activities, will support future growth of research within the sector. The SPSs in Component 3 established a mechanism for expanding participation from disadvantaged groups. This mechanism can be replicated in the future should the government wish to expand the service. The World Bank impact evaluation of this intervention will be an important factor in supporting its expansion. Another key factor contributing to the success of the project was the country context. The Royal Government of Cambodia has a comprehensive sector strategy to address challenges in education provision. The National Poverty Reduction Strategy, Industrial Policy, and the Higher Education Reform Agenda as well as recent increases in funding for HE have placed an emphasis on strengthening the sub-sector to meet the demands of economic development. There is also an environment that encourages productive relations between the development partners and MoEYS. Regular meetings are held at the Joint Technical Working Group to discuss strategic issues and monitor achievement against objectives. The annual Joint Education Sector Review adds an analytical dimension to the annual Education Congress. This leads to clear comprehensive approaches to HE provision and quality improvement, as well as the development of realistic ESPs, challenging but achievable goals, and a policy framework against which to monitor achievement. The results against performance indicators and project outputs show that the project exceeded or met the majority of the targets. The sections on sustainability evidence that the majority of the project interventions have potential for sustainability. Therefore, based on project performance, it can be said that HEQCIP overall has successfully met the PDOs.


of 71

ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) Project Appraisal Document, World Bank, 2010

Aides-memoire and Implementation Status Reports 2011–2017

Impact of Financial Aid on University Enrollment, Retention and Completion in Cambodia, 2017

Kingdom of Cambodia, Ministry of Education, Youth and Sport. 2017. Implementation Completion Report, Higher Education Quality and Capacity Improvement Project.

Greatbatch, D., and J. Holland. 2016. Teaching Quality in Higher Education: Literature Review and Quality Research.

Henard, F. 2010. Learning Our Lesson: Review of Quality Teaching in Higher Education. Organisation for Economic Co-operation and Development and Programme on Institutional Management in Higher Education.

Ryan, T. 2015. Quality Assurance in Higher Education: A Review of Literature.

Ayres, D. M. 2000. Anatomy of a Crisis: Education, Development, and the State of Cambodia, 1953-1998. Hawaii: University of Hawaii Press

Prasertsri, S. 1996. Rebirth of the Learning Tradition: A Case Study on the Achievements of Education for All in Cambodia. Phnom Penh: United Nations Educational, Scientific and Cultural Organization (UNESCO).


of 71

ANNEX 7. DETAIL OF MODIFICATIONS TO RESULTS FRAMEWORK DURING OCTOBER 2014 RESTRUCTURING

Original Indicators Adjustments at the Time of October 2014 and September 2015 Restructuring

PDO Indicators Adjusted PDO Indicators 50 percent of sub-projects approved under the first round of proposals

Mistakenly left off the Results Framework in the 2014 restructuring paper. Officially reintroduced during the 2015 restructuring. Was always being monitored by DHE.

Annual retention rate of 95 percent of project-supported scholarship recipients

Wording modified to ‘average annual retention rate’ rather than ‘annual retention rate’ to measure average retention rate over the five-year project period. (2014 restructuring)

40 percent of HEIs supported under Components 1 and 2 showing an increased aggregate score against a modified subset of ACC accreditation criteria

Added during the 2014 restructuring to provide a more robust measure of quality within the PDO.

Intermediate Indicators Adjusted Intermediate Indicators Component 1 Number of staff who participate in training courses related to HE quality, management, assessment, and accreditation, English or computer proficiency

Number of staff (MoEYS and HEIs) who participate in training courses which improved their professional competencies. (added during the 2014 restructuring)

Number of staff who receive a Master’s Degree in Education Management

Number of staff (MoEYS and HEIs) who were conferred Master or Doctor degrees in academic specialty, educational management or related fields. (added during the 2014 restructuring))

Number of faculty at HEIs who receive a Master’s Degree in an academic specialty

Dropped during 2014 restructuring

Number of staff who receive training in areas related to: university administration, FM, procurement, curriculum development, or research methods

Dropped during 2014 restructuring

Component 2 Proportion of HEI academic staff in participating institutions completing training in research methods.

Number and percentage of HEI academic staff in participating institutions completing training in research methodologies and design (modified during 2014 restructuring)

Proportion of eligible HEIs submitting proposals to RGMC competing for funds

Number and percentage of HEIs submitting proposals to RGMC competing for funds. (modified during 2014 restructuring)

Proportion of complete research projects (competitively selected) funded by the project

Number and percentage of complete research projects (competitively selected) funded by the project(modified during 2014 restructuring)


of 71

Original Indicators Adjustments at the Time of October 2014 and September 2015 Restructuring

Within RGMC/DHE, proportion of proposals reviewed, evaluated and awarded in accordance with project processing timeline.

Within RGMC/DHE, number and percentage of proposals awarded after review and evaluation in accordance with project processing timeline. (modified during 2014 restructuring)

Component 3 Share of scholarship beneficiaries who receive stipend funds in their bank accounts within a month of schedule payment/transfer date

Dropped during the 2014 restructuring

Pass rate of scholarship recipients at each grade categorized by female, rural, poor

Pass rate of retained scholarship recipients at each grade categorized by female, rural, poor (not cumulative)

Dropout rate of scholarship recipients categorized by female, rural, poor

Dropout rate of scholarship recipients categorized by female, rural, poor

Completion rate of scholarship recipients categorized by female, rural, poor

Completion rate of scholarship recipients categorized by female, rural, poor

Component 4 Number of project management team staff who have received training on monitoring and evaluation


Number of project management team staff who have received training on financial management and procurement


Higher Education Governance and Finance Policy Paper adopted by the MoEYS (added during 2014 restructuring)

Higher Education Vision and Strategy 2030 adopted by the MoEYS (added during 2014 restructuring)

HEMIS developed and piloted (added during 2014 restructuring)


of 71

ANNEX 8. DETAILS OF PDO RELATED TO ACC ACCREDITATION CRITERIA PDO Indicator: ‘40 percent of HEIs supported under Components 1 and 2 showing an increased aggregate score against a modified subset of ACC accreditation criteria’—Achievement Details of the 36 Participating Project HEIs

Indicators (36 Project HEIs Participated)

Baseline 2010 (HEIs Scoring

Yes)

Actual August 2017 (HEIs

Scoring Yes)

HEIs Showing Increased Score

over Baseline Standard 1: Governance, Planning, and Evaluation. This standard is defined as: the fact that a good governance system in an institution enables the successful accomplishment of the institution’s vision, mission, and goals. The institution’s governance system includes the governing board and its institutional leaders. They play an important role in defining vision, mission, and goals; developing strategic plans for the development of the institution in compliance with relevant decrees and regulations; and identifying governing mechanism and priority to be addressed. Substandard 1-1: HEI results-based plan developed and submitted to MoEYS

0.0 21.0 21.0

Substandard 1-2: HEI annual operation plan developed and submitted to MoEYS

11.0 31.0 20.0

Substandard 1-3: HEI M&E framework developed and submitted to MoEYS

0.5 20.5 20.0

Standard 2: Academic Program. This standard is defined as: the achievement and maintenance of quality academic programs is the primary responsibility of any institution of HE. As conditions and needs change frequently, the institution should review and evaluate its curriculum on a regular basis in response to the national context and the changing labor market. Planning for new academic programs should be based on a regular assessment of its mission and goals, students’ needs, and its relevance to national development goals. The curriculum must clearly define the expected learning outcomes and students’ competencies. Program learning outcomes are appropriate to the level of qualifications awarded and are consistent with the National Qualifications Framework. The institution shall have appropriate strategies for the delivery of the academic programs, student assessment, and record keeping for students. They are central to sustaining the quality of the programs. This also includes an internal quality assurance program that indicates the institution shows its commitment to quality assurance and continuing improvement by systematically evaluating the effectiveness of all aspects of its operations and academic programs. The institution uses the results of these evaluations in planning, budgeting, establishing priorities, and improving academic programs and services. Substandard 2-1-1: Committee for curriculum development established, implementation plan developed, funds allocated, and progress report submitted to MoEYS

18.8 31.8 13.0

Substandard 2-1-2: Each HEI has a handbook on curriculum publicly available

36.0 36.0 0.0

Substandard 2-2-1: Research promotion policy with allocation of HR, financial resources and facilities to support research developed

2.0 16.0 14.0

Substandard 2-3-1: IQA function established, physical office set up with implementation plan developed, funds allocated, and progress report submitted to MoEYS

2.0 24.8 22.8


of 71

Substandard 2-3-2: IQA framework with standard guidelines and key performance indicators developed and introduced within HEI

1.5 25.0 23.5

Standard 3: Academic and Administrative Staff. This standard is defined as: the recruitment, selection, hiring, and retention of teaching staff and their continuous professional development are of paramount importance to every HEI. The central responsibility of the teaching staff is for the achievement of academic programs and the assurance of educational quality. To facilitate this responsibility, the teaching staff should be empowered with academic freedom and have key roles in decision-making processes that affect the educational programs of the institution. Substandard 3-1: HEI capacity development plan created and funds allocated

6.5 31.5 25.0

Substandard 3-2: HEI policy on incentives for capacity building developed

6.0 29.0 23.0

Standard 4: Financial Management. This standard is defined as: the institution demonstrates present and future financial stability to accomplish its mission effectively and to ensure the sustainability of its academic programs and services. Policies and procedures for budgetary planning, expenditure control, accounting, and external audits are in place. Substandard 4-1-1: Strategic financial plan in place

0.0 13.0 13.0

Substandard 4-1-2: Annual financial report reflecting the annual operational plan in place

0.0 13.0 13.0

Standard 5: Dissemination of Information. The institution defines the characteristics of the students it seeks to enroll and provides an atmosphere that contributes to their cultural, social, moral, intellectual, and physical development. Student services and programs are based upon an assessment of students’ needs, support the achievement of their educational goals, and are consistent with the institution’s mission, and the information is shared publicly along with a management information system with university information. Substandard 5-1: Information disseminated publicly and the information shared publicly

36.0 36.0 0.0

Substandard 5-2: HEMIS established 0.0 24.0 24.0


of 71

ANNEX 9. LIST OF COMPLETED SUB-PROJECTS

SGA # Project Name HEI Closing Date Publication of Research Status

001 Industry Based Curriculum Development on Financial Services

BBU April 30, 2014 Report published internally for internal use and further research

002 Climate Change Adaptation towards curriculum Development

BHI June 30, 2014 Report published internally for internal use and further research

003 Reducing Soil Fertility Decline in Upland Cropping Systems by Using Crop Rotations in Prey Veng and Svay Rieng Provinces

CSKU October 20, 2015

International Journal on Environment and Rural Development

004 Improving Rice Post Harvest Management to Improve Poor Farmers’ Livelihoods in Cambodia

CSKU May 31, 2015 International Journal on Environment and Rural Development

005 Promotion of the Efficient Use of Pico Hydropower for Rural Electrification in Cambodia (PHYREC)

ITC August 30, 2015 Report published internally for internal use and further research

006 Feasibility Study and Implementation Plan for a Locally-Produced Fluoride Toothpaste for Use in School Preventive Programs in Cambodia

IU June 30, 2014 Report published internally for internal use and further research

007 Improve Cattle Production System for More Efficient Cow-Calf Production in Kampong Cham Province

KCNSA September 30, 2014

Report published internally for internal use and further research

008 Food Security Improvement in Cambodia through Applying the Science and Technology in Environment-Friendly Agriculture (EFA)

MCU August 31, 2015 Report published internally for internal use and further research

009 Achieving Research Sustainability in NUM: Setting-up of a National Inland Waterway / Maritime Socioeconomic Activities Observatory

NUM September 30, 2015

Internal publication of 3 books on Stakeholder Analysis and Lessons Learned from the Sub-project

010 Develop the Research Team through Intensive Training and Learning from Regional Best Practices in Social Sciences

NUM June 30, 2014 Internal publication of book on Regional Best Practices on Research Capacity Building

011 Rice Fish Farming Adaptation in Rain-fed and Irrigated Rice Fields

PNCA August 31, 2015 Report published internally for internal use and further research


of 71


012 Innovative Approaches to Student Socialization in Cambodian HEI Sector

PPIU June 30, 2014 Report published internally for internal use and further research

013 Developing an Evidence-Based Model University Curriculum for Early Childhood Care and Development in Cambodia

PUC August 30, 2015 PUC, Foreign Teachers Cambodia, and University of London are currently in the process of drafting an MOU to run an introductory course in Childhood Speech and Language Development and six minor courses of the program and plan to begin the second batch in October 2018.

014 Green and Sustainable Transportation Study of Phnom Penh

PUC June 30, 2015 No publications but skills developed through the project are still being used by staff, for example, how to develop questionnaires, select appropriate research methods, conduct data collection, and write reports

015 Integrated Climate Change into Curricular Development for Agriculture University in Cambodia

RUA August 31, 2015 Publication of 5 books on climate change curriculum for internal RUA use and use by 7 other HEIs 1. Basics of Climate Change 2. Climate Change Adaptation 3. Climate Change Mitigation 4. Legal Frameworks Support to Climate Change The HEIs that use the curriculum and books are: RUA, CSUK, MCU, SRU, UBB, PLNCA, and KCNCA


of 71


016 Promoting Organic Innovation through Introduction of Ecological Aquaculture for Food Security and Livelihood for Cambodian Youth

RUA September 30, 2015


017 Development and Implementation of the Doctor of Veterinary Medicine Curriculum in the Royal University of Agriculture

RUA October 31, 2015

Curriculum document peer reviewed and produced for internal use

018 Increase Learning Quality and Bring International Standards to Archeology Bachelor’s Degree Studies in Cambodia

RUFA June 30, 2015 Curriculum document peer reviewed and produced for internal use

019 Opportunities and Risks of ASEAN 2015 for Cambodia

RULE June 30, 2015 Report published internally for internal use and further research

020 Improve the Quality in Teaching and Learning Biochemistry at RUPP

RUPP February 28, 2015


021 Transforming Teaching Methodologies at Royal University of Phnom Penh (RUPP) and Svey Rieng University (SRU) from Lecture to Inquiry

RUPP August 31, 2014 Report published internally for internal use and further research

022 Renewable Energy: Curriculum Development in the Physics Department

RUPP June 30, 2015 Report published internally for internal use and further research

023 Mathematical Modeling, Simulation, and Optimization in Science and Engineering


024 Harmonizing Human Society and Nature for Sustainable Development: A Symbiosis on the Development of Integrated Approaches for Improved Environmental Governance and Community Livelihood Strategies in Northeastern Cambodia


025 Rural Livelihood Improvement through Learning and Sharing Knowledge of Model Farmers in Cambodia: Case Study in Tunlorb District, Takeo Province

RUPP June 30, 2015 Results included in articles posted to the Nagoya University Forum

026 Improving Fruit Quality and Quantity in Huanglongbing (HLB) Infected Sweet Orange through Antibiotic Injection and Follar Fertilization

UBB October 31, 2015



of 71


027 Development of Approaches to Minimizing Losses, Maximizing Yield, and Reducing Market Risk of Cassava in Cambodia (DAMLOMY Project)

UBB October 31, 2015


028 Modeling a Postgraduate Training in Health Research at the University of Health Sciences

UHS August 31, 2015 Report published internally for internal use and further research

029 Analysis of Mercury and Other Toxic Compounds in Skin Whitening Creams by X-Ray Fluorescence

UHS August 31, 2015 Published in International Journal. J Health Pollut.: Metal Contamination in Low-Cost Jewelry and Toys in Cambodia

030 Improving the Quality of Teaching and Learning in Higher Education in Cambodia (Curriculum Development of Tourism and Hospitality Management (THM) at University of South-East Asia, Cambodia)

USEA June 30, 2015 New Curriculum Document published for internal use

031 Poverty and Livelihoods of the Community Fisheries in Cambodia Coastal Area

HRU June 30, 2014 Report published internally for internal use and further research

032 Plant-based Extracts in Medicine Box according to National Pharmacopoeia

UHS August 31, 2015 Report published internally for internal use and further research

033 Collective Memory within the Khmer Space: Identity and Variation

RUFA August 28, 2015 Report published internally for internal use and further research

034 Development of Suitable Technologies using Municipal Solid Waste (MSW) as Solid Fuel in Cambodia

ITC September 30, 2015


035 Application of Extruded Pellet Feeds in the Grow-out of Seabass (Late calcarifer)

RUA June 30, 2015 Report published internally for internal use and further research

036 How Intangible Human Capital of Self-Esteem and Motivation Impact Their Perception on Economic, Political, and Social Environment: Using University Students as a Case Study

UC June 30, 2015 Report published internally for internal use and further research

037 Development of Khmer Language Processing Tools ITC June 30, 2015 Report published internally for internal use and further research


of 71


038 The Rise of Angkor: Archaeological and Anthropological Study of the Dangrek Range and Its Conservation

AKU June 30, 2015 Report published internally for internal use and further research

039 Postharvest Mechanization Development of Cassava in Cambodia

MCU August 31, 2015 Report published internally for internal use and further research

040 Online Support for Education and Training in Cambodia

IIC December 31, 2014


041 Studies on Land Road Accidents in Phnom Penh CUS June 30, 2015 Report published internally for internal use and further research

042 Integrated Community and University Action through Research on Vegetables and Mushroom Production - Model for Improving the Nutritional, Health, and Economic Status of Smallholders in Community

SRU June 30, 2015 Report published internally for internal use and further research

043 Community Saving through Credit Union BHI March 31, 2015 Report published internally for internal use and further research

044 Impact of Microfinance Institutions on Poor Agricultural Households: Implication for Improving the Status of Women around Great Lake

HRU December 31, 2014


045 Reduction of Chemical Fertilizer through Farm Manure Using for Rice Production on Prateah Lang Soil Type in Cambodia

KCNSA August 31, 2015 Report published internally for internal use and further research

Number of sub-projects that produced final reports on their research to be used for further research internally

40

Number of sub-projects that published domestic peer-reviewed documents to be used in-house or with other HEIs

2

Number of sub-projects that were published in international journals 3 Note: AKU = Angkor Khemara University; BBU = Build Bright University; BHI = Bright Hope Institute; CSKU = Chea Sim University of Kamchay Mear; CUS = Cambodia University for Specialties; HRU = Human Resources University; IIC = IIC University of Technology; ITC = Institute of Technology of Cambodia; IU = International University Cambodia; KCNSA = Kampong Cham National School of Agriculture; MCU = Mean Chey University; NUM = National University of Management; PNCA = Prek Leap National College of Agriculture; PPIU = Phnom Penh International University; PUC = Pannasastra University of Cambodia;; RUA = Royal University of Agriculture; RUFA = Royal University of Fine Arts; RULE = Royal University of Law and Economics; RUPP = Royal University of Phnom Penh; UBB = University of Battambang; UHS = University of Health Science; USEA = University of South-East Asia

document of the world bank...02 19-dec-2011 satisfactory satisfactory 1.43 03 07-jul-2012...

Documents