agricultural technology schools project (loan 1050-ino)

7/31/2019 Agricultural Technology Schools Project (Loan 1050-InO)

1/37

ASIAN DEVELOPMENT BANK PPA: INO 20185

PROJECT PERFORMANCE AUDIT REPORT

ON THE

AGRICULTURAL TECHNOLOGY SCHOOLS PROJECT(Loan 1050-INO)

IN

INDONESIA

December 2001


2/37

CURRENCY EQUIVALENTS

Currency Unit Rupiah (Rp)

At Appraisal At Project Completion At Operations Evaluation(August1990) (July 1999) (May 2001)

Rp1.00 = $0.000536 $0.000113 $0.00008$1.00 = Rp1,864 Rp8,875 Rp11,250

ABBREVIATIONS

ADB Asian Development BankAEP Agricultural Education ProjectATS agricultural technology schoolDTVE Directorate of Technical and Vocational EducationMOEC Ministry of Education and CultureO&M operation and maintenanceOEM Operations Evaluation MissionPCR project completion reportPIU project implementation unit

PPAR

project performance audit reportPU production unit

NOTES

(i) The fiscal year (FY) of the Government ends on 31 March.(ii) In this report, $ refers to US dollars.

Operations Evaluation Department, PE-582


3/37

CONTENTSPage

BASIC DATA iiEXECUTIVE SUMMARY iii

MAP v

I. BACKGROUND 7A. Rationale 7B. Formulation 7C. Purpose and Outputs 7D. Cost, Financing, and Executing Arrangements 8E. Completion and Self-Evaluation 8F. Operations Evaluation 9

II. PLANNING AND IMPLEMENTATION PERFORMANCE 10A. Formulation and Design 10

B. Achievement of Outputs 10C. Cost and Scheduling 11D. Procurement and Construction 11E. Organization and Management 12

III. ACHIEVEMENT OF PROJECT PURPOSE 13A. Operational Performance 13B. Financing of School Operations 11C. Sustainability 19

IV. ACHIEVEMENT OF OTHER DEVELOPMENT IMPACTS 20A. Socioeconomic Impact 20

B. Environmental Impact 20C. Impact on Institutions and Policy 20

V. OVERALL ASSESSMENT 22A. Relevance 22B. Efficacy 22C. Efficiency 22D. Sustainability 22E. Institutional Development and Other impacts 23F. Overall Project Rating 23G. Assessment of ADB and Borrower Performance 23

VI. ISSUES, LESSONS, AND FOLLOW-UP ACTIONS 24A. Key Issues for the Future 24B. Lessons Identified 26C. Follow-Up Actions 27

APPENDIXES 20


4/37

BASIC DATAAgricultural Technology Schools Project (Loan 1050-INO)

Project Preparation/Institution Building

TA No.

TA Name Type Person-Months

Amount

($)

Approval

Date

1147 Agricultural TechnologySchools

PPTA 16 362,000 19 Apr 1989

Key Project Data ($ million)As per ADB

Loan Documents ActualTotal Project Cost 119.5 86.8Foreign Exchange Cost 49.0 52.0Local Currency Cost 70.5 34.8

ADB Loan Amount/Utilization 85.0 77.2

Key Dates Expected ActualAppraisal 31 Jul-16 Aug 1990Loan Negotiations 8-10 Oct 1990Board Approval 13 Nov 1990Loan Agreement 18 Dec 1990Loan Effectiveness 18 Mar 1991 19 Feb 1991First Disbursement 20 Sep 1991Loan Closing 30 Sep 1997 15 Oct 1998Project Completion 30 Jun 1997 30 Sep 1997

Months (effectiveness to completion) 75 79

Borrower Government of the Republic of Indonesia

Executing Agency Directorate of Technical and Vocational Education in theMinistry of Education and Culture

Type of Mission No. of Missions No. of Person-DaysFact-Finding 1 92

Appraisal 1 85Inception 1 3Project Administration

Review 7 138Project Completion 1 44Operations Evaluation1 1 42

ADB = Asian Development Bank, PPTA = project preparatory technical assistance, TA = technical assistance.1

The Operations Evaluation Mission comprised Ellen Qiaolun Ye, Senior Evaluation Specialist (Mission Leader);and Samuel Mancebo (International Consultant). In addition, Soesarsono Wijandi (Domestic Consultant) conducted aschool survey.


5/37

EXECUTIVE SUMMARY

The Project1 was designed to improve the quality and efficiency of agricultural technology

education at the upper secondary level, provide equitable access to quality agricultural education(especially on outer islands), and develop entrepreneurial skills of teachers and students. TheProject comprised three components: (i) staff training and curriculum development; (ii) upgrading8 agricultural technology schools and establishing 16 new ones; and (iii) establishing agriculturalproduction units (PUs) to provide practical training in agricultural skills, and to generate revenues tosupplement school operation and maintenance (O&M).

The project cost was estimated at $119.5 million at appraisal, to be financed by an AsianDevelopment Bank (ADB) loan of $85 million from ordinary capital resources and the remainderby the Government. The actual project cost was $86.8 million, about 27 percent below theappraisal estimate, with a loan disbursement of $77.2 million and cancellation of the remaining loanproceeds. The cost underrun was attributable to free provision of land by local governments,

reduced consulting services, lower than estimated payments for project staff, and a sharpdepreciation of the local currency.

The Project supported the Government's objectives of promoting agricultural growth andbalanced regional development, and was consistent with ADB's operational strategy atappraisal when ADBs lending to education focused on vocational training and higher education.

At the time of postevaluation, the priority of ADBs lending shifted to basic education. It isargued that the public sector should not compete directly with the private sector when privateschools are more responsive to labor market demand and can produce the required skills atlower unit cost with no loss of quality. The Project achieved most of its major targets. Asplanned, 8 existing schools were upgraded and 16 new schools were constructed. All projectschools are fully functional, but many, particularly on outer islands, operate below capacity with

low utilization of teachers and school facilities. Furthermore, dropout rates are relatively highand external efficiency is low, as only half of the graduates engage in agriculture-related jobs.Nevertheless, the Projects sustainability is likely as all project schools have been operating andtheir continued operation in the future is expected. The Project established a computerizedinformation system in the project schools, provided a substantial amount of training for teachersand principals, and enhanced the capacity of the Executing Agency, the Directorate of Technicaland Vocational Education.

Overall, the Project has increased the countrys capacity of agricultural technologyeducation at the upper secondary level by about 15 percent; about two thirds of the capacityincrease has been on outer islands. The Project has improved the quality of education by providingmodern school facilities and better-trained teachers. However, the objectives of improving

educational efficiency and developing entrepreneurial skills have not been fully achieved. Onbalance, the overall project rating is successful.

Four issues arise from the evaluation. First, the analysis of demand for and supply of theproject schools was based on a static observation at appraisal without further analysis of the likelytrend after the substantial increase of student places under the Project. The reliance ongovernment forecasts for sector development and human resource needs proved to be unreliable.

1 Loan 1050-INO:Agricultural Technology Schools Project, for $85 million, approved on 13 November 1990.


6/37

The forecast for nationwide demand without sufficient attention to regional variations, in conjunctionwith a standard model of school size for all schools, has led to underutilization of school capacity inless populated areas on outer islands. Second, the PU concept was an innovation in project designwith the intention of promoting entrepreneurial skills and generating revenues for school O&M.While PUs were established in all project schools with production facilities and seed capital, not all

of them function well, and many suffer operating losses. Since most teachers and principals did nothave business experience, and the Project provided physical inputs and capital but not commercialmanagers, many PUs have not had proper management and cannot compete in markets, resultingin operating losses and underutilization of farmland and production facilities.

Third, project design included detailed O&M arrangements, such as an analysis ofgovernment budget implications, government assurance on budgetary provision, and suggestionson the use of student fees and PU revenues for school maintenance. In spite of the careful design,maintenance of school facilities is still inadequate due to lack of a regular funding source. As aresult, minor cracks in school buildings and some broken facilities have not been fixed; the latterremain out of service.

Fourth, future financial support for the project schools will likely vary by school dependingon district governments, which may provide a budget for teacher salaries but not enough to coverall school expenditures. The project schools will have to generate supplementary revenues forschool O&M. While good potential for revenue generation exists in the project schools, the currentmanagement systems and incentive schemes do not provide strong incentives nor pressure tomotivate the schools to fully utilize their resources.

The Project has provided the following valuable lessons for future improvements: (i) thesole use of government forecasts for sector development and human resource needs to estimatedemand for schools is insufficient; (ii) using only a nationwide forecast without sufficient attention tolocal demand for individual schools, in conjunction with a standard model for all project schoolsregardless of regional variations, may lead to surplus school capacity in areas with less population

or high competition from other schools; (iii) the provision of hardware and software inputs toschools can improve the quality of educational inputs such as better school facilities and bettertrained teachers, but without effective management systems and incentive schemes, the improvedinputs may not lead to improved quality of educational outputs and efficiency; (iv) while the cost ofschool maintenance is not particularly high if conducted regularly and promptly, the lack of aregular funding source and ad hoc arrangements for school maintenance may delay the requiredrepairs, leading to further deterioration of broken facilities; (v) the analysis of government budgetaryimplications from the viewpoint of a single project is insufficient as the concerned government mayhave many competing uses for its budgetary resources; (vi) government assurance on providingsufficient resources for school maintenance is necessary but insufficientmore effective measuresare needed to ensure the establishment of a reliable funding source for school maintenance afterproject completion; (vii) the provision of a large amount of one-off training under a project without a

budget for continued upgrading of teachers skills after project completion may not be costeffective; (viii) innovative project design such as the PU concept needs to be accompanied byclose supervision and flexible adjustment during implementation; (ix) the provision of a standardinvestment package for all project schools may simplify project implementation but at a long-termcost of underutilizing some facilities and the additional burden of maintaining them; (x) theestablishment of a public school needs to avoid its potential adverse impact on private schools inthe project area that serve the same set of clients; and (xi) the design of an education projectshould pay particular attention to the long-term operation of the project schools after completionwhen the schools may face funding problems.


7/37

I. BACKGROUND

A. Rationale

1. After two decades of rapid development, availability of well-trained middle-level agriculturalhuman resources became a critical need in Indonesia in the late 1980s. At the time of appraisal,substantial excess demand for agricultural technology schools (ATSs) was observed, as therewere more than 20,000 applicants for about 5,000 student places in public ATSs every year, a ratioof 4:1 (appraisal report, para. 43). The Fifth Five-Year National Development Plan of theGovernment (Repelita V, 1989/90-1993/94) called for an improvement in the quality and relevanceof agricultural education and continued development of ATSs. Equitable access to agriculturaleducation, especially on outer islands, was another priority of the Government in view of the needfor balanced regional development. With its focus on technical/vocational education and highereducation, the lending program of the Asian Development Bank (ADB) in Indonesias educationsector supported the Governments strategy. In 1983, ADB financed the first Agricultural EducationProject (AEP),1 which upgraded 18 ATSs and built 6 new ones.

B. Formulation

2. A project preparatory technical assistance2 was approved in April 1989. Completed inDecember 1989, it provided an in-depth review of the sector issues relating to agriculturaltechnology education together with a detailed project design based on the achievements of AEP. Itconcluded that AEP was being implemented well and achieving its objectives of improving thequality of agricultural technology education.

C. Purpose and Outputs

3. The primary objectives of the Agricultural Technology Schools Project3 were to (i) improvethe quality and efficiency of agricultural technology education at the upper secondary level;(ii) provide equitable access to quality agricultural education, especially on outer islands; and(iii) develop entrepreneurial skills of teachers and students. The Project comprised threecomponents: (i) educational programs development, including staff training and curriculumdevelopment; (ii) educational facilities development, including upgrading of 8 ATSs andestablishment of 16 new ones; and (iii) agricultural production unit (PU) and entrepreneurialsupport, including the provision of seed capital and production facilities for the establishment andoperation of a PU in each project school. The PU component supported a government initiative to

expand such activities to all ATSs to provide practical training in agricultural skills, and generaterevenues to supplement school operation and maintenance (O&M).

4. It was expected at appraisal that, at full operation, the 24 project schools would have astudent population of about 12,000 and produce about 3,500 graduates each year, representingabout a 15 percent increase in the capacity of ATSs in the country. Other anticipated outputs

1Loan 675-INO:Agricultural Education Project, for $68 million, approved on 20 December 1983.

2TA 1147-INO: Agricultural Technology Schools, for $362,000, cofinanced by the Japan Special Fund and theGovernment of Norway, approved on 19 April 1989.

3Loan 1050-INO:Agricultural Technology Schools Project, for $85 million, approved on 13 November 1990.


8/37

included (i) new facilities in all 24 project schools, including 16 on outer islands; (ii) better trainedteachers, school principals, and other administrative staff; (iii) a functional PU in each projectschool; and (iv) a computerized information system at the school and national levels.

D. Cost, Financing, and Executing Arrangements

5. The project cost was estimated at $119.5 million at appraisal, comprising $49.0 million inforeign exchange and $70.5 million equivalent in local currency (Appendix 1). An ADB loan of$85.0 million from the ordinary capital resources, representing 71 percent of the total projectcost, was provided to finance the entire foreign exchange cost and 51 percent of the localcurrency cost. The Government was to finance the remaining $34.5 million of local currencycost. The Directorate of Technical and Vocational Education (DTVE) under the Ministry ofEducation and Culture (MOEC) was the Executing Agency. The project implementation unit (PIU)of AEP (para. 1) was retained to be the PIU of this Project, with provincial offices of education andculture responsible for site-level project implementation and supervision. In addition, a project

steering committee and a project advisory group were to be established to provide policy directionand oversee coordination of project activities. At the school level, a school advisory committee wasto be established in each project school to provide guidance for curriculum development andoperation of the PU, and coordinate with industries and local communities.

E. Completion and Self-Evaluation

6. The Project was completed in September 1997. A project completion report (PCR)prepared by the Agriculture and Social Sectors Department (East) of ADB was circulated to theBoard in December 1999. The PCR rated the Project generally successful because (i) most targets

envisaged at appraisal were largely achieved; (ii) in spite of initial delays, the Project wascompleted largely on time; and (iii) the Project generated a set of benefits, such as increasednumber of students and graduates, well-trained teachers, and new school facilities. Problemsidentified in the PCR included (i) the training of private school teachers was not implemented,(ii) the Project suffered serious delays in the first three years, (iii) not all PUs were fully functionaland not all students had access to practical training, and (iv) inadequate maintenance of schoolfacilities and most schools required minor repair of buildings.

7. The PCR was well prepared; it discussed issues openly and provided usefulrecommendations. In particular, the PCR identified two important issues. First, project supervisionshould have been more intensive in the initial years. Second, the demand forecast for projectschools should have been realistic to minimize underutilization of school facilities. The PCR also

recommended a set of actions, including (i) financial support from MOEC and provincialgovernments to enable immediate repair of school buildings, (ii) increased student enrollment, (iii) amore diversified curriculum, and (iv) full utilization of PUs in all schools to provide all students withaccess to practical training. Due to a lack of specific measures for implementation and a lack ofmotivation in the project schools, however, these recommendations have not been implemented.


9/37

F. Operations Evaluation

8. This project performance audit report (PPAR) assesses the design, implementation,

outputs, and impacts of the Project, focusing on its sustainability and long-term developmentimpacts, and identifying lessons learned for future improvement. The PPAR presents thefindings of an Operations Evaluation Mission (OEM) that visited relevant government agenciesand conducted fieldwork in project areas in May 2001.4 The OEM draws its conclusions andrecommendations from three sources: (i) a desk review of project files; (ii) the field inspection ofschool facilities and discussions with stakeholders; and (iii) a survey of 21 project schoolsconducted in June-July 2001, including group discussions with school principals, teachers,students, parents, and tracer studies on 145 graduates, 71 dropouts, 37 companies thataccepted student internship, and 17 employers of graduates.5 Copies of the draft PPAR weresubmitted for review to DTVE, other relevant government agencies, and concerned ADBdepartments or divisions. Comments received from the reviewers were taken into considerationin finalizing the PPAR.

4The OEM visited seven ATSs and held discussions with school principals, teachers, students, parents, graduates,employers of graduates, companies that accepted student internship, and district governments that were given theresponsibility to finance ATSs. The OEM also held discussions with the DTVE, the National Development PlanningAgency (BAPPENAS), other relevant government agencies, as well as provincial and district offices of MOEC.

5For practical reasons, three project schools were excluded from the survey. These were ATS Tapak Tuan in Aceh,ATS Makariki in Maluku, and ATS Suai in East Timor.


10/37

II. PLANNING AND IMPLEMENTATION PERFORMANCE

A. Formulation and Design

9. Building on the achievements of AEP, the Project was designed in accordance with theGovernment's objectives of promoting agricultural growth and balanced regional development,and was consistent with ADB's operational strategy at the time of appraisal. The project designwas innovative as it included training for teachers from private schools with a view to supportingprivate sector provision of vocational education, and the establishment of a PU in all projectschools to develop entrepreneurial skills of students. At project approval, the Board expressedits appreciation for the careful design of the Project and its envisaged contribution to balancedregional development.

10. After assessing the outputs and development impact of the Project, however, the OEMidentified the following weaknesses in project design. First, the estimation of demand for theproject schools was based on static data observed at appraisal as well as the Governmentsplan for sector development and human resource needs. After a substantial increase of studentplaces under the Project, the excess demand for public ATSs observed at appraisal wasreplaced by a surplus of school capacity, leading to underutilization of school facilities.

11. Second, the demand forecast was conducted at the national level and a standard modelof 500 students per school was adopted for all project schools, regardless of the localconditions. While the size of the model school fits well the conditions in Java, many schoolslocated in less populated areas on outer islands do not have enough student applicants.

12. Third, project design focused on the provision of hardware and software inputs such asdevelopment of curriculum, training of teachers and principals, construction of new schools, andprovision of teaching and learning facilities. Without the establishment of effective managementsystems in the schools, the facilities and teachers have not been fully utilized.

13. Fourth, project design provided detailed O&M arrangements, including an analysis of thegovernment budget implication, which concluded that the required budgetary allocation for all ATSsrepresented only 0.7 percent of MOECs routine budget. Project design also suggested the use oftuition fees and PU revenues for school maintenance. Government assurance was obtained on theprovision of adequate staff, funds, facilities, services, and other resources required for sufficientand timely maintenance of project facilities. In spite of the careful planning, inadequatemaintenance of school facilities has still occurred, primarily due to a lack of a reliable funding

source.

B. Achievements of Outputs

14. Appendix 2 provides a summary of the major project outputs achieved versus theappraisal targets. Detailed discussions of the achievements by project component are given insection III. Overall, most of the project components and activities were implemented as plannedexcept for training of private school teachers. A total of 8 schools were upgraded and 16 newlyconstructed. Eight of the schools were located in Java, and 16 on the outer islands (see Map).The PUs were established in all schools, but are not fully functional in some schools. At


11/37

completion, the total number of students in the 24 project schools reached 11,386, only5 percent below the appraisal target of 12,000. The project schools produced 3,458 graduatesper year, or 99 percent of the appraisal target of 3,500. These indicators, however, deterioratedafter project completion. The OEMs school survey found that, on average, the surveyedschools had 408 students and produced 130 graduates each year, or 18 and 11 percent,

respectively, below the appraisal targets of 500 students and 146 graduates per school(Appendix 2).

C. Cost and Scheduling

15. There was a significant cost underrun on the Project (Appendix 1). The actual project costwas $86.8 million, about 27 percent below the appraisal estimate of $119.5 million. Actual loandisbursement was $77.2 million against an original provision of $85.0 million. The remaining$7.8 million of loan proceeds was canceled at loan closing. The major causes of cost underrunand loan cancellation were (i) free provision of land by local governments, (ii) reduced

consulting services due to a long delay in their commencement, and (iii) lower than estimatedpayments for PIU staff. In addition, a substantial depreciation of the local currency contributed tothe cost underrun in US dollar terms. Consequently, the Government's financing was reducedfrom $34.5 million envisaged at appraisal to $9.6 million at project completion.

16. Project implementation suffered significant delays in the first three years. In particular, therecruitment of international and local consultants was delayed by three and two years, respectively.The OEM identified the following causes of the delays. First, the PIU was busy implementing anew government policy of a dual-tech system (Pendidikan Sistem Ganda) that combinededucation with training, and failed to give full attention to the Project. Second, the new policydirection of the dual-tech system necessitated adjustments in the expert mix of the consultingservices, which required additional time. Third, there were disagreements between the

Government and ADB about the consultant inputs; the former requested the use of domesticconsultants and the latter insisted on the original plan of using both international and localconsultants. It took time to reach the final consensus of following the appraisal plan. After theslow start, implementation accelerated in later years and the Project was completed in September1997, just three months behind schedule. The loan was closed on 15 October 1998, one year afterthe original closing date, mainly due to the slow return of remaining project funds in the imprestaccount.6

D. Procurement and Construction

17. A standard package of school buildings and equipment was provided to each of the24 project schools. The equipment was procured through international competitive bidding orinternational shopping in accordance with ADB guidelines. Library books and teaching materialswere procured through local competitive bidding acceptable to ADB. The PCR reported thatmost of the equipment was delivered on schedule and in accordance with specifications.

18. Civil works contracts were awarded to prequalified contractors. While the quality of theconstruction was generally good, minor problems such as leaking roofs, cracks in walls, andincomplete construction were reported in isolated cases, largely due to poor quality of

6The delay was due to slow government procedures compounded by the financial crisis in 1997-1998.


12/37

construction materials, insufficient construction supervision by provincial staff and consultants,and inadequate maintenance.

E. Organization and Management

19. Retaining the existing PIU in charge of AEP had the merit of having experienced staff inplace before project commencement. However, this arrangement resulted in a two-year overlap inimplementing both AEP and this Project, mainly due to the delayed completion of AEP. In addition,the PIU was busy implementing the new dual-tech system and failed to pay sufficient attention tothe Project in the initial stages (para. 16). The performance of PIU improved significantly in the lateryears, resulting in accelerated implementation.

20. The project steering committee and project advisory group (para. 5) were established asplanned and provided useful guidance for implementation. However, weak coordinationbetween DTVE and another government agency resulted in non-implementation of the training

for teachers from private schools (para. 26). At the school level, the school advisory committeeswere established in most but not all project schools. Their operations varied by school,especially in terms of providing guidance to the PUs and building linkages with local industries.

21. The start of consulting services suffered a long delay, which prevented the consultantsfrom providing technical inputs for the formulation of the 1994 curriculum. The delayed arrival of theconsultants also limited their assistance in the preparation of educational architectural briefs, whichwere needed for land acquisition and civil works planning. Thereafter, the consultants performedin a satisfactory manner. They did not contribute to the development of the 1994 curriculum butassisted in detailing its modular contents as well as in preparing its 1999 revision.

22. The OEM concurs with the PCR assessment that the Borrower largely fulfilled the major

loan covenants with some exceptions, such as the training of teachers from private schools. Theestablishment of a monitoring system was partially implemented, as the system was establishedbut the tracer studies required were not conducted. The government assurance on providingsufficient resources for school O&M was not fully complied with, as the government budgetcovered teachers salaries but not school maintenance.

23. ADB fielded one loan review mission every year during the project period. As DTVE wasimplementing five or six ADB loans simultaneously during most of the project period, ADB's loanreview missions usually conducted reviews of several projects within short mission duration.7Insufficient supervision by ADB partly contributed to the slow project implementation in the earlyyears, as the back-to-office reports of review missions did not discuss outstanding issues in atimely manner.8 A significant improvement was observed in the later review missions, which spent

time visiting project schools and identified issues. In particular, the midterm review missionprovided a good analysis of implementation issues with sound recommendations.

7For example, the first loan review mission reviewed four loans in its nine days of visit; the second mission reviewedthree loans in its five days of visit.

8For example, the significant delay in consultant recruitment was not reported in a timely manner; its underlying causeswere not analyzed until the PCR stage.


13/37

III. ACHIEVEMENT OF PROJECT PURPOSE

A. Operational Performance

1. Educational Programs Development

24. This component provided a substantial amount of budget for staff training, including1,872 person-months of overseas training for 204 teachers, principals, and other administrativestaff as well as 20,287 person-months of domestic training for 3,350 staff. About 15 percent of thedomestic training was to be allocated for the training of 500 teachers from private schools tosupport private sector participation in vocational education.

25. Most training activities were implemented successfully (Appendix 2). At project completion,342 staff had received overseas training, totaling 1,874 person-months. This included skills trainingfor 214 teachers ranging from two weeks to eight months, six months of training on schoolmanagement for 60 principals, and study tours to 11 countries for 68 participants. The domestictraining was provided to 2,999 teachers as well as 414 technicians and administrative staff for atotal of 20,338 person-months. DTVE and the project schools reported that the trainees enhancedtheir knowledge and skills of modern agricultural technology and school management; manyparticipants applied the new knowledge and skills in their work.

26. The OEM identified several problems. First, the training of private school teachers wasnot implemented, largely due to weak coordination between DTVE and the Directorate of

Private Schools, which was also under MOEC and in charge of training for private schoolteachers. Furthermore, PIU was busy catching up with the project schedule after the substantialinitial delays, paying little attention to private schools. Second, the PCR reported limitedapplication of the new knowledge and skills in some fields such as fisheries and horticulture,mainly due to the standard curriculum that provided few opportunities for introducing newsubjects and skills. Third, the current management systems in the project schools, especiallythe low and fixed salaries, do not provide strong incentives to teachers to fully utilize their skillsafter training. Group discussions with teachers in the project schools revealed that manyteachers had other jobs outside school, and consequently could not give their full attention toteaching and learning activities.9

27. In retrospect, the training component might have been more cost effective if the training

budget had been used for local study tours, for example, to visit the best ATSs, the best PUs,the best internship programs, or the ATSs with the best employment of graduates. The studytours could have allowed the participants to exchange experience in school management, inteaching and learning activities, and in PU operations, and developed measures forimprovement. Annual awards could have been granted to the best performers to encouragecompetition among schools, teachers, and principals.

9The teachers considered the additional jobs necessary to supplement their low salaries.


14/37

2. Educational Facilities Development

28. This component was to upgrade 8 existing ATSs and build 16 new ATSs. The upgrading ofATSs included site improvement, farm improvement, rehabilitation of buildings, construction of newbuildings, and improvement of water and electricity supply. The construction of new ATSs includeddevelopment of school sites and farm sites as well as construction of new buildings and farms.Student dormitories and staff houses were provided for schools in remote areas. In addition, allproject schools were to receive a set of equipment, farm machinery, vehicles, furniture, librarybooks, and instruction materials. It was expected that each project school would have a studentpopulation of 500 at full operation. The standard school size was considered appropriate forutilizing the school facilities, and was similar to the size of many ATSs in Java at the time ofappraisal. For schools on outer islands, it was expected that student enrollment would increase asthe population grew under the Governments plan for balanced regional development.

29. This component was implemented as planned: 8 ATSs were upgraded and 16 wereconstructed. Six of the 8 upgraded ATSs are in Java and 14 of the 16 new ATSs are on outerislands (see Map). The OEM confirms the PCR assessment that most campuses were welldesigned with beautiful landscapes, and the quality of most construction was generally good. Theappraisal targets of students and graduates were largely fulfilled at project completion(Appendix 2).

30. Due to declining student enrollment, however, the number of students and graduates fellto about 18 percent and 11 percent, respectively, below the appraisal target at the time of theOEM (Appendix 2). Based on the survey of 21 project schools, the average number of studentswas 408 in 1999, with a large variation ranging from 130 students in ATS Jembrana to637 students in ATS Jalaksana. The upgraded ATSs performed generally better than the new

ATSs. Appendix 3 shows that the former had an average of 527 students in 1999 as comparedto the average of 335 students in the new ATSs. Of the 13 new ATSs surveyed, 11 had fewerthan 400 students, including 2 that had fewer than 200 students.

31. The primary cause of the above shortfalls was an insufficient number of applicants,especially on outer islands. Appendix 4 shows that, while applicants increased in all but one of theupgraded ATSs during the period 1995-1999, all but one of the new ATSs experienced decliningnumbers of applicants in the same period. In 1999, 4 of the 13 new ATSs surveyed had fewer than100 applicants. The worst cases were found in ATS Takisung and ATS Manna in 1999, where thenumber of applicants dropped to 11 and 15 percent, respectively, of their 1995 level.

32. The survey results suggest that the location of an ATS was crucial to student enrollment, as

the upgraded ATSs in Java had higher student enrollment than most new ATSs on outer islands.Appendix 5 shows that in 1999, the upgraded ATSs enrolled an average of 201 new students,about 63 percent higher than the average of 123 new students in the new ATSs. Moreover, 6 of the13 new ATSs surveyed enrolled fewer than 100 new students in that year. Discussions withteachers and students revealed that the availability of nonagricultural employment opportunities ina school area was an important factor influencing student enrollment, together with other factorssuch as population density, agricultural growth, and perceived earnings from agriculturalemployment. It was found that ATSs were not particularly attractive to many students, who weremore interested in vocational training in other sectors. For example, ATS Jembrana in Bali enrolled

10Six of the eight upgraded ATSs were in Java and 14 of the 16 new ATSs were on outer islands.


15/37

only 29 new students in 1999 and had a total student population of 130. Since this ATS waslocated in an area with tourist industry growth, it was unable to compete with other vocationalschools.

33. While the number of applicants declined, the number of student places in public ATSs

increased substantially after the Project, which added 500 places in each of the 24 project schools.As a result, the ratio of applicants to new students dropped from 4:1 at appraisal to about 1.2:1(Appendix 6). In particular, 5 of the 13 new ATSs surveyed had a ratio as low as 1.0:1 in 1999.

3. Production Unit and Entrepreneurial Support

34. This component had dual objectives: to develop entrepreneurial capability of teachersand students and to generate additional revenues to support O&M of the project schools. It wasenvisaged at appraisal that, by providing farmland, machinery, other facilities, animals, andseed capital, the PU would be established and become operational in each project school.

School cooperatives would market farm products produced by the PUs, and provide financialassistance to teachers, students, and graduates for their farming activities.

35. The PUs were established as planned; each project school received a standard packageof farm machinery, farm buildings, production facilities, and start-up capital. Local governmentsdonated farmland, the area of which was small in schools in Java and large on outer islands.The actual PU operation, however, varied significantly across project schools. The PCRreported good performance of PUs in some schools where principals and teachers wereproactive in exploring ways to build up linkages with private companies and generate sizablerevenues, but found significant underutilization of PUs in others. The OEMs field visits and theschool survey found that many PUs were still in an early stage of operation with few productionactivities. A more worrisome finding centered on the financial difficulties that many PU

experienced, as they had lost their start-up capital provided by the Project due to operatinglosses. The school cooperatives envisaged at appraisal were not established; students andgraduates had no access to credit services.

36. The OEM identified a set of factors underlying the above problems. First, not manyschool principals and teachers had business experience or were strong in a competitive market.Second, some PUs faced physical difficulties, such as poor quality of farmland and lack ofaccess to irrigation. Some ATSs on outer islands also faced marketing difficulties due to theirisolated location. Third, all schools used currently employed teachers to part-time manage thePUs instead of hiring a full-time manager with business experience, as the Project did notallocate a budget for the hiring of managers and the revenues from PU operations were too lowto finance such a position. Fourth, most PUs did not have sound management systems for

operational planning, marketing, or financial control.

37. In retrospect, this component might have performed better if it had provided a budget tohire commercial managers with business experience to manage the PUs for at least two orthree years initially. A contract could have been signed between a project school and a hiredmanager to require the establishment of operational strategy and management systems for thePU, and require the PU to generate sufficient income to finance its operations after the initialsupport. Since this component was an innovation, a process approach could have beenadopted to allow for timely identification of issues, such as the financial losses of PUs, emergingduring project implementation, and appropriate timely adjustment in project design. In fact, itmay still be possible today for some project schools to contract out their PUs to interested


16/37

commercial firms, letting them use the PU facilities such as farmland and machinery, requiringthem to provide students with practical training, and submitting a portion of their profits to theschools for school maintenance when appropriate.

4. Internal Efficiency

a. Utilization of School Facilities

38. The project schools have been functional and most school buildings have been used. TheOEM observed that some laboratories had been converted into conference rooms because theywere too big for the small classes in the project schools. In the schools visited by the OEM,equipment was frequently used and laboratories were functional. However, some farmmachinery was idle or rented out for outside services as it did not well match the needs of theschools. For example, each school received the same set of large, medium, and small tractors.But the large tractors were too bulky for the small farms in the ATSs in Java. In contrast, the

hand tractors were too small for the schools on outer islands as these had large farms. The rootcause of this issue was the standard investment package to each project school with no tailoringof the investment to the specific needs of individual schools, which varied significantly due to thelarge regional variations in the country.

b. Utilization of Teachers

39. The OEM observed significant underutilization of teachers in the project schools. Theresults of the school survey showed that, on average, a teacher in the project schools taught13 students (Appendix 7). This ratio was substantially below the norm of 36 students perteacher for general and adaptive subjects and 16 students per teacher for production subjectsas recommended by MOEC. Appendix 7 shows that 6 of the 21 project schools surveyed had

an average student/teacher ratio of less than 10 in the period 1995-1999. The lowest ratio wasfound in ATS Jembrana, where each teacher had an average of only three students in 1999. Inthe schools visited by the OEM, many teachers taught only one or two courses each year, lowerthan the minimum of three courses per teacher per year recommended by MOEC.

40. A number of factors have contributed to this issue. First, ATSs were specialized schoolsthat required more subjects than general high schools. The standard size of 500 students perschool was on the low end of the economic scale for utilizing the specialized teachers. Second,the school survey found that many new schools on outer islands could not meet the target of500 students, primarily due to insufficient number of applicants. Third, the project schools wereused to relying on government budget allocations without strong incentives or pressure toimprove efficiency. In two schools visited by the OEM where a teacher had an average of fewer

than 10 students, the school principals and teachers still considered that there was a shortageof teachers in their schools.

c. Graduation and Dropout Rates

41. The results of the school survey revealed relatively low graduation rates, especially inthe new ATSs. Appendix 8 shows that the average graduation rates in the period 1997-1999were 78 percent for the new ATSs and 83 percent for the upgraded ATSs. The relatively lowgraduation rates imply a high ratio of repeaters and dropouts. Due to incomplete data, the OEMwas unable to separate repeating rates from dropout rates for all project schools. Nevertheless,student data by grade was collected in two project schools visited by the OEM; the results


17/37

showed that the dropout rate was high and increasing, from 13 percent in 1997/98 to 24 percentin 2000/01 (Appendix 9).

42. Group discussions with school principals, teachers, and students, as well as tracerstudies on 71 dropouts, identified the following causes of the high dropout rates. First, the low

number of applicants for the project schools resulted in insufficient screening of applicantsduring student recruitment. It was reported that ATSs were the last choice of many applicants.In some cases, students were not serious when applying to an ATS due to insufficient interest inagricultural employment. They used ATS as a temporary step before making a final decision,such as finding a job. A more important cause quoted by many principals and teachers,however, was the financial crisis that started in 1997, resulting in deterioration of financial statusof many families that would otherwise have been able to afford upper secondary education. Thedepressed economy after the financial crisis and the gloomy prospect of agricultural earningsmay also have discouraged some applicants. It is possible that ATSs still had a certain value tothe dropouts, who might have learned something during their time there.

5. External Efficiency

43. External efficiency of the project schools was relatively low. The PCR estimated that abouthalf of the graduates of the project schools were employed in agriculture-related jobs. The OEMfound a similar pattern in its discussions with teachers in charge of job placement in the projectschools. It was estimated that about 5 percent of the graduates were employed in the publicsector, 15-20 percent went to colleges, and 15-25 percent were hired by private companies.11Some graduates interviewed by the OEM said that about 20 percent of their classmates werestill waiting for work after one year.

44. The OEM identified the following causes of the relatively low external efficiency. First,

searching for agricultural employment in the public sector was difficult because the Governmentsubstantially reduced hiring of agricultural extension workers, which had been the primarymotivation of students when they applied for ATS. Furthermore, large companies preferredgraduates with a college diploma; small companies, while open to ATS graduates, offeredsalaries not much higher than those of unskilled workers.12 While some graduates went back totheir parents farms, many were reluctant to apply for plantation farms located in remote areas,or jobs outside their provinces. Second, self-employment in agriculture was difficult. Forgraduates in Java, a lack of access to farmland was the primary constraint. For graduates onouter islands, the major difficulty was marketing of farm products. Furthermore, most graduateslacked start-up capital and access to credit services. The lack of business experience and theinability to handle price risks also discouraged many graduates from starting their own business.

45. Insufficient efforts of the project schools in marketing their students were another majorcause of the problem. While some project schools had teachers or student counselors in chargeof job placement, they just provided employment information but no further services, such asinviting companies to interview students on campus. Internship was potentially an effective

11There were no data on the remainder. The OEM found that the precise distribution of ATS graduates is difficult toestimate due to a lack of records on graduates in the project schools. Usually, graduates employed in the publicsector or private companies are more willing to inform their schools of their employment than the self-employedand unemployed. Thus, it is not possible to distinguish the self-employed from the unemployed or unreported.

12Feedback from graduates, private companies, as well as teachers in charge of job placement suggests thatmonthly salaries offered to ATS graduates by small agricultural companies are generally Rp200,000 ($20) or lessfor their probation period and Rp200,000 to Rp300,000 ($30) thereafter.


18/37

mechanism of marketing students and building up close linkages with potential employers.However, few project schools actively contacted private companies to promote internship andemployment for their students. By contrast, private ATSs seemed to be more proactive; someused their alumni in private companies to promote employment for their students. Finally,MOEC and DTVE believe that the quality and reputation of an ATS is crucial for the employment

of its graduates as well as student enrollment. More efforts are needed for the project schoolsto build up their reputation.

B. Financing of School Operations

46. Ongoing decentralization in Indonesia has shifted the responsibility for financing schoolsto district governments. Decentralization has the merit of fostering strong ownership by localgovernments and enabling them to allocate resources based on local needs. The OEMobserved increased budget allocations to some project schools and reduced allocations forothers after decentralization. One district government head interviewed by the OEM said that

education was the third priority of his government, after transport and health. Within education,the urgent need for school repairs in elementary and junior high schools had a much higherpriority than ATSs. Furthermore, some officials at the central and provincial levels expressedtheir concern over the willingness and ability of local governments to finance ATSs in the future.

47. It appears that future ATS financing will vary by school depending on the priorities ofdistrict governments and their views about the contributions that an ATS can make to the localeconomy. In most cases, it is likely that the district governments will continue to provide abudget to cover teacher salaries. However, this will certainly not be large enough to cover allschool expenses. ATSs will have to increasingly rely on their own income generation tosupplement insufficient funding.

48. The OEM found that the project schools have strong potential to generatesupplementary revenues to finance their O&M. Currently, private ATSs do not receive regularfinancial support from the Government and are still able to finance their operations.13 Ascompared to private ATSs, the project schools have much better endowments, such as a largeramount of farmland, new school buildings, better facilities, and good farm machinery andequipment. The quality of teachers in the project schools is also higher than that in private

ATSs.14 However, these good resources have not been fully utilized, largely due to insufficientincentives and motivation in the project schools.

49. Suggestions on how to generate supplementary revenues for school operations werecollected during the OEMs discussions with school principals and teachers. These included(i) enlarging PU operations by contracting out PUs to commercial firms when appropriate;

(ii) renting out farm machinery and equipment, and leasing farmland that was not used;(iii) engaging in agricultural and nonagricultural businesses based on local market demand;(iv) increasing student enrollment by offering attractive courses such as English and computer

13The OEM observed that some private ATSs had a few teachers from public schools who taught in the privateschools at government cost. Occasionally, some private ATSs also received a small grant from the Government. Ingeneral, however, private ATSs were financed by student fees, which were higher than in public ATSs. Generatingrevenues through land donated by their foundations was another major income source. Some private schools alsoreceived financial support from their foundations.

14The OEM found that many private schools had few permanent staff and relied largely on part-time teachers toreduce operating costs.


19/37

technology; and (v) providing short-term training to local companies on a full-cost-recovery basisto better utilize teachers and school facilities.

50. Difficulties in implementing the above suggestions were discussed. First, teachers maynot have the required skills and experience to teach the new courses; their retraining will require

government financial support. Second, ATSs may not have the required facilities for the newcourses. Third, the current management systems in public ATSs, especially the low and fixedsalaries, do not provide strong incentives or pressure for principals and teachers to makechanges. The OEM considers the last factor as the most critical. If appropriate incentiveschemes are established in the project schools, the teachers and principals will be motivated tofind the best approaches to utilize the school resources and seek support from localcommunities and industries.

C. Sustainability

51. All project schools have been operating to date, although some have been workingbelow capacity. It is likely that the project schools will continue to operate in the future as thedistrict governments may continue to meet teacher salaries, which is the single largest item inschool expenditure. It is recognized that the sustainability of the project schools relies oncontinued government budgetary support.

52. While the majority of the school facilities and farm machinery is functional, the OEM foundthat some broken equipment and machinery has not been repaired and remains out of service.Minor repairs to school buildings in most schools, although recommended by the PCR missionin 1999, have not been implemented. Discussions with teachers and principals showed that thecosts of normal school maintenance and repair of school facilities were not particularly high ifthey were conducted in a timely manner. Due to lack of regular funding, however, the schools

had to report their needs and request for funding. The request took time and was often delayed,leading to further deterioration of the broken facilities. In spite of the detailed O&Marrangements in project design (para. 13), none of the project schools had a budget itemallocated for school maintenance, which has been financed by ad hoc arrangements. Theunavailability of spare parts of farm machinery in local markets was given as another reason forthe problem in a few cases.


20/37

IV. ACHIEVEMENT OF OTHER DEVELOPMENT IMPACTS

A. Socioeconomic Impact

53. By upgrading 8 ATSs and building 16 new ones, the Project contributed to the expansionof agricultural education in Indonesia. As a result, the high demand for public ATSs observed atappraisal has been met. In particular, the Project provided sufficient access to agriculturaltechnology education on outer islands. In addition to the 3,755 teachers and administrative staffthat received training under the Project, the direct project beneficiaries were students andgraduates, estimated at about 10,000 and 2,700, respectively, every year. The OEM found thatthe majority of the ATS students were from middle- or low-income families, including somestudents from poor families. These students were not enrolled in general high schools forvarious reasons. For those with low academic performance, an ATS was their last chance of

obtaining credentials of upper secondary education.

54. The Project benefited women by increasing their access to agricultural technologyeducation; the project schools enrolled about 37 percent female students, higher than theaverage of 20 percent female students in ATSs in Java and 30 percent outside Java estimatedat appraisal. Furthermore, the curriculum modified under the Project provided better training inpostharvesting, food processing, marketing, and management of small enterprises, which werepopular subjects among female students. However, the OEM found that female graduates of

ATSs faced more difficulties in finding employment than male graduates, as many agriculturalfirms preferred male employees.

55. The Project was intended to assist private ATSs by providing training for their teachers.

Such training was not implemented. By contrast, the OEM observed an unintended adverseimpact of the Project on private ATSs. In one case, a private ATS was closed due to a sharpdecline in applicants after the establishment of a project school in the same area.

B. Environmental Impact

56. As envisaged at appraisal, the Project did not have an adverse impact on theenvironment. The development of school sites and farm sites was well planned, and theproduction activities were on a small scale without a significant impact on the ecology of theproject areas. Indirectly, the Project is likely to have a positive impact on the environment, as it

provided environmental education to the ATS students, such as soil and water conservation andmanagement, as well as other sound farming techniques.

C. Impact on Institutions and Policy

57. The Project established a computerized information system in the project schools. As aresult, the schools visited by the OEM were able to produce requested data within a short time,although the data were incomplete and inconsistent in some cases. At the national level, theinformation system established at DTVE contains detailed data by school. However, the system


21/37

overly focuses on physical indicators such as school buildings, equipment, furniture, machinery,courses, and teachers without sufficient information on financing of school operations such asoperating expenses, annual budget allocations, student fees, and revenues from PU operations.There was also a lack of information on employment of graduates. Although the conduct oftracer studies by project schools was envisaged at appraisal, no such study was conducted,

largely due to lack of experience, interest, motivation, and budget allocation. DTVE publishedschool data of public ATSs in 1995-1997. Due to a shortage of funds, however, publication hasbeen discontinued since 1998.

58. The Project provided a substantial amount of training to improve the capacity of the schoolteachers and principals. Since its completion, however, there has been no training budget for theseschools. This one-off training is less effective as teachers need periodic upgrading of knowledgeand skills based on new technologies as well as changes in local industries.


22/37

V. OVERALL ASSESSMENT

A. Relevance

59. The Project was relevant at appraisal as it supported the Government's objectives ofpromoting agricultural growth and balanced regional development, and was consistent with

ADB's operational strategy at that time. At postevaluation, the priority of ADBs lending in theeducation sector had shifted to basic education. While recognizing continued relevance ofagricultural technology education in Indonesia due to the critical importance of agriculture in thecountry, the issue is what should be the best approach to provide such education. ADBs Policyon Education (draft, September 2001) argues that the case for free government provision ofupper secondary education is not strong because children from poor families tend not to enrollin upper secondary schools. Moreover, government provision may compete directly with the

private sector when private schools are more responsive to labor market demand and canproduce the required skills at a lower unit cost with no loss of quality. Although the Projectcontributed to balanced regional development by expanding ATS capacity on outer islands,many of the new schools there suffered from insufficient demand and declining numbers ofapplicants. Based on these considerations, the Project is rated partly relevant.

B. Efficacy

60. The appraisal targets were largely met. In particular, all project schools are fullyfunctional. However, the training for private school teachers was not implemented; the PUs

have not been fully functional in some schools; and student enrollment has been below target inmany new schools. On balance, the Project is rated efficacious.

C. Efficiency

61. Most of the project facilities are used, although underutilization of school capacity hasbeen observed. In particular, many teachers have had light teaching loads due mainly to lowstudent enrollment. The use of the standard school size for all project schools also contributedto the low utilization of teachers and school facilities in many schools on outer islands with lesspopulation and lower demand for ATSs. Dropout rates have been relatively high, partly affectedby the financial crisis in 1997. External efficiency has been relatively low, with only roughly halfof the graduates engaged in agriculture-related jobs. Overall, the Project is rated less efficient.

D. Sustainability

62. All project schools have been operating; their continued operation in the future is likelyas local governments will probably continue their budgetary support for the schools. The currentstatus of school O&M is satisfactory, as the schools are still relatively new and do not requiremajor repair. Due to a lack of a regular funding source for school maintenance, however, some


23/37

broken equipment and farm machinery has not been fixed and remains out of service. Minorrepairs of school buildings have not been implemented; this could lead to further deterioration.On balance, the Projects sustainability is rated likely.

E. Institutional Development and Other Impacts

63. The Project had established a computerized information system in the project schools,although the system emphasizes physical indicators without sufficient information on schoolfinancing and graduate employment. The Project provided a substantial amount of training forteachers and principals, but the lack of a training budget since project completion has restrictedthe full impact of the training. The Project enhanced the capacity of DTVE staff in projectplanning and implementation; most of them have remained in relevant government service. TheProject addressed the high demand for public ATSs at the time of appraisal, and enhancedaccess to agricultural technology education for a large number of students, especially those onouter islands. Overall, the Projects institutional impact is significant.

F. Overall Project Rating

64. The Project aimed to improve the quality and efficiency of agricultural technologyeducation, provide equitable access to such education especially on outer islands, and developentrepreneurial skills of teachers and students. As planned, the Project increased the capacityof agricultural technology education at the upper secondary level by about 15 percent. Bylocating two thirds of the new capacity on outer islands, the Project contributed to a morebalanced allocation of public ATSs in the country. The Project also improved the quality ofeducational inputs by providing new school facilities and better trained teachers, principals, andother administrative staff. However, the objectives of improving educational efficiency and

developing entrepreneurial skills have not been fully achieved. Based on this assessment, theoverall project rating is successful.15

G. Assessment of ADB and Borrower Performance

65. The Government and ADB prepared the Project in accordance with their developmentstrategies at the time of appraisal. The process of project preparation was quick and efficient.However, the Government did not give sufficient attention to implementation in its early stages,resulting in substantial delays. ADBs project supervision was weak initially but improvedsignificantly later. As a result, implementation accelerated and the Project was completed

almost on schedule. The Government complied with most loan covenants. Overall, theperformance of both the Government and ADB is rated satisfactory.

15Based on the current four-category rating system (highly successful, successful, partly successful, orunsuccessful).


24/37

VI. ISSUES, LESSONS, AND FOLLOW-UP ACTIONS

A. Key Issues for the Future

1. Estimation of Demand for Project Schools

66. One major cause of the underutilization of school capacity has been a low level ofstudent enrollment due to insufficient number of applicants, especially in the new schools. TheOEM identified the following causes. First, the analysis of demand for and supply of the projectschools was based on a static observation at appraisal without further analysis on the likelytrend after the substantial increase of student places under the Project. Second, the demand

estimation relied heavily on the government forecast for sector development and humanresource needs. More attention should have been given to other factors that influenced privatedecisions on school choices, such as the perception of students and parents on the prospects ofagricultural employment and earnings of ATS graduates as well as alternative options availablefor lower secondary graduates. Given the sharply reduced government hiring of agriculturalextension workers, the few openings in private agricultural companies, and the difficulties inself-employment in agriculture, ATS have not been particularly attractive to many lowersecondary graduates. Third, the demand was estimated at the national level, and the Projectadopted a standard model of 500 students per school for all schools without giving full attentionto the local demand for individual ATSs. As a result, the ATS located in highly populated areassuch as Java have had student applicants that matched the standard size, but many new ATSson outer islands have had student enrollment far below target.

67. In retrospect, project design could have adopted a flexible approach to permit variationsin school size based on local demand. Local governments and communities as well as schoolrepresentatives could have been invited to participate in the design of a subproject and jointlydetermine the school size and investment package. If such flexibility was not feasible, analternative approach could have been adopted to develop at least two models: one for schoolsin Java and the other for those on outer islands. The school size on outer islands could havebeen smaller than that on Java.

2. Design and Implementation of Production Units

68. The PU concept was an innovation in project design that aimed to provide ampleopportunities for students to develop entrepreneurial skills and generate revenues for school O&M.While PUs were established in all project schools as planned, not all of them have functioned well;not all students have had access to practical training; and few PUs have generated sizablerevenues. Furthermore, many PUs have suffered operating losses, lost the start-up capitalprovided by the Project, and are facing financial difficulties. One major cause lies in project design,which focused on the provision of physical inputs and seed capital without the establishment ofproper management systems and the provision of managers with business experience. If theproject design had provided a budget for the hiring of a commercial manager for each PU andrequired him or her to establish proper systems for operational planning, marketing, and financial


25/37

management, or if the issues of PU losses had been identified promptly during implementation withtimely modification of project design, the large amount of farmland, machinery, and productionfacilities might have been fully utilized, and the PU might have achieved the envisaged objectives.

3. Financing of School Maintenance

69. The project design contained detailed O&M arrangements, including an analysis ofgovernment budgetary implications, obtaining of government assurance on budgetary provision,and arrangements on the use of student fees and PU revenues for school maintenance (para. 13).In spite of the careful design, inadequate maintenance of school facilities has still occurred, largelyfor the following reasons. First, the analysis of government budgetary implications from theviewpoint of a single project was insufficient, as the Government faced many other competing usesof its budgetary resources. Second, although the Government has continued to meet the schoolbudget, the amount has been insufficient to cover all school expenditures. In most cases, thebudget has covered teacher salaries but not school maintenance. Third, tuition fees have

continued to be low in public ATSs. Many schools have charged additional development fees,which have been used for special investment such as water and power supply instead of schoolmaintenance. Fourth, revenues from PU operations have remained negligible in most schools;some PUs have even operated at a loss. Although the cost of school maintenance has not beenhigh if conducted regularly and promptly, the lack of a regular funding source has resulted in adhoc arrangements. As a result, the minor cracks in school buildings and some broken facilitieshave not been fixed; the latter remain out of service. These problems could have been minimized ifthe project design had established a reliable funding source for school maintenance as discussedin para. 72 (vi).

4. Future Financing of Project Schools

70. The responsibility for financing public ATSs has been delegated to district governmentssince 2001. It is likely that future financial support for the project schools will vary by schooldepending on the priority of district governments and their views on the contribution of ATSs tolocal economies. In most cases, government budgetary support will not be enough to cover allschool expenditures; the project schools will have to generate supplementary revenues for O&M.While good potential for revenue generation exists in the project schools (paras. 48-49), the currentmanagement systems and incentive schemes in the project schools, especially the low and fixedsalaries for teachers, do not provide strong incentives or pressure to motivate the schools to fullyutilize their resources.

71. Since the central Government will continue to play an important role in agriculturaltechnology education in terms of policy guidance and supervision, DTVE may focus its efforts onthe creation of a policy environment that encourages competition and generates pressure on theproject schools to perform well. DTVE should consider providing annual awards to the best ATSswith good management systems and successful PU operations, and disseminating theirexperiences to other ATSs. DTVE may also study the financing and management systems ofprivate schools including in particular their incentive schemes, which could be introduced to theproject schools as appropriate. To maximize the utilization of school resources, DTVE shouldconsider conducting an inventory survey of all school facilities as well as teaching load of allteachers in the project schools. Based on the survey results and in close consultation with localgovernments, DTVE could then develop appropriate mechanisms and incentive schemes to


26/37

encourage rational reallocation of school facilities and teachers among the schools to optimize theutilization of public resources.

B. Lessons Identified

72. The Project provided the following valuable lessons for future improvements:

(i) The sole use of government forecasts for human resources and static data toestimate demand for schools is insufficient. Efforts are needed also to considerfactors that influence private decisions on school choices as well as the future trendof demand for schools of the given type after the project investment.

(ii) Using only a nationwide forecast for demand for schools is also insufficient. Effortsare needed to estimate local demand for individual schools, which take into accountregional variations especially in terms of population density and competition from

other schools. Instead of using a standard model for all schools, flexibility should beexercised in the design of subprojects based on local conditions. Local communitiesand school representatives should be invited to participate in the subproject design.

(iii) Provision of hardware and software inputs to schools can improve the quality ofeducational inputs, such as better school facilities and better trained teachers.Without effective management systems and incentive schemes in schools,however, the improved inputs may not lead to improved quality of educationaloutputs and efficiency.

(iv) The cost of school maintenance is not particularly high if conducted regularly andpromptly. The lack of a regular funding source for school maintenance, however,

may lead to ad hoc arrangements, delays in the required repairs, and furtherdeterioration of facilities.

(v) The analysis of government budgetary implications from the viewpoint of a singleproject is insufficient as the concerned government may have many competinguses for its budgetary resources.

(vi) Government assurance on providing sufficient resources for school maintenance isnecessary but insufficient. More effective measures are needed to ensure theestablishment of a reliable funding source for school maintenance after projectcompletion, including, for example, setting aside a fixed amount from thegovernment budgetary allocation or student fees to establish a maintenance fund,

and developing regulations to ensure exclusive use of the fund for schoolmaintenance.

(vii) The provision of a large amount of one-off training under a project without a budgetfor continued upgrading of teachers skills after project completion may not be costeffective. The lack of proper incentive schemes in schools may also restrict the fullimpact of the training as teachers may not be motivated to fully utilize their skills.

(viii) Innovative project design such as the PU concept needs to be accompanied byclose supervision of project implementation. The supervision should not focus on anexact conduct of a blueprint project design. A process approach needs to be


27/37

adopted to permit flexible adjustment of project design during implementation, andthe supervision should focus on timely identification of issues emerging duringimplementation.

(ix) The provision of a standard investment package for all project schools may simplify

project implementation and obtain better terms in procurement. The short-term gain,however, is at a long-term cost of inefficiency and weak sustainability, as theinvestment package may not match well the needs of individual schools leading toidle facilities and the additional burden of maintaining these facilities.

(x) The establishment of a public school needs to avoid any potential adverse impacton private schools in the project area that serve the same set of clients.

(xi) The design of an education project should give particular attention to the long-termoperation of the project schools after completion when there are no more fundsfrom the project, and the financial support from the government may not be fullyavailable and may be subject to delays in its release. Intensive consultation needs

to be conducted during the design stage with concerned government authorities,school principals and teachers, and local communities to develop effectivemeasures to ensure a reliable funding source, including, when appropriate, the useof loan proceeds to establish income-generating activities for the schools.

C. Follow-Up Actions

73. The lack of a reliable funding source for school maintenance has led to continueddeterioration of some out-of-repair facilities. DTVE, in close consultation with local governments,should issue regulations to (i) establish a school maintenance fund in each project school by

apportioning a fixed amount from the annual budgetary allocation or from student fees for schoolmaintenance, and (ii) ensure exclusive use of the fund for school maintenance. A target date ofDecember 2002 is proposed for issuing the regulations. The Social Sectors Division in theSoutheast Asia Department of ADB should monitor the implementation of this action.

74. DTVE, in close consultation with local governments, should conduct pilot testing inselected schools of contracting out their PUs to interested commercial firms as discussed inpara. 37, with a view to fully utilizing the school facilities and sustaining the PU operations. Atarget date of August 2002 is proposed for initiating the pilot testing. The Social Sectors Divisionin the Southeast Asia Department of ADB should monitor the implementation of this action.


28/37

APPENDIXES

Number Title Page Cited on(page, para.)

1 Estimated and Actual Project Costs 21 2, 5

2 Achievement of Project Targets 22 4, 14

3 Total Number of Students 23 7, 30

4 Number of Student Applicants 24 7, 31

5 Number of New Students 25 7, 32

6 Demand for Agricultural Technology Schools 26 8, 33

7 Ratio of Students to Teachers 27 9, 39

8 Graduation Rates 28 9, 41

9 Cohort Enrollment and Dropout Rates in TwoSchools

29 10, 41


29/37

Expenditure Item Foreign Local Total Foreign Local Total

Land Acquisition 0.00 9.92 9.92 0.00 3.27 3.27

Civil Works 9.97 24.28 34.25 11.48 19.49 30.97

Equipment, Furniture, and Materials 13.56 4.56 18.12 17.24 1.37 18.60

Consulting Services 2.64 4.73 7.37 2.08 2.37 4.46

Staff Development

- International Training 4.07 0.27 4.34 4.24 0.00 4.24

- Local Training 2.79 9.21 11.99 1.05 2.94 3.99

Benefit Monitoring and Evaluation 0.00 0.53 0.53 0.00 0.42 0.42

Project Implementation 0.00 9.87 9.87 0.00 0.84 0.84

Production Unit 0.00 1.55 1.55 0.00 0.42 0.42

Taxes and Duties 0.00 5.64 5.64 0.00 3.70 3.70

Interest and Commitment Charges 15.94 0.00 15.94 15.90 0.00 15.90

Total 48.97 70.55 119.52 51.98 34.82 86.81

AR = appraisal report, PCR = project completion report.

ESTIMATED AND ACTUAL PROJECT COSTS

Appraisal Estimates Actual

($ million)

Sources: Loan 1050-INO:Agricultural Technology Schools Project, Appraisal Report, 1990; and PCR: 1050-INO:Agricultural Techn

December 1999.

(per AR) (per PCR)


30/37

Appendix 2

Appraisal Actual Degree of

Target Achievement Achievement

Component (per AR) (per PCR and OEM) (%)

A. Educational Programs Development

1. Person-Months

Overseas Training 1,872 1,874 100

Domestic Training 20,287 20,338 100

2. No. of Trainees 3,554 3,755 106

Overseas Training 204 342 168

Teachers NA 214

Principals NA 60

Participants of Study Visits NA 68

Domestic Training 3,350 3,413 102

Teachers NA 2,999

Technicians and Administrative Staff NA 414

Private Schools teachers 500 0 0

B. Educational Facilities Development

1. No. of ATSs Upgraded 8 8 100

of which: Outer Islands 2 2 100

Java 6 6 100

2. No. of New ATSs 16 16 100

of which: Outer Islands 14 14 100

Java 2 2 100

3. No. of Students and Graduates

At project completion

Student population 12,000 11,386

a

95Graduates per year 3,500 3,458a

99

At postevaluation

Student population per school 500 408 82

Graduates per year per school 146 130 89

C. Production Units (PUs) and Entrepreneurial Support

No. of Schools with established PU 24 24 100

= not calculated, NA = not available.

AR = appraisal report, ATS = agricultural technology school, OEM = Operations Evaluation Mission, PCR = project completion report.a

Based on the PCR, as of 1998.b

Based on the results of the survey of 21 project schools, as of 1999.

Sources: Loan 1050-INO: Agricultural Technology Schools Project, Appraisal Report, 1990; PCR: 1050-INO: Agricultural Technology

Schools Project, December 1999; and OEM school survey.

ACHIEVEMENT OF PROJECT TARGETS


31/37

Appendix 3

1995 Growth Index 199

Outer No. of No.

No. School Name Java Islands Students 1995 1996 1997 1998 1999 Stud

Upgraded ATSs

1 ATS Cibadak a 424 100 104 113 113 117 4

2 ATS Cianjur a 599 100 93 97 89 97 5

3 ATS Jalaksana a 478 100 132 137 150 133 6

4 ATS Salam a 401 b

100 108 121 120 4

5 ATS Delanggu a 403 100 112 119 121 118 4

6 ATS Buntul a 365 100 109 109 114 113 4

7 ATS Way Abung a 651 100 95 78 78 80 5

8 ATS Sakra a 574 100 95 72 100 107 6

Average of Upgraded ATSs 556 5

New ATSs

9 ATS Pandeglang a 0 100 151 184 227 3

10 ATS Jepara a 224 100 134 140 128 157 3

11 ATS Gelumbang a 332 100 158 174 192 177 5

12 ATS Bungo Tebo a 426 100 130 112 85 71 3

13 ATS Manna a 216 100 197 277 235 180 3

14 ATS Mempawah a 567 b

100 80 70 63 3

15 ATS Takisung a 222 100 159 208 144 88 1

16 ATS Tnh Grogot a 399 b

100 106 109 86 3

17 ATS Toli Toli a 346 100 89 88 72 60 2

18 ATS Muna a 362 100 118 104 75 60 2

19 ATS Sentari a 320 100 171 215 198 189 6

20 ATS Sumba Brt a 176 100 165 164 165 164 2

21 ATS Jembranaa

209 100 147 111 106 62 1Average of New ATSs 380 3

Average of 21 Project Schools 453 4

= not calculated, ATS = agricultural technology school.a

The standard capacity is 500 students per project school.b

Data for 1996.

Source: School survey in 2001.

(%)

TOTAL NUMBER OF STUDENTSa

Location


32/37

Appendix 4


Outer No. of (%) No. of

No. School Name Java Islands Applicants 1995 1996 1997 1998 1999 Applicants

Upgraded ATSs

1 ATS Cibadak a 180 100 114 102 139 158 285

2 ATS Cianjur a 216 100 105 131 104 122 263

3 ATS Jalaksana a 170 100 190 144 171 135 229

4 ATS Salam a 160 100 135 127 136 193 309

5 ATS Delanggu a 193 100 117 109 118 109 211

6 ATS Buntul a 140 100 132 124 171 148 207

7 ATS Way Abung a 308 100 63 63 67 78 240

8 ATS Sakra a 211 100 109 116 120 146 309

New ATSs

9 ATS Pandeglanga a 208 a 100 42 75 91 190

10 ATS Jepara a 166 100 83 86 104 73 122

11 ATS Gelumbang a 232 100 147 124 122 104 241

12 ATS Bungo Tebo a 277 100 68 66 47 49 137

13 ATS Manna a 786 100 46 31 17 15 120

14 ATS Mempawah a 304 100 73 52 50 56 169

15 ATS Takisung a 516 100 44 28 16 11 57

16 ATS Tnh Grogot a 189 100 112 88 95 73 138

17 ATS Toli Toli a 185 100 62 65 81 49 90

18 ATS Muna a 172 100 93 59 56 52 90

19 ATS Sentari a 368 100 93 80 84 69 254

20 ATS Sumba Brt a 125 100 174 126 90 93 116

21 ATS Jembrana a 92 100 126 62 78 51 47


Data for 1996.


NUMBER OF STUDENT APPLICANTS

Location


33/37

Appendix 5


Outer No. of (%) No. of

No. School Name Java Islands Enrollments 1995 1996 1997 1998 1999 Enrollments

Upgraded ATSs

1 ATS Cibadak a 174 100 103 105 109 132 229

2 ATS Cianjur a 214 100 101 101 101 101 216

3 ATS Jalaksana a 170 100 190 144 129 130 221

4 ATS Salam a 122 100 139 141 147 150 183

5 ATS Delanggu a 156 100 122 115 115 115 180

6 ATS Buntul a 128 100 113 111 111 113 144

7 ATS Way Abung a 216 100 84 89 90 84 181

8 ATS Sakra a 207 100 104 114 118 121 250

Average of Upgraded ATSs 173 201

New ATSs

9 ATS Pandeglanga

a 108a

100 68 144 170 184

10 ATS Jepara a 128 100 80 93 112 75 96

11 ATS Gelumbang a 188 100 114 114 132 117 220

12 ATS Bungo Tebo a 223 100 81 75 52 48 108

13 ATS Manna a 216 100 100 90 52 55 118

14 ATS Mempawah a 232 100 89 68 63 63 146

15 ATS Takisung a 222 100 65 56 32 17 37

16 ATS Tnh Grogot a 143 100 99 100 126 91 130

17 ATS Toli Toli a 173 100 62 56 53 52 90

18 ATS Muna a 172 100 93 59 56 52 90

19 ATS Sentari a 324 100 82 71 95 78 254

20 ATS Sumba Brt a 96 100 130 115 118 101 97

21 ATS Jembrana a 92 100 126 62 78 32 29

Average of New ATSs 193 123

Average of 21 Project Schools 180 153

Private ATSs

22 ATS Unggaran a 45 100 89 89 67 78 35

23 ATS Darusalam a 25 100 224 216 168 92 23

24 ATS Kornita a 26 100 138 85 131 104 27

25 ATS Sukaraja a 35 100 134 134 194 203 71

26 ATS Metro a 27 100 93 89 100 56 15

Average of Private ATSs 32 34


Data for 1996.


NUMBER OF NEW STUDENTS

Location


34/37

Appendix 6

Outer

No. School Name Java Islands 1995 1996 1997 1998 1999

Upgraded ATSs

1 ATS Cibadak a 1.0 1.1 1.0 1.3 1.2

2 ATS Cianjur a 1.0 1.0 1.3 1.0 1.2

3 ATS Jalaksana a 1.0 1.0 1.0 1.3 1.0

4 ATS Salam a 1.3 1.3 1.2 1.2 1.7

5 ATS Delanggu a 1.2 1.2 1.2 1.3 1.2

6 ATS Buntul a 1.1 1.3 1.2 1.7 1.4

7 ATS Way Abung a 1.4 1.1 1.0 1.1 1.3

8 ATS Sakra a 1.0 1.1 1.0 1.0 1.2

Average of Upgraded ATSs 1.1 1.1 1.1 1.2 1.3

New ATSs

9 ATS Pandeglang a 1.9 1.2 1.0 1.0

10 ATS Jepara a 1.3 1.3 1.2 1.2 1.3

11 ATS Gelumbang a 1.2 1.6 1.3 1.1 1.1

12 ATS Bungo Tebo a 1.2 1.1 1.1 1.1 1.3

13 ATS Manna a 3.6 1.7 1.3 1.2 1.0

14 ATS Mempawah a 1.3 1.1 1.0 1.0 1.2

15 ATS Takisung a 2.3 1.6 1.2 1.1 1.5

16 ATS Tnh Grogot a 1.3 1.5 1.2 1.0 1.1

17 ATS Toli Toli a 1.1 1.1 1.2 1.6 1.0

18 ATS Muna a 1.0