patent final evaluation report summary - text · final project evaluation report patent page 4 of...

12
Final Project Evaluation Report Poverty Alleviation in Tajikistan through Education and Non-formal Training (PATENT) Cologne, January 2011 Compiled by: Thekla Kelbert & Sinziger Str. 11 D-50968 Köln Sherali Toshmurodov 13 Valadarskiy street, Dushanbe, Tajikistan European Commission budget line: BGUE-B2008-19.100200-C8-AIDCO Reference: EuropeAid/126-844/L/ACT/TJ

Upload: others

Post on 18-Oct-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: PATENT Final Evaluation Report Summary - TEXT · Final Project Evaluation Report PATENT Page 4 of 12 2. Summary The project activities were implemented as planned, with only very

Final Project Evaluation Report

Poverty Alleviation in Tajikistan

through Education and Non-formal Training (PATENT)

Cologne, January 2011

Compiled by:

Thekla Kelbert &

Sinziger Str. 11

D-50968 Köln

Sherali Toshmurodov

13 Valadarskiy street,

Dushanbe, Tajikistan

European Commission budget line: BGUE-B2008-19.100200-C8-AIDCO

Reference: EuropeAid/126-844/L/ACT/TJ

Page 2: PATENT Final Evaluation Report Summary - TEXT · Final Project Evaluation Report PATENT Page 4 of 12 2. Summary The project activities were implemented as planned, with only very

Final Project Evaluation Report PATENT Page 2 of 12

The project “Poverty Alleviation in Tajikistan through Education and Non-formal Training (PATENT)

was implemented by the Institute for International Cooperation of the German Adult Education

Association (dvv international) in partnership with the Association of Scientific and Technical

Intelligentsia (ASTI) and funded by the European Commission (80%) and the German Ministry for

Economic Co-operation and Development - BMZ (20%).

During the last month of project implementation (December 2010), this final, external evaluation of

the above mentioned project was conducted as originally stated in the planning documents. The

given evaluation of the ‘PATENT’ project was initiated by dvv international as the applicant of the

action and was prepared, discussed and agreed between the partners. The evaluation was conducted

in accordance with the TOR of the evaluation adopted by dvv international.

The results of the evaluation will be used to

• assess the relevance, effectiveness and weaknesses of the project planning and

implementation;

• measure the project impact on the target groups;

• improve the planning and implementation of similar projects in the future;

• assess the cooperation between the partners and stakeholders;

• generate ideas for follow up interventions and;

• fine tune the general country strategy of both partners in Tajikistan.

The results of the evaluation will be shared and discussed between the partners in order to improve

the performance in other/ new projects. The results of the evaluation will be provided to the

European Union ,the German Ministry for Economic Co-operation and Development (donors) and the

Ministry of Education of Tajikistan, as well as to other interested institutions working in the field of

adult education in Tajikistan.

Evaluation methodology

The project consists of a number of components including a labour market assessment, the

development of curricula for short-term vocational trainings at ten selected vocational education and

training (VET) schools, the provision of training equipment and machinery as well as written training

material to the schools, the further qualification of teachers and the school management, provision

of vocational short term courses for 2000 students, the provision of professional guidance and

business consultancy for the graduates, the promotion of social partnership mechanisms, and a

public relations campaign promoting a positive image of VET in the Tajik society.

The methods applied during the investigation thus included desk studies of relevant project

documentation in the evaluators’ home countries and upon arrival in Tajikistan as well as during the

course of the evaluation as relevant documents kept being supplied to the evaluation team (also

referred to as ‘the team’) throughout the investigation. Six out of ten VET schools who had received

inputs through the project were visited – three of which in Khatlon Province and three in Sughd

Province. At the schools, the material inputs in form of equipment and written training materials

were assessed and interviews with the different groups of beneficiaries – school management,

teachers and course participants – were conducted. In addition, focus group discussions were held

with the participants of those courses that were still on-going or had just finished, and selected

businesses that had started up as a result of the project intervention were visited. ASTI staff was

interviewed in Dushanbe, Kulyab and Khujand and accompanied the evaluation team during school

visits. Informal methods like informal conversations and observation were used in addition. Wherever

possible, the team tried to triangulate information received from different sources like dvv

international, ASTI staff and target groups and answers to common questions were compared and

analysed. Where possible, certain stakeholders and members of the target group were interviewed

separately, especially the students in order to have open democratic discussions without authorities

Page 3: PATENT Final Evaluation Report Summary - TEXT · Final Project Evaluation Report PATENT Page 4 of 12 2. Summary The project activities were implemented as planned, with only very

Final Project Evaluation Report PATENT Page 3 of 12

from the schools or teachers present. In some cases groups of women were interviewed separately

from men for instance a small group of graduates from an accounting course in Djabor Rasulovskij

District, the staff of one of the small businesses that had opened as a result of the business

competition and some all-female groups of the sewing courses.

Field investigations for the evaluation took place between 11 and 18 December 2010. The evaluation

team consisted of an international team leader with several years of work experience in international

development assistance in Tajikistan and a local co-evaluator with a long-standing background of

project administration and management in Tajikistan and abroad.

PATENT sewing class participants

PATENT welding class participants

Page 4: PATENT Final Evaluation Report Summary - TEXT · Final Project Evaluation Report PATENT Page 4 of 12 2. Summary The project activities were implemented as planned, with only very

Final Project Evaluation Report PATENT Page 4 of 12

2. Summary

The project activities were implemented as planned, with only very minor alterations which are

described below and which did not have any negative effect on the achievement of project results.

The status quo in the beneficiary VET schools has been successfully improved in terms of equipment

as well as teaching and management methods. The implementation of short-term vocational courses

was geared towards the poorer parts of the local population and applying the interactive teaching

methods and new equipment in order to raise the practical component of the course work. The

courses were evaluated very positively by the student groups interviewed. A visible motivation of VET

school staff and management to upkeep these positive changes was observed in all the schools.

The establishment of school advisory boards connecting social partners and VET schools and

especially the PR campaign preceding and accompanying the implementation process at school level

are seen as further strong sides of the project.

The economic impact at target group level has been less clearly visible and measurable - partly due to

weaknesses in the project's monitoring and evaluation framework. Especially the information on the

employment situation and income levels of course participants was patchy. One more round of data

collection 3 months after the graduation of the last participants however is still outstanding and will

increase clarity on the achievement of result No.1.

2.1 Summary of major findings

Main project results according to logframe indicators

Teaching material

The curricula and handbooks on management and modern methods of teaching will have a lasting

impact on the effectiveness of teaching and management and increase the quality of education

offered by schools as they will be applied to overall school management and in short-term courses

offered – in most cases on a commercial basis – in the future. The evaluation team got very positive

comments on the modern teaching methods from both teachers and course participants interviewed:

“It is even easier for us teachers when we employ new methods like group work and active utilization

of the theoretical contents during classes than what it used to be previously.”

Curriculum development

See “institutional impacts” and “sustainability” below.

Provision of equipment to schools

The equipment provided to the schools was being used by the course participants the evaluation

team has met and it was mentioned by all of the VET school directors and teachers as one of the

project components having the most positive and lasting impact on an improved learning (and

teaching) situation in the schools.

Impact of short-term courses on the participants

The underlying problem to the difficulties in measuring and evaluating project impact in terms of the

effect on the beneficiary level is actually related to planning – systematic follow-up and information

collection on course graduates was not planned as a project activity and thus no time and financial

resources had been allocated to it. If there is insufficient data collected to verify the improvement of

the employment situation of course participants after graduation of if some initial monitoring results

are weakly documented, the evaluation of the relevant project results and the specific objective

becomes difficult.

Page 5: PATENT Final Evaluation Report Summary - TEXT · Final Project Evaluation Report PATENT Page 4 of 12 2. Summary The project activities were implemented as planned, with only very

Final Project Evaluation Report PATENT Page 5 of 12

Trainings for management and teaching staff

One can describe the level of participation of the school managements in project activities and their

use of the gained professional skills on management in day-to-day management as active, but varying

across the 10 schools to some extent. Some schools can be singled out very positively regarding

active project engagement of their directors and deputies, who coordinated and managed the short

term courses well together with their team of teachers and who have changed some management

processes of the school (Istaravshan, Sarband and Khujand city).

Across the board the quality of the trainings on subjects like leadership, fundraising, and

management of VET schools were judged positively by directors and their deputies and seen as a

good input into capacity development for the staff towards reforming the management style used in

the VET system.

The capacities of the VET schools were also improved through conducting a big number of trainings

for the teachers on modern teaching methodologies and labour market analysis. The trainings aimed

at enhancing their knowledge of interactive teaching methods, team work and conflict management,

substantially changing the old teaching methodologies of vocational schools. The teachers have found

these methods easy to use and more productive compared to the old system of teaching. Interviews

with teachers and course participants have shown that the new methods quickly led to visible

change, and the students who were taught using new methods have gained more knowledge

especially in the realm of the practical application of their vocational skills.

Development of handbooks for VET school staff

A considerable amount of the copies of the two handbooks on management of vocational education

and on methods of interactive education were delivered to the schools with a big delay, starting only

from the third quarter of 2010. Some amount of handbooks was still to be printed and distributed at

the time of the evaluation.

Business consulting desks

During interviews it became clear that the business consulting desks have been established at all

schools visited. One person around from among the teaching staff was usually assigned the

responsibility for this activity, had received training as part of PATENT, and some were keeping

records on their clients and the kind of services provided

In focus group discussions with course participants – many of who had attended business workshops

organized by ASTI – it was found striking how confident some were about opening their own

businesses – men as much as women. They were also informed about the existence of the business

consulting desks and knew that they can turn to the desk for support and advise any time.

Professional guidance

Assuming that the workshops and round tables were conducted professionally and with meaningful

content (as according to plan), one can suppose that the understanding of the importance of

acquiring a profession and of employment and self-employment was raised among the VET school

staff and stakeholders.

Advisory Boards

According to result number 4 and the related project activities, a partnership strategy should be

developed for further support to the vocational schools towards employment or self-employment of

graduates and the establishment of a lasting cooperation between vocational schools, local

communities and governmental authorities. However, no written strategy, plan of activities,

framework or responsibilities concerning the operation and more importantly the future of the

advisory boards were presented to the evaluation team. There is no real mechanism jointly

developed for the advisory boards to continue their activities in support of the respective VET

schools.

Page 6: PATENT Final Evaluation Report Summary - TEXT · Final Project Evaluation Report PATENT Page 4 of 12 2. Summary The project activities were implemented as planned, with only very

Final Project Evaluation Report PATENT Page 6 of 12

Study Tour to Germany

The study tour to Germany organised by dvv international for VET school directors and social partners

from local authorities had a lasting impact on the vision on further development of adult education in

the country. According to some interviewed participants they received many ideas on new

management and teaching methods, especially on how to meet market and population needs

through the VET courses offered, and on an effective, systematic cooperation between VET schools

and social partners.

Business contest

164 learners got business planning training and 64 took part in the competition. Out of these, 8

winners were awarded € 500 each. As part of the project revision in June 2010 the number of grants

given to the winners of the business idea competition was increased from 2 to 8 which was a good

decision as the money benefits the target group directly – not only the winner, but also their future

employees – and it increased the incentive for people to take business planning seriously and

participate in the competition.

The owners of three businesses newly opened with the help of a PATENT grant and their employees

who the evaluation team met made a satisfied impression, they were receiving clients and were able

to fully equip their business premises with the help of the grant and in some cases their own capital.

Out of the 11 people working in the successful businesses 10 were women.

PR campaign

See the findings on “institutional impacts”, below.

Project effectiveness

The selection of the short-course format for PATENT proved to be a very effective choice, well-

adapted to the local context, especially as it also included a scholarship component to encourage

unemployed parts of the population to invest a smaller amount of their time in vocational education

than usually required for the classical – longer – VET courses. The quality of the courses was judged

as very good by the students and the stationeries, training materials and text books provided as well

as the scholarships at the amount of 10 € were found useful and effective.

A positive impact at target group level that can be singled out is the income generation opportunities

that a big number of women (100% of participants in sewing classes and more than 50% of

participants in accounting/ IT classes) have received. Especially the seamstresses now have the

opportunity to create incomes for their families working from home which is important especially to

those who have children and a household to take care of and are often the only breadwinners

present in Tajikistan when their husbands have left for labour migration.

It is seen as a very good idea that during the courses on business planning, ASTI was also presenting

some local banks who offer credits to the course participants – as well as ASTI’s own small micro-

finance project. This could have improved the effectiveness of the project’s component on business

development. Unfortunately for unknown reasons, up until the time of the evaluation none of the

participants had applied to ASTI for micro-credits.

Institutional impacts

Positive impacts have clearly also been achieved at the level of the schools – concerning the

equipment provided for the PATENT short-courses which will be kept in use in the future, as well as

the improved capacity base among school management and teachers. However, sustainability issues

which might be connected with the improved human capacity will be touched upon below.

Another positive impact at school level which has not been included in the planning is the possibility

of the schools to start offering the short-term courses promoted under PATENT on a commercial basis

in the future. Many schools had already started doing this before the end of the project

implementation period. Together with the Department of Vocational Education of the Ministry of

Education, ASTI has developed curricula for welding, tailoring, electrician and accounting courses

Page 7: PATENT Final Evaluation Report Summary - TEXT · Final Project Evaluation Report PATENT Page 4 of 12 2. Summary The project activities were implemented as planned, with only very

Final Project Evaluation Report PATENT Page 7 of 12

within the project. These curricula have been officially approved by MoE and will thus be used more

widely for other VET schools to conduct similar short term courses.

Looking at institutional impacts, furthermore the evaluation team got the impression that two years

after the transfer of the VET portfolio from MoLSP to MoE and given the on-going duplications in

some of their respective involvements, through the project their cooperation on the ground has been

strengthened and to some extent institutionalised as the employment centres of MoLSP became

strongly involved in the activities of the VET schools through the advisory boards and mutual

incentives to this cooperation were obvious.

The impacts of PATENT initiatives under results 3 and 5 – the promotion of the idea of social

partnerships and the PR campaign to raise the image of the VET sector – among potential social

partners and the wider public in the project sites have partly been hard to observe. However, a

selection of the very extensive and interesting material and documentation on the PR activities was

reviewed by the evaluation team in ASTI’s office in Khujand. The material can serve as a good basis

and best practice for future project interventions. It has to be stated positively that a high visibility of

the VET sector in general and public discussions of the social partnership concept have been achieved

through a very systematic and active PR campaign.

Sustainability

The official approval of the curricula developed as part of PATENT by MoE is a very positive achievement

in terms of the sustainable use of one of the central project outputs. This can thus have sustainable

impacts on the VET sector and potentially on a big number of future learners.

The sustainability of several project components and related results could be under threat due to a

combination of potential turnover of VET school staff and a lack of institutionalisation of knowledge

sharing, handover-procedures.

The advisory boards and business consulting desks in the schools – two important institutions created

as part of the PATENT intervention – are also at some risk of being unsustainable as their functioning,

mandates and tasks have not been institutionalised and documented.

From different meetings with ASTI staff we gained the impression that a lot of attention was paid to

day-to-day project management and the implementation of the planned activities whereas the aspect

of sustainability of some of the outcomes and impacts was not so present in the discussions and could

have been taken more seriously.

Monitoring and evaluation

The monitoring and evaluation framework of the project was not very elaborate and weakly

documented. It was also found to be not results- and impact-oriented enough. Quarterly monitoring

reports as mentioned in the grant proposal were not shown to the evaluation team. According to dvv

international Tashkent, project visits were taking place, but no findings and recommendations seem to

be documented in writing. Out of the three project evaluations planned, two were carried out after EU

approval had been sought for the change in plan. A first, internal project evaluation took place in the

20th month of project implementation which seems rather late given the plan to conduct one at 6 and

one at 12 months and that then the final evaluation was supposed to take place in month 22.

The main short-coming of project monitoring is the fact that merely the monitoring of inputs and

activities was done systematically. Any monitoring to be carried out with the purpose of measuring the

impact-related project indicators would have involved planning for that, including providing time and

financial resources as well as staff-time. This was not done well enough. A fact that leads to the finding

related to project effectiveness and impacts that the measurement of one of the most important

impact indicators – the employment situation of the 2000 VET school graduates from PATENT courses –

is hard to undertake.

One more finding is found as being crucial to the monitoring and evaluation framework of the project:

Except for a mini-baseline survey specifically related to result No.5, no baseline information has been

Page 8: PATENT Final Evaluation Report Summary - TEXT · Final Project Evaluation Report PATENT Page 4 of 12 2. Summary The project activities were implemented as planned, with only very

Final Project Evaluation Report PATENT Page 8 of 12

collected – at least not to the knowledge of the evaluation team. Especially concerning the monitoring

and evaluation of results-achievement for results No. 1 and 2 it would have been highly

recommendable to conduct pragmatic baseline surveys as part of project planning which would have

had to be cost-efficient in terms of time and money spent, but could have significantly improved

planning and the opportunity to later evaluate and demonstrate the impacts made under PATENT.

Logframe Indicators

The specific objective “to increase the employment and self-employment opportunities for poor

unemployed” if achieved has the potential to contribute to the overall project objective - poverty

alleviation in Khatlon and Sughd. Unfortunately the employment rate as the indicator set to measure

the achievement of the specific objective seems rather unsuitable as already pointed out by the internal

project evaluation in August 2010 (Annex V, p.19).

The choice of indicators to measure the achievement of results No. 2, 4 and 5 in the logframe appear

suitable only to check on the implementation of activities, but not on the achievement of the related

results or on impacts. To carry out more meaningful monitoring and evaluation activities throughout

project implementation and during the final evaluation some impact-oriented indicators could have

been chosen in addition.

Project Planning

The overall project aims are compatible with the priorities of the partner Government and with

international development goals. Education is one of the priority sectors in the Government’s PRSP

and NDS documents as well as an important area of intervention outlined in the Joint Country

Partnership Strategy for 2010-2012 compiled jointly by a group of 12 international development

partners including the European Commission and signed with the Tajik Government.

Some of the documentation on components of project planning which should serve to facilitate a

smooth management of the project was not developed: A work plan document as a central reference

point clearly linking results and activities with timelines, responsibilities and budget figures was

lacking. The evaluation team has not seen an organogram or detailed TOR for the different project

staff which could have served to clarify the hierarchical levels of the relatively complex staffing set-up

of PATENT. No elaborate M&E plan was developed beyond the scope of the overview given in the

grant proposal. It was also found that a simplified project procurement plan adapted to the local

setting and project profile could have helped analyse and mitigate the risks of the delays in

publication of materials and in the purchase of the equipment for the course.

Risks and assumptions estimated during project planning

No major obstacles as outlined in the risks and assumptions matrix to project implementation as such

have occurred. Some risks were not taken into consideration during project planning that have led to

minor delays in the implementation of some activities: The first round of scholarship payments, the

delivery of equipment to the VET schools and the production of some of the education material.

Some of these issues were beyond the control of the implementer and of dvv international and

resulted only in minor delays in the implementation of activities.

The risk of high turnover of VET school staff during and after the implementation phase was

identified and included in the assumptions in the logframe. During the project implementation period

four out of the ten directors of the beneficiary VET schools were exchanged which is a considerable

proportion, especially as much of the implementation at school level will depend on them. Also for

the period after the end of the project implementation phase the risk that acquired knowledge and

improvements made will be lost or partly lost because of later turnover and a lack of

institutionalisation of VET school management processes still exists.

The internal project monitoring and evaluation system was not developed to an extent of foreseeing

periodic risk assessments and taking the developments of risks and assumptions into account during

project implementation more systematically.

Poverty orientation and selection of target groups

Page 9: PATENT Final Evaluation Report Summary - TEXT · Final Project Evaluation Report PATENT Page 4 of 12 2. Summary The project activities were implemented as planned, with only very

Final Project Evaluation Report PATENT Page 9 of 12

The poverty orientation of the entire project and especially of result No.1 regarding the target group

of unemployed participants of the newly established short-term VET courses is prominent in the

planning documents and from interviews with project and VET school staff. Apart from some

unclarities regarding the selection of beneficiary VET schools, later on in the selection of the

individual participants a lot of attention has clearly been paid to poverty criteria – the family and

economic status of applicants. It is seen as a well-adapted and efficient approach to have involved

social partners and their knowledge of the local population in the selection of participants.

The selection of the beneficiary VET schools by ASTI and the Ministry of Education is not well

documented. The selection criteria remain unclear as an initial assessment of VET schools in the two

project regions done by ASTI was not presented to the evaluation team. Clearly, the major criterion

should have been poverty levels in the catchment area of the respective school and – based on this –

the perspectives for poverty reduction. Given the unclear accounts of the selection process, poverty

orientation in this regard was hard to assess for the evaluation team.

Overall project management

Activities at school level were carried out in good cooperation between stakeholders and on the

whole the activities were implemented satisfactorily and showing good results especially at the level

of the VET schools.

Despite this fact the evaluation team is drawing attention to some findings regarding day-to-day and

overall management which could be used to still improve overall performance, efficiency and results-

orientation in potential future projects:

Clear roles and responsibilities within the overall management hierarchy – involving Khujand,

Tashkent and Bonn – have been weakly documented. The chosen set-up bears consequences for

project efficiency. For example: Are four different management positions involved in the given project

– 2 of which in Tajikistan, 1 in Tashkent and one in Germany justified, and if so, how do they

coordinate their activities? Coordination and communication between the different levels as well as

oversight functions might have benefited from the existence of an overall work plan assigning roles

and responsibilities matched with the targeted project results.

As becomes clear from many of the more detailed findings below, project documentation on the

parts of ASTI and dvv international Tashkent, but also on the part of the individual VET schools could

have been more elaborate and systematic.

Project efficiency and financial management

Financial management

Certain issues regarding financial management had been pointed out already by the internal project

evaluation undertaken by dvv international Bonn in August 2010: “Due to the delay of project

activities the budget balance for the remaining implementation period is high (in relation to the

remaining time), creating the need for tighter budget management. The budget provides for

expenditure to be made at different locations and by different actors / offices. While each office (dvv

international in Bonn and Tashkent and ASTI in Khujand) seems to be well aware and up to date on

the actual expenditure for their location and activities, there seems to be a lack of effective central

steering when it comes to the overall expenditures and savings by budget item. In particular when

savings in specific budget lines create options to transfer the balance to a different location and

activity (which might be short of money), there is so far no standard communication procedure in

place. Usually this does not create problems in the beginning of project implementation, but can

create shortcomings at the end.” (Internal evaluation report p.11)

The evaluation team would agree that improvements are possible regarding internal communication

and systematic organisation of functions and duties of the various managers involved in financial

management of PATENT. It was not sufficiently clear from our interviews and observations who was

responsible for what in expenditure monitoring and oversight.

Page 10: PATENT Final Evaluation Report Summary - TEXT · Final Project Evaluation Report PATENT Page 4 of 12 2. Summary The project activities were implemented as planned, with only very

Final Project Evaluation Report PATENT Page 10 of 12

Project efficiency

The evaluation team calculated the ratio of expenditures directly impacting the target group vis-à-vis

the ratio spent on project management and administration. According to this, amounts spent on

activities directly benefiting the target group amounted to 64% of the total budget. Out of the total

budget, 27% were spent on management and administration-related activities of the project

implementer while 9% were spent on various staff of dvv international (Tashkent and Bonn).

2.2 Summary of major recommendations

Institutional impacts

Cooperation between stakeholders

(recommendation to dvv international and EU/BMZ, national and international partners)

Attention should be paid during potential future projects by both dvv international and any

implementing organisation on the ground to the involvement of both the Ministry of Education and

the Ministry of Labour and Social Protection in the VET sector. Any future project should keep in mind

as an overall goal the establishment/ furthering of good cooperation and coordination between the

two ministries and their partners and their mutual influence on results achievement. In their support,

international organisations should prioritise those adult and professional education institutions which

are really in need no matter which part of the VET sector portfolio (MoE or MoLSP) they belong to.

A more in-depth assessment of the status of the vocational education sector is highly recommended

to take well-adapted decisions in support of the VET schools. An assessment could be carried out by

interested international organisations and partners.

Sustainability

(recommendations to dvv international and ASTI)

More emphasis could have been given to sustainability aspects from the side of dvv international

during project planning and oversight. The sustainability of project outputs like certain services,

committees or boards – e.g. the advisory boards and business consulting desks of PATENT – should

have been taken into account. Strategies and plans could have been developed during planning

and/or implementation in order to improve their potential sustainability.

Labour market analysis

(recommendation to VET school management, MoE, MoLSP)

It is highly recommended that the labour market analysis be systematized and conducted at regular

intervals by those stakeholders best positioned for this task in the vocational sector and their

partners. Locally this might be an activity to be taken up by individual VET schools or a group of

schools situated close to one another. To produce more general analyses however, the labour markets

and economic and social situation in Tajikistan, Russian and Kazakhstan would have to be considered.

Such an exercise would probably have to be carried out by the relevant ministries with the assistance

of international partners like ILO, IOM and others. A pragmatic idea for the level of the VET schools

could be to conduct a survey among former graduates whenever they might do a next labour market

analysis in order to generate data on the employment situation of their graduates.

(recommendation to EC and BMZ)

In any project focusing mainly on capacity-building among individuals and institutions it will be

difficult to achieve lasting change in a two-year timeframe. Support to dvv international for future

initiatives in a similar direction or for follow-ups to the input provided as part of PATENT might be

considered.

Page 11: PATENT Final Evaluation Report Summary - TEXT · Final Project Evaluation Report PATENT Page 4 of 12 2. Summary The project activities were implemented as planned, with only very

Final Project Evaluation Report PATENT Page 11 of 12

Monitoring and evaluation

(recommendation to dvv international and EC/BMZ)

In future projects monitoring and evaluation should be better integrated in project planning and

implementation. Related activities including the generation of baseline data should be planned,

sufficiently resourced and then carried out systematically according to plan. Timeframes for

evaluation activities should be kept. The planned M&E system should certainly provide detailed

information on activity implementation, but also take into account questions related to project

impact.

(recommendations to dvv international and ASTI)

Monitoring and evaluation data should be well kept and made available to all concerned with project

implementation and management in formats that are easily understandable.

Especially with regards to the indicator for result No.1 – it should be strongly considered by dvv

international and ASTI to still invest into some more mini-surveys to generate data on the

employment situation of PATENT graduates.

Project planning and logical framework

(recommendations to dvv international)

In future interventions attention should be paid during planning on the potential need to conduct

well-adapted, cost-efficient baseline studies at early stages in order to generate data that can later be

vital for the evaluation of some important project results and impacts.

Impact-related indicators and sources of verification rather than only activity-related indicators (like

for results 2 and 5 of PATENT) should be developed wherever possible to measure and demonstrate

project impacts – something that should be kept in mind in future potential initiatives.

Risk assumptions estimated during project planning

(recommendation to ASTI)

To mitigate the risks to implementation and sustainability caused by high turnover of school directors

(four out of ten already during the on-going project implementation), it is advisable that deputy

directors are included in most trainings and management-related activities as far as possible.

Knowledge-sharing events should also be encouraged at the level of schools and included in project

activities so that the knowledge and skills acquired also by teaching staff are disseminated more

widely and retained in the schools once trained staff members leave.

Poverty alleviation and the selection of target groups

(recommendation to dvv international and implementing partners)

The process of selecting VET schools as beneficiaries and partners in future interventions is seen as

crucial for the achievement of project results – in terms of the varying levels of VET schools’

capacities and willingness to cooperate – and for the attainment of overall objectives such as poverty

reduction taking into account the schools’ catchment area. It is thus of utmost importance to do this

activity well. In future, dvv international should pay very active attention to the way the selection

process is undertaken by the implementer and the Ministry of Education.

Overall project management

(recommendations to dvv international)

In future project initiatives it would be advisable to develop certain management documents like an M&E

plan and an overall work plan document in some more detail. This process can serve the clarification of

management inputs and responsibilities during project planning and then offers an improved the basis for

implementation.

A filing system should have been kept by dvv international to ensure the availability of information

Page 12: PATENT Final Evaluation Report Summary - TEXT · Final Project Evaluation Report PATENT Page 4 of 12 2. Summary The project activities were implemented as planned, with only very

Final Project Evaluation Report PATENT Page 12 of 12

relevant for project management, monitoring and evaluation and reporting in one central place. This

would also be a good source to demonstrate some of the outputs and positive results of projects.

For future projects an oversight of project implementation and monitoring and evaluation will be more

effective through the office of dvv international in Dushanbe in order to ensure a closer coordination with

implementing partners and stakeholders and to closely monitor the activities in the place. Assigning these

functions to the Tajikistan country office will improve efficiency in monitoring and ensure quick reactions

from the side of dvv international in case of need.

Documentation and transparency at school level

(recommendation to VET school management and its partners)

As different actors are supporting the schools, it would be good to improve the records management and

filing systems of the schools, especially a database of received external and internal inputs could improve

management and serve as a good information basis for national and international stakeholders.

Elements to be applied in potential future projects

(recommendations to dvv international and the EC)

We recommend that it would be pragmatic and efficient to use standards adopted in different fields as

part of the PATENT intervention in future projects where applicable: questionnaires and methodology for

the labour market analysis, criteria and procedures for the selection of students and methodology of

curriculum development.

Trainings in results-oriented programme and project management in order to improve project planning,

implementation and M&E aspects for future potential project managers of ASTI or other potential partner

organisations implementing future projects are recommended.

Project efficiency and financial management

(recommendations to dvv international)

In future initiatives of dvv international cooperating with a local implementer, the ratio of administration

and management costs to expenditures directly benefiting the target groups of PATENT should be kept in

mind when planning for staffing structures and equipment provided to the implementer. It might be

worthwhile considering leaner and less complex management structures both with a view to increasing

the proportion of the budget spent on target group benefits and with a view to facilitating project

management by arranging the levels and responsibilities involved more clearly.

Tailor shop with four employees opened with PATENT grant