promotion of employment through training · 2011-03-01 · pett-terminal evaluation the european...

PETT-TERMINAL EVALUATION

The European Union’s 9th EDF Programme for Somalia

Promotion of Employment Through Training

CONTRACT NUMBER: 527-NGO-ED-4-05/EC

Terminal Evaluation Report

February 2009

MORRIS W. MATHENGE ABDI OSMAN ABDILLAHI HYDRO TECH CONSULT P.O. BOX 390-60300 P O BOX 755-00618 ISIOLO, KENYA Tel: +254-721-235 036 [email protected] [email protected]

This evaluation report was prepared at the request of SCD and with the financial support of the Commission of the European Union. The views expressed are those of the Consultant and do not necessarily reflect those of SCD or the Commission of the European Union

mailto:[email protected]


PETT-TERMINAL EVALUATION - 2 -

ACKNOWLEDGEMENTS The Consultants would like to appreciate and acknowledge the input and contribution of the many stakeholders which made this study possible, especially the European Union for funding the evaluation study and Save the Children Denmark (SCD) for facilitating and coordinating the study. Gratitude goes to the SCD Country Director, Deputy Country Director, the Project Manager, and other SCD staff members for their invaluable guidance, inputs, logistical support, provision of project reference documents and information sharing. Thanks also go to the consortium partners CARE, DS and GI whose contribution added value to the information contained in this report. The concerted effort of the enumerators in organizing meetings and discussions with the respondents ensured stakeholder participation, involvement and consultation. This enabled valuable data collection and is highly acknowledged. Special thanks and gratitude also go to the relevant Ministries in Somaliland and Puntland, the TVET institutions, employers, parents and the direct beneficiaries for participating in individual interviews and group discussions in providing valuable inputs in terms of time, discussions, information and support wherever and whenever help was needed during the study. Acknowledgements also go to all who participated in the validation workshops for their valuable contributions and feedback. Finally the Consultants wish to gratefully acknowledge all those who helped the evaluation team in one way or another to compile this report within the limited time. It is our firm belief that this evaluation study will play a vital role in planning, strengthening and implementing TVET programmes undertaken by the consortium partners in the future. Consultants Morris W. Mathenge (Lead Consultant) Abdi Osman Abdillahi (Co-consultant)


LIST OF ACRONYMS AND ABBREVIATIONS ANDP Activists Network for Disabled people CARE UK Cooperative for Assistance and Relief Everywhere (United Kingdom) EBTVET Enterprise Based Technical Vocational Education Training EC European Commission EFA Education For All EPS Employment Promotion Services DS Diakonia Sweden FFT Food for Training FGDs Focus Group Discussions GAVO General Assistance & Voluntary Organisation GI Gothenburg Initiative HTI Hargeisa Training Institute IAEC Integrated Adult education Centre IBTVET Institution Based Technical Vocational Education Training IDPs Internally Displaced Persons IGAs Income Generating Activities LCD Least Developed Country LED Local Economic Development LIP Local Implementing Partner MDGs Millennium Development Goals MOE Ministry of Education MOL Ministry of Labour NFE Non Formal Education NGO Non-Governmental Organisation PCM Project Cycle Management PETT Promotion of Employment Through Training SCD Save the Children Denmark SNEP Somaliland National Education Policy TOR Terms of Reference STEO Skill Training for Employment Opportunities TVET Technical Vocational Education Training UNDP United Nations Development Program UNESCO United Nations Educational Scientific and Cultural Organisation UNHCR United Nations High Commission for Refugees VTC Vocational Training Centre WFP World Food Programme


TABLE OF CONTENTS

ACKNOWLEDGEMENTS ............................................................................................................ 2

LIST OF ACRONYMS AND ABBREVIATIONS ............................................................................... 3

TABLE OF CONTENTS.............................................................................................................. 4

EXECUTIVE SUMMARY............................................................................................................. 6

1.0 INTRODUCTION .......................................................................................................... 14

1.1 PROJECT BACKGROUND ...................................................................................... 14

1.1.1 The PETT Project ............................................................................................ 14

1.1.2 The Beneficiaries ............................................................................................. 16

1.1.3 Project Objectives ........................................................................................... 16

1.1.4 Project Location .............................................................................................. 17

1.2 PURPOSE / OBJECTIVE OF THE EVALUATION ........................................................ 17

1.2.1 Scope of Work ................................................................................................ 18

1.2.2 Methodology and Approach .............................................................................. 19

1.2.3 Sampling ........................................................................................................ 20

2.0 KEY FINDINGS AND DISCUSSIONS ............................................................................... 22

2.1 RELEVANCE ........................................................................................................ 22

2.2 EFFICIENCY ........................................................................................................ 24

2.3 EFFECTIVENESS .................................................................................................. 28

2.4 PROJECT IMPACT ................................................................................................ 49

2.5 SUSTAINABILITY ................................................................................................. 52

2.6 CROSS-CUTTING ISSUES ..................................................................................... 53

2.7 STRENGTHS AND WEAKNESSES ........................................................................... 55

3.0 CONSTRAINTS AND CHALLENGES ................................................................................ 56

4.0 LESSONS LEARNED ..................................................................................................... 56

5.0 CONCLUSIONS AND RECOMMENDATIONS..................................................................... 57

5.1 CONCLUSIONS .................................................................................................... 57

5.2 RECOMMENDATIONS ........................................................................................... 61

6.0 ANNEXES .................................................................................................................... 65


LIST OF FIGURES Figure 1: Gender enrolment in Somaliland during the 3-year project duration……………………………… 30

Figure 2: Gender enrolment in the four towns of Punt land……………………………………………………… 31

Figure 3: % of TVET (both IBTVET and EBTVET) trainees desegregated into gender...........................31

Figure 4: Age of respondents interviewed during this study in the four Somaliland towns....................32

Figure 5: Selection criteria of beneficiaries in Somaliland...................................................................32

Figure 6: Selection status of beneficiaries in Punt land disaggregated into gender...............................33

Figure 7: Hargeisa Technical Institute before and after renovation…………........……………………34

Figure 8: The various skills offered by the PETT project……………………………………………………..36

Figure 9: Gender enrolment of the skills offered during the PETT project…………………………….36

Figure 10: Case study 1……………………………………………………………........……………………………..50 Figure 11: Case study 2…………………………………………………………………………………………………..51 LIST OF TABLES Table 1: No. of direct beneficiaries contacted based on location and gender………………………..21

Table 2: Comparison between the original (approved) and revised budget (EOP)…........……….24

Table 3: Total number of TVET Trainees....................................................................….......37

Table 4: Employment status in Punt land………………………………………………………………………….38

Table 5: Employment status in Somaliland………………………………………………………………………..38

Table 6: Number of trainees who utilised EPS ………………………….........……………………………… 40

Table 7: Total number of NFE trainees ………………………….........……………………………………….. 41

Table 8: Percentage of trainees who passed exams.…………….........…………………………………….43

Table 9: Management of assumptions and risks………….........……………………………………………..48

Table 10: Overall project scoring …………………….......…………………………………………………………60

LIST OF ANNEXES ANNEX I: TERMS OF REFERENCE

ANNEX II: EVALUATORS

ANNEX III: LIST OF DOCUMENTS CONSULTED

ANNEX IV: EVALUATION TOOLS

ANNEX V: LIST OF PERSONS CONTACTED/SAMPLE SIZE

ANNEX VI: LOGICAL FRAMEWORK MATRIX


EXECUTIVE SUMMARY Background The Promotion of Employment Through Training (PETT) has been the centrepiece of EC’s efforts to promote employment opportunities in Somalia targeting women, school leavers, ex-militia and other disadvantaged groups. Started in 2005, the 3-year PETT project has been implemented by Save the Children Denmark (SCD) as the Lead Agency of consortium partners namely, CARE UK, Diakonia Sweden (DS) and Gotenburg Initiative (GI) in Somaliland and Puntland. It is implemented in close partnership with local actors (Government and the private sector) with funding from the European Commission (EC). The initiative was grounded in the premise that by targeting these beneficiaries, the programme would reach a major proportion of the population that has been denied access to education and increase their employability. This would in turn translate to improved livelihoods to the beneficiaries, their families and the country as a whole. The overall objective of the project was to improve the livelihood of disadvantaged youth and women in both Somaliland and Puntland thus contributing to reduce their vulnerability. Consequently, the origin of the project, its preparation, its objectives, its activities, and structures were geared to sustainably improve the livelihood and employability opportunities of 4,525 beneficiaries through provision of technical and vocational education and training (TVET) and by availing information on employment opportunities. It was based on an EC commissioned feasibility study that was conducted in June 2004. The expected project results were:

1. Improved access of disadvantaged groups to TVET and to employment promotion services (EPS).

2. Competencies of the target groups increased in accordance with employment opportunities.

3. Improved availability and use of information on local economic development (LED) opportunities.

4. Capacities of local implementing partners and authorities increased with respect to TVET and employment promotion.

It was anticipated that 75% of those trained would be in sustainable employment within six months after the training. Objectives of the evaluation Part of the deliverables of the project was to document its achievements against the stated objectives through undertaking a terminal evaluation. The intention of the terminal evaluation is principally for learning and accountability purposes. The study sought to assess the performance of the project against key parameters of relevance, efficiency, effectiveness, impact and sustainability as well as timeliness of implementation, lessons learnt, strengths and weaknesses. The evaluation has equally revealed a number of strategic issues that are interwoven.


This evaluation also lays a hallmark through which SCD or other agencies can replicate in future TVET programmes. On the other hand, it will serve as a management tool and contribute to improved future performance, results and knowledge within the European Agency for Reconstruction in consistency with international practice and the EC evaluation guidelines. Methodology Evaluation data was collected through: (a) comprehensive review of project documents, (b) individual interviews and focus group discussions with project beneficiaries (c) in-depth interviews with key partners involved in the design and implementation of PETT including consortium partners, government ministries and training institutions and (d) direct observations and most significant change stories. To the extent possible, information was triangulated, validated and cross-checked to analyse the aspects that determine the five parameters, namely: relevance, effectiveness, efficiency, impact and sustainability. The documents reviewed included the project proposal (including the logical framework), local labour market survey reports, local economic development survey reports, MOE policies and strategic plans, technical and financial reports, tracer study reports and interim project reports among others. The interim reports were found to be particularly helpful in examining the project progress in the 3 year timeline. The evaluation sampling criteria was done through consultation with the project staff and MOE officials to ensure equity in gender, geographical distribution, skills and batches. The Consultants conducted the evaluation for a total of 30 days, with 18 days spent in the field. The rest of the period was spent on technical analysis, validation and presentation workshops, report writing and submission.

Key Findings Relevance: This evaluation found that PETT was a relevant programme in addressing the identified and real needs of the beneficiaries. The target group lagged behind in education and learning opportunities as a result of the civil war that raged on in Somalia after the fall of the Siad Barre regime which completely shattered lives and displaced large populations. As a result, many of the target groups lacked education and access to skill training and hence employment opportunities. The objectives of the project were geared towards these constraints and have been judged to be very relevant throughout the implementation period.

The project activities and approaches are also in line with the Education for All (EFA) Dakar Framework and the Millennium Development Goals (MDGs) of eradicating extreme poverty and hunger, promoting gender equity and empowerment, achieving universal primary education, ensuring environmental sustainability and developing a global partnership for development. It also resonates well to the EC support in Somalia in the education and


training sector which aims to improve the quality and relevance of secondary and tertiary education, vis-à-vis employment opportunities; and to reduce youth unemployment by at least 50% for both genders. The relevance of giving support to vocational training centres is evident from the number of disadvantaged young men and women trained within 3 years with a drop-out rate of less than 4%. It has addressed the needs of new adolescents coming ‘on board’ every year in harnessing much talent and energy for the development of the individuals, their families and the wider community and reducing potential for trouble and instability. It is notable that the PETT project paid special attention to the critical issue of identifying and supporting opportunities for employment of women, many of whom were illiterate. This included the provision of functional literacy and numeracy skills, appropriate identification and provision of market-driven and women-friendly technical/vocational skills as well as encouraging women to take employment opportunities in male-dominated technical/vocational occupations. Due to its relevance, the PETT project has been able to ‘connect’ with a similar project albeit with a new name – Skill Training for Employment Opportunities (STEO) – (to reflect inclusion of lessons learned through monitoring and evaluation), also implemented by SCD and its consortium partners and funded by the European Commission. The PETT project has also been able to attract extra support from other donors namely UNESCO, UNHCR (through provision of start-up toolkits) and WFP (through FFT and provision of non-food items/tools) and Band Aid Trust Foundation (through provision of start-up tool kits and curricular development activities). After the analysis of all the parameters that determine relevance the conclusion is that the project relevance is highly satisfactory. Efficiency: PETT adopted a participatory approach whereby it provided major external support for investments (mainly infrastructure, training and materials) against a commitment from the governments of Somaliland and Punt land to be the local counterparts and the TVET training centres to be the implementing partners to ensure they would cover future project activities (amortization of investments, operation and maintenance of infrastructure, training, salaries, etc). This was to ensure ownership of the project activities. Review of the project documents revealed that project resources were satisfactorily transferred into the intended results since the target numbers of beneficiaries have successfully completed the intended training and majority of them have accessed the labour market either in the formal or the informal sector.

The TVET implementing partners assumed the role of project implementation and management with support from the consortium partners and the local counterparts. This meant that they received and managed the training funds for the project and co-ordinated activities at field level. In all these roles they performed fairly well. Moreover, the ability of


the institute managers to mobilise community support on the local level assisted in promoting the project to the intended beneficiaries. SCD and its partners support the training institutes from their field offices in Hargeisa, Somaliland and Garowe in Puntland. This ensured ample logistical and technical assistance to the local partners in the implementation of the project. The project also supports the NFE Department and the TVET unit in MOE and EPS unit in the Ministry of Labour (MOL) to develop their capacity to manage activities in their dockets. The planning and co-ordination function was well undertaken by the consortium partners and the local counterparts (though this was observed to be weak in Punt land). Consistent interactions in steering committee meetings of the stakeholders to review project progress helped in achieving project results. The partners were adaptable and able to resolve problems, provide advice and support and helped to build the capacity of the training institutions through field visits from time to time. The donor, EC, also maintained a close interest in the project. The project was able to complete its core activities within the proposed timeframe and budget, albeit a 3-month no-cost extension. Overall the efficiency of the project was considered satisfactory. Effectiveness: In targeting 4,525 disadvantaged youth, ex-militia, minority groups, out-of-school youth (of which at least 40% would be women), the project was designed to sustainably improve their livelihood and employability opportunities through provision of technical and vocational education and training (TVET) and by availing information on employment opportunities, and thus reducing their vulnerability. The project design was such that there would be equal enrolment of vulnerable men and women within the age bracket of 16-24 years, who have had no or limited access to education, training and employment opportunities. Gender distribution, age, educational background and vulnerability level were factors in the selection criteria. The study established that the overall gender representation in TVET was 52 % male and 48 % female. It was also established that 74% the respondents were aged between 16-24 years while 26% were aged over 25 years. The study showed that 64% of the respondents were enrolled on the basis of limited education, 18% returnee, 5% marginalised women, 12% ex-militia and 2% for other reasons. Due to the interventions, 5 vocational training centers have been renovated and made operational in the towns of Hargeisa, and Berbera (in Somaliland); Bosasso and Galkayo (in Puntland). Identification and selection of trainees was participatory and conducted by a panel consisting of MOE/TVET, IBTVET, EPS project officers, community elders, youth representatives with due gender consideration.


In the IBTVET component of the project, 2,548 trainees (1,002 (39%) female and 1,546 (61%) male) underwent a 7.5 months training in the 5 IBTVET centres. A total of 2,050 trainees (942 (50%); female and 1,108 (50%) male) graduated in the EBTVET component of the project. The project also saw the training of 2,957 (1,838 (62%) women and 1,119 (38%) men learners in the twelve NFE literacy and numeracy centres in Somaliland and Puntland. A total of 3,436 trainees of which 1,377 females (representing 40.2%) benefited from the EPS activities comprising of trainings on business planning, resume writing, career counselling and interview techniques. This represents 69% of the target figure of 5,000. The findings of the evaluation indicate that the graduates’ rate of employment was at 72% in Somaliland and 70% in Puntland compared to the expected rate of 75%. Overall, the project effectiveness was rated as highly satisfactory. Project Impact: The underlying factor of PETT was to address unemployment of the youth. The general picture seems to be that graduates appreciate the training they received. 98% of all the employed graduates indicated they are in employment or have started their own businesses applying skills they had acquired. All indicated that their livelihood has improved. Rehabilitation of the infrastructure has guaranteed access to vocational education among the beneficiaries thus reducing their vulnerability and increasing their socio-economic well-being. The study has established that 4,598 young men and women (i.e. 102%) have been trained in these institutions. Many more are likely to be trained in the institutions in the future. The increased competency in technical and entrepreneurship skills did result to improved opportunities for gainful employment. The participatory approach of the project has built capacities in TVET management and training as well as developed standardized curricula materials for new skills which have been accepted and adopted by the government and distributed to training centres. This was cost-effective because the materials will be used by many more beneficiaries in the future. The project has proved its impact with the increased participation of women. It has enhanced women positioning in the socio-economic well-being as some have started their own businesses or have been absorbed in gainful employment as a result of the intervention. These enhanced capacities have promoted their self-reliance and the well-being which will contribute to national development especially to child protection. The attention and emphasis given to the women participation was a plus in ensuring gender equity. In summary the project has been beneficial to the target beneficiaries, which will have a wider overall effect on a larger number of people in the different categories, their regions and in the country as a whole. These achievements contributed directly to achieving the overall objective and the impact was therefore found to be highly satisfactory.


Sustainability PETT was designed to focus on the establishment of long-term capacities for local counterparts and partners to continue activities in the delivery of service once the external aid ends. The project put more focus on the development of local infrastructure and reinforcing existing institutions which contributed to an increased sense of local ownership. Institutional sustainability has been increased by working more in collaboration with local structures. This is evidenced by the adoption of the TVET policy by MOE, the establishment of an EPS unit in MOL and the adoption of examination guidelines (that resulted to standardised examination. The building of capacities within the governments and the TVET centres through training and on job mentoring will contribute to long-term sustainability. In terms of financial sustainability, SCD has been working in an environment where the governments are in a weak position to maintain or finance vocational educational support. However, the likelihood of the governments and the TVET centres sustaining activities initiated by PETT beyond external support appears to be fairly substantial. This is evident from the fact that the Somaliland MOE has accepted to support TVET through payment of salaries to all the TVET trainers/instructors. On the same note, the development of the financial (cost-recovery) and management systems manual will assist the TVET centres to start income generating activities. The great challenge will be to ensure the governments and the TVET centres are moving at the same pace. The education and competency provided to the beneficiaries is life-long allowing the flow of benefits to their families and the society at large which increases the likelihood of continuity of these benefits even after the conclusion of the project. After analysis of the measures that have been put in place for sustainability, the conclusion is that it was satisfactory.

Finale The PETT project was designed particularly to address widespread unemployment and poverty among disadvantaged groups as well as inadequate capacity of the local partners in the implementation of the TVET activities. Consequently, PETT activities were directly aimed at reducing these constraints through improved access to quality marketable and demand-driven skill training and employment promotion services. The projections for enhancing skill trainings and increasing employability as well as competency and local capacity to implement long term development projects was adequately addressed. However, more effort is needed to enhance employment promotion services and dissemination of information on local economic development studies/reports to the beneficiaries. By and large, the project was well implemented and timely.


Lessons Learned The direct involvement of local counterparts and stakeholders in PCM is vital to the success of a project because it provides a direct link of project performance to systems of local control and develops a sense of ownership, which greatly contributes to sense of ownership and sustainability. Private enterprises play a crucial role in TVET activities as host-trainers. Their involvement in the PETT project enhanced the project’s impact and cost-effectiveness and if well and carefully supported through financial and technical advice, their participation leads to improved project performance. The inclusion of cross-cutting issues like HIV/AIDS, peace and democracy, FGM and environmental sustainability into the NFE component of the project as well as into the skill training syllabi is a good approach of advocating for these issues. Challenges The availability of qualified trainers/instructors and curricula materials, especially for new skills is still a challenge in Somalia. The same applies for women trainers. Selection of new skills was problematic due to non-availability of relevant curricula and experienced trainers. Furthermore most of the trainers have low English language comprehension. The relevant Ministries are still understaffed and under resourced which makes it difficult for appropriate follow-up on project activities, especially the financial accountability in the training centres. Gender discrimination and stronger affiliation to clanism is still evident in acquiring employment opportunities. Recommendations The project design remains relevant in the Somalia context and can be replicated, though adjustments should be made in the Log Frame to include measurable effect indicators, especially for some of the activities within results 2, 3 and 4. Some of the other recommendations made are given below.

Management and financial accountability within the institutions and enterprises need to be closely monitored and the relevant personnel mentored through on-job training. This will enhance the implementation of cost-recovery systems that in turn would promote self reliance and sustainability.

Strengthening of the EPS component at the ministerial level is strongly

recommended since it is key to linkages between the employer and the trainee, especially taking into consideration that the project purpose is to assist the beneficiaries get gainful employment. The Ministries of Labour should also develop coherent and comprehensive youth employment policies to support EPS activities. As


well, TVET and EPS units could possibly be converted to full-fledged ministerial departments.

There is a need to establish national and regional TVET departments to be

responsible for the management and implementation of the approved curriculum by all institutions offering similar TVET courses. The directorate of inspection could further ensure that curriculum materials are standardized for all NGOs implementing TVET projects. As well, a standardized skill grading system should be introduced such that learners graduate on various categories of grading depending on proficiency levels.

The study established that the user fee and training duration is dependent on type of

skill. Therefore the blanket cost and time appended to all the skills should be reviewed to improve cost-efficiency. Some skills require more investment than others. Similarly some require more time than others.

Clear articulation pathways need to be established between NFE and TVET

components as well the feasibility of establishing a centre of excellence for advanced skill levels should be explored. The long-term goal of upgrading Hargeisa Technical Institute in Somaliland and Galkayo IBTVET Centre in Puntland into national polytechnics that will train technical human resource at certificate, higher national diploma and/or degree level is strongly recommended. However this requires careful planning and heavy investment.

The Somali language is widely used in the training since few students (and some

trainers) have an adequate command of English to understand the curricula. The trainers therefore have to spend considerable time translating from English textbooks to Somali for the benefit of their students. Although some of the curricula materials have been translated into the Somali language, it is recommended that efforts be made to translate all curricula materials into the Somali language.


1.0 INTRODUCTION

1.1 PROJECT BACKGROUND

1.1.1 The PETT Project Somalia has been defined as a Least Developed Country (LDC) and ranks amongst the ten poorest countries in the world. In 2003 UNDP estimated that over 43% of the population live in extreme poverty (i.e. on less than one US dollar a day) while 62% of the urban labour force and 41% of the rural/nomadic labour force are reportedly unemployed. Only 56% of the labour force is estimated to be economically active1. During the protracted war that erupted in 1991 following the fall of Said Barre’s government, majority of the population lost their economic assets, including homes, farms, land and jobs, and went into exile or became internally displaced. Somalia’s political and social disintegration brought the economic development on its knees. However, those in the relatively stable North West Somalia (Somaliland) and North East Somalia (Puntland) resettled in their hometowns and villages and are striving hard to rebuild their live and economic bases. The dynamic and expanding private sector that is growing steadily is supplying the majority of jobs and income for the people and appears to be one of the fundamental elements for the country’s reconstruction. However, the sector is challenged by the low supply of qualified and skilled labour force. Most of the educated and skilled labour force has either fled the country or are in their twilight years. The majority of the young population is uneducated and unskilled. Less than 17% of the youth have basic literacy and numeracy skills. Most of the youth lack marketable employment skills and can only be employed in the livestock sector. The ministry of planning estimates that nearly 70% of the youth are unemployed. There has been a tremendous increase in the number of unemployed youth in major towns and urban centres of Somaliland/Puntland in the last few years. Limited capacity and infrastructure, continued influx of returnees, increasing number of school leavers, increased rural-urban migration and worsening economic conditions as a result of the livestock export ban have contributed to worsening the chances and opportunities of the youth in obtaining gainful employment opportunities. The immediate causes of the high rate of unemployment is also exacerbated by the existence of a large number of disadvantaged people, lack of government economic structures, loss of traditional livelihood systems and limited availability of people with employable skills and basic education. There is also a limited availability of jobs and livelihood opportunities due to over-reliance on traditional economic activities and limited capacity to recognize economic development opportunities to start and expand businesses and livelihood activities. The traditional coping mechanisms seems to have failed and unless action is taken in order

1 EC Support to Employment Promotion in Somalia, Feasibility Study Report, June 2004


to improve employment opportunities and the livelihood situation the problems will continue to mount and stability of the society will be threatened. “Rising unemployment takes a heavy toll among young people who are particularly vulnerable to shocks in the labour market. Insufficient opportunities to enter the world of work condemn many to a life of economic hardship and despair. We have seen, all too often, the tragedy of young lives misspent in crime, drug abuse, civil conflict and even terrorism...........” The Director General, Ministry of Labour (Somaliland)

Reiterating the aforementioned statement of the DG, the feasibility study commissioned by EC in June 2004, revealed that whilst local economies are growing at a rapid rate in many areas, they are starting from a very low base. Notably, resources are limited and there are no significant labour market opportunities. At the same time, there are only marginal prospects for local economic development in some areas. As Somaliland/Puntland faces an almost overwhelming demand for more and better jobs, there is a pressing need to provide skill training and employment promotion services to these ever-increasing number of youth.

It was against this background that the European Union funded the Promotion of Employment Through Training (PETT) project in Somaliland/Puntland. The project aims to sustainably improve the livelihood of the vulnerable youth through provision of skills training and employment promotion services. The project is implemented by a consortium of CARE UK, Diakonia Sweden, Gutenberg Initiative and Save the Children Denmark. The latter is the lead agency in close partnership with local actors (Government and the private sector).

The three year project was funded to the tune of Euro 2.5 M. The preparatory activities started in September 2005, while the initial training of the first batch started in January 2006 and ended in July 2006. The training duration for IBTVET trainees was seven and half-months (including examinations and attachment) while EBTVET training was six months. The second batch was admitted in January 2007 and the third batch in March 2008. The project is in line with one of the priorities of the EC Intervention Strategy and Policy Considerations for Education and Training Sector in Somalia2 – ‘In order to improve the education sector contribution towards peace building and better governance and improve the livelihood of focus target groups, a combined approach of literacy adult education coupled with skills training and economic measures is identified as a priority’ .

2 Draft document in support of the EC Strategy for the Implementation of Special Aid to Somalia, January 2004


1.1.2 The Beneficiaries The action primary target was 4,525 disadvantaged youth within the age bracket of 16-24 years of which 40% would be women and girls, who have had no/limited access to education, training and employment opportunities due to the protracted armed conflict that plagued the country. Other members of the disadvantaged groups include: internally displaced persons (IDPs), returnees, out-of-school youth, ex-militia, the physically disabled, and minority groups. The long-term beneficiaries of the project included parents and families of the trainees, enterprise owners where the target groups would receive training, instructors, managers, local authority personnel, staff of the implementing partners, private factories and the general public through spill over effects of the availability of skilled labour force. The action was also designed to collaborate with the local chambers of commerce and build their capacity to contribute to EPS, thus creating employment and increased household incomes. The interventions also included renovations of existing physical structures of the IBTVET centres and improving the management and operational capacities of both the IBTVET and EBTVET host-trainers in collaboration with the MOE in both regions.

1.1.3 Project Objectives The overall objective of the project is to improve the livelihood of disadvantaged youth and women in both Somaliland and Puntland thus contributing to reduce their vulnerability. Consequently, the origin of the project, its preparation, its objectives, its activities, and structures were geared to sustainably improve the livelihood and employability opportunities of the target group through provision of technical and vocational education and training (TVET) and by availing information on employment opportunities, and thus reducing their vulnerability. This was based on the EC commissioned feasibility study that was conducted in June 2004. The expected project results of the project were: 1. Improved access of disadvantaged groups to TVET and to employment promotion

services. 2. Competencies of target groups increased in accordance with employment

opportunities. 3. Improved availability and use of information on local economic development

opportunities. 4. Capacities of local implementing partners and authorities increased with respect to

TVET and employment promotion. It was anticipated that 75% of those trained will be in sustainable employment by the end of the project period. The project objectives accord well with, and reinforce, current efforts by local authorities in the education sector in trying to address the issue of youth unemployment


through provision of skill trainings and employment promotion services. In both regions, the importance of TVET has been acknowledged by local authorities in response to the challenge posed by the high rate of unemployed illiterate youth, the limited number of tertiary education opportunities, the growing number of secondary school graduates. The objectives, targets, indicators and intervention measures for TVET are set out in the MOE National Strategic Plan (2007-2011). Since the local authorities responsible for education face severe resource and capacity constraints, there is a strong case for capacity building to improve TVET delivery. The project was evaluated against its specific objective -‘employability of disadvantaged groups sustainably improved’ and the four major results. Thus the evaluation was carried out in order to determine its success, operational working and performance and monitoring of the ultimate results. The results of this study are important in the sense that the findings provide a process document for replicating similar projects in the future.

1.1.4 Project Location The project covered Somaliland urban centres of Hargeisa, Borama, Burao and Berbera and the towns of Bosasso, Qardho, Garowe and Galkayo in Puntland. The authorities in Somaliland and Puntland have to a certain degree succeeded in establishing a functioning administration, promoted internal security and stability and created a positive environment for social and economic development. These zones are more politically organized having their own self administration.

1.2 PURPOSE / OBJECTIVE OF THE EVALUATION Part of the deliverables of the project is to document its achievements against the project document and the log frame; for learning and accountability purposes through undertaking a terminal evaluation of the project and assess the achievements and document lessons learnt from the project implementation. Evaluation surveys are effective mechanisms for bringing in systematic changes and eliciting alternative strategies for improving and sustaining interventions that is increasingly emphasized in community development. The findings herein are based on an evaluation conducted to improve the quality, outcome and accountability of the PETT project to the targeting beneficiary country. This evaluation will thus serve as a management tool and aim to contribute to improved future performance, results and knowledge within the European Agency for Reconstruction in consistent with international practice and the EC evaluation guidelines. In line with the terms of reference, the specific objective of the assignment was to evaluate the level of achievement according to the five Project Cycle Management (PCM) criteria (Relevance, Efficiency, Effectiveness, Impact and Sustainability) and to learn lessons for application to future projects. In endeavouring to address the


aforesaid criteria, the Consultants strived to seek the views of the various stakeholders and especially the targeted beneficiaries, including cross-cutting issues like gender, minority, and social impact. Since the project is now completed, it is not the intention of this study to make detailed recommendations on the operational aspect of the PETT project. Furthermore, the many reviews and studies conducted in the course of the project provided recommendations most of which were taken up and helped to improve the implementation of the project. The recommendations in this report are intended to point the way to improve interventions and practice in the future.

1.2.1 Scope of Work The Consultants were contracted to undertake the terminal evaluation of the PETT project within its implementation context to cover the accomplishment of all the expected results as outlined in the project document. To achieve this, specific tasks undertaken included assessment of:

Extent to which set objectives have been realized and effectiveness of strategies employed

How the project resources have been utilized to achieve the stated results Current institutional and organizational set-up at the MOE; Establishment of network and linkage system between TVET centres and

relevant stakeholders; Development and implementation of policy guidelines; Presence of operational management, financial recovery systems at the TVET

centres; Strengths and weaknesses of the project and document lessons learned and

inform future TVET programmes Sustainability of TVET and EPS activities after project ends.

The evaluation therefore sought to assess the performance of the project along the following dimensions, as per the TOR:

i. Relevance i.e. how appropriate was the project design and to what extent the stated objectives correctly addressed the problems and real needs of the target groups.

ii. Efficiency i.e. how economically the project inputs (funds, expertise, time, etc)

converted into results in the required quantity and quality and in good time, whether the use of the project resources was cost-effective and if they could have been done more cost efficiently.

iii. Effectiveness i.e. the extent to which the stated project results and purpose

has been achieved in a sustainable way.


iv. Impacts i.e. the sustainable changes (positive/negative, intended/un-intended) that the project has produced on the target groups as well as MOE, TVET Centres and other stakeholders.

v. Sustainability i.e. the likelihood of the project to continue after end of the

project and the sustainability measures that have been put in place.

vi. Implementation processes i.e. the extent to which the activities of the project have been implemented in participatory and empowering manner and the involvement of the key stakeholders of the project in planning and execution of activities, and steering the project.

The evaluation also sought to:

1. Identify the strengths and weaknesses of the project to come up with findings, lessons and recommendations to guide and inform future programme work with special focus on the delivery of TVET and EPS.

2. Examine and give recommendations on the future expansion of the TVET (with

specific focus on the levels and skill areas) vis-à-vis the employment prospects/needs of the Somaliland/Puntland economy.

1.2.2 Methodology and Approach The evaluation was carried out in accordance with the evaluation guidelines of the European Commission3 and the Terms of Reference for the Consultants (Annex 1 and II). In order to make an assessment of all the intended outcomes/outputs, it was necessary to go beyond the review of secondary data and collect primary data related to the PETT project. This comprised of participatory approaches with all stakeholders including the PETT project staff and beneficiaries, Focus Group Discussions (FGDs), key informant interviews, field visits and observations including most significant change stories and scoring of the achievements against the logical framework. A list of the main documents consulted is provided in Annex III. These provided an insight into understanding the project background and the achievements and subsequently cross check the information with data gathered in interviews and discussions with the various stakeholders. Focus group discussions and individual interviews (one-to-one) were held with the project beneficiaries while in–depth key informant interviews were conducted with the training institutions, consortium partners and relevant government ministries. Both

3 A guide to the evaluation procedures and structures currently operational in the Commission’s External Cooperation Programme,

March 2001


structured and semi-structured questionnaires were used to gather quantitative and qualitative information from the different categories of respondents. These interview tools and guidelines are presented in Annex IV. Information collected from the respondents was expected to give insight into the perspectives of the project, the issues that have contributed to the performance of the project, effects and impacts, their perspectives on the way forward in case of future similar interventions as well as networking and collaboration between themselves and the implementing agencies. A full list of people met is attached as Annex V. During project visits, the consultants were able to observe project activities and results as well as scrutinize various tools and outputs of the project e.g. operational IBTVET centres, employed PETT graduates, curriculum materials used in the training centres and get information of the most significant change stories. The evaluation has documented one case study as an example of project effectiveness and impact (See Fig. 6). To facilitate the field visits, PETT project staffs and government officials accompanied the evaluation consultants, but were not present during key interviews. Observations and impressions from all these encounters guided the Consultants in the overall intuition of the project. They were particularly helpful in relating the discussions and interviews to what was observed on the ground. To the extent possible, information was triangulated, validated and cross-checked to enable consistency and help make an informed decision. Scoring of the logical framework provided in the project implementation document, which set out the objectives, indicators, activities and outputs, was also used as an overall guide to what the project was intended to achieve. It was also used in scoring the overall performance of the project.

1.2.3 Sampling For FGDs and individual interviews the respondents were selected randomly with the assistance of the implementing consortium partners and MOE officials, so as to reflect a balance of the different skills offered, gender, batches and of sufficient geographical spread to provide a consolidated holistic picture. For key informant interviews, the consultants employed purposive sampling in targeting the local institutions/partners, local counterparts and collaborating agencies. Sample size: The sample size comprised of 655 respondents of which 553 were direct beneficiaries i.e. the TVET graduates (Table 1). Other respondents included training institutions representatives, employers, MOE and MOL officers, parents and guardians, SCD and its consortium partners. A list of the different categories of respondents is annexed in Annex V.


Table 1: No. of direct beneficiaries contacted based on location and gender TOWN FEMALE MALE Total Hargeisa 88 26 114 Berbera 40 40 80 Borame 14 19 33 Burao 11 30 41 Sheick 10 0 10 Bossaso 27 43 70 Qardo 15 45 60 Garowe 30 42 72 Galkayo 38 35 73 Grand Total 273 280 553

Sample Distribution: The data collection made sure that information was collected from diverse sources. The study ensured that the 45% female representation was achieved. The study also ensured that all the geographical areas under the project implementation were covered. The distribution of the interviews was made prorate to the number of beneficiaries in each town. In Somaliland, more than 30% of interviews were in Hargeisa while the other 3 major towns were covered by two thirds of the interviewees because of the number of skills training and training institutions in the capital city. In Puntland, more than 30% of the interviews were conducted in Bosasso, the business hub and the main commercial town where most of the target beneficiaries were. The Process: After reviewing the project proposal, the Lead Consultant developed questionnaires and other tools for the study. For the field data gathering, the consultants were assisted by PETT project staff and selected enumerators who initially went through a training session to equip them with relevant data collection techniques. They were also involved in pre-testing of the tools around the main towns of Hargeisa and Garowe. The study team then mapped out the areas of coverage and planned on how they were to reach the targeted respondents who had been informed ahead of time. Collected data was “cleaned”, analyzed and triangulated to generate the contents of this report. Validation workshops were also conducted for stakeholders in Hargeisa and Garowe.

The consultants conducted the evaluation for a total of 30 days, with 18 days spent in the field. The rest of the period was spent on technical analysis, validation and presentation workshops, report writing and submission.


2.0 KEY FINDINGS AND DISCUSSIONS

2.1 RELEVANCE The relevance of a project relates primarily to its design and concerns the extent to which its stated objectives correctly address the identified problems or real needs i.e. the appropriateness of the project design to the problems to be resolved. The PETT project, as set out in the log frame (Annex VI), was designed based on a feasibility study commissioned by EC in June 20044. The core problem identified by the feasibility study showed that there was a very high rate of unemployment among young adults, which contributes to vulnerable livelihoods and widespread poverty of the target groups. To date, Somalia is still among the countries with the lowest literacy rates in the world due to the civil war that destroyed the little education infrastructure that had been developed by the Siad Barre Government. The country is now on the path of reconstruction and development, but a major constraint is the low level of education and lack of skilled human resources. The governments of Somaliland and Puntland therefore have put in place a policy framework to develop TVET within the framework of National Education Policy (SNEP)5 6. The PETT project initiative has played a leading role in this process to build a foundation for development and expansion of TVET. Recognizing the importance of TVET for the reconstruction and development, the Governments of Somaliland and Puntland has also decided, as a policy, to channel the majority of school leavers to technical and vocational training. The MOE education sector strategic Plan (2007-2011) states: ‘It is envisaged that by the end of the Education Sector Strategic Plan (2007-2011), 60% of primary school leavers will follow technical or vocational education.7’ One of the six goals in the “Dakar Framework for Action, Education for All (EFA)” which can be used as guidance to PETT, is the access to learning and life skills programmes and the improvement in levels of adult literacy; and the provision of vocational training for out-of school youth, including ex-combatants. Furthermore, the provision of livelihood skills is most relevant in relation to EFA goals. The project also targeted the disadvantaged groups in ways that are consistent with four of the eight Millennium Development Goals (MDGs). Goal No. 1 advocates the eradication of extreme poverty and hunger by reducing by half the proportion of people living on less than a dollar a day as well as those who suffer from hunger. Goal No. 2 aims at achieving universal primary education while MDG No. 3 concerns the promotion of gender equality and empowerment of women through elimination of gender disparity

4 EC Support to Employment Promotion in Somalia, Feasibility Study Report, June 2004 5 Somaliland TVET Policy Document of the Ministry of Education, August 2007 (Draft), funded by EU 6 The Puntland Policy for Technical Vocational Education and Training (TVET) Draft Report, Aug 2007 7 Ministry of Education, Education Sector Strategic Plan (2007-2011) May 2007 (Final consultative Draft)


at all levels of education. Goal No.7 aims at ensuring environmental sustainability. By promoting employment through training, the incomes and livelihoods of the disadvantaged have been enhanced. Through Non Formal Education (NFE), youth and adults of both sexes were trained on literacy and numeracy enabling them to cope with demands of EBTVET. Women were targeted as a special category of the disadvantaged. The project also attempted to promote the use of energy saving technologies by introducing skills in solar energy. Therefore PETT is in line with these MDG aspirations. According to the Somalia Joint Needs Assessment conducted in 20068 the economy throughout the country is developing and the reconstruction period will bring with it demands for skilled personnel. Employment in the public and private sectors will be limited by the economy’s capacity to sustain it. For this reason the education and training provided should prepare students for the labour market or for self employment by; advocating for support to a relevant TEVT curriculum in tandem with the needs of the labour market; establishment of vocational training centres for out of school youth and targeting women; expanding the enterprise-based training programmes which have proved successful in past. The PETT project was designed to meet this need. A study conducted in 20089 indicated that TVET needs to be expanded in order to produce skilled artisans and middle level technicians who will eventually strengthen the Somali economy, which can then increasingly rely on their own domestic work force. Consequently, the MOE needs support in order to expand their capacities, resulting in a clear need to strengthen the TVET framework through the establishment of independent TVET departments at the MOE. The origin of the project, its preparation, its objectives, its activities, and structures were therefore geared to sustainably improve the livelihood and employability opportunities of the target group through provision of technical and vocational education and training (TVET) and by availing information on employment opportunities, and thus reducing their vulnerability. The consultants found that the project design was appropriate and has correctly addressed the key issues facing the target groups in Somaliland and Puntland. The project design has described the objectives, definition of target groups, and the expected outputs of each training activity, as well as the use of inputs and the assumptions and risks though adjustments should be made in the Log Frame to have concise measurable effect indicators in some of the activities within results 2, 3 and 4. The evaluation study found that the project was relevant both at the time of design and at the time of this evaluation to the needs of the beneficiaries in the target area. Relevance has been rated highly satisfactory.

8 Somalia Joint Needs Assessment, Draft Sept. 2006, Social Services and Protection of Vulnerable Groups Cluster Report 9 Identification Study for the Education Sector Development Support Programme for Somalia, Final Report, June 2008.


2.2 EFFICIENCY Efficiency underscores how economically resources (materials, personnel, finances) and inputs put at the disposal of the project have been used in terms of quantity, quality and timeliness. Under this criterion, the consultants evaluated the extent to which the project had used resources in a productive manner to achieve its results under financial efficiency (i.e. the relationship between financial costs and the achieved outputs), implementation efficiency (referring to the approach that was applied in the delivery of training) and management efficiency (i.e. the way in which trainings were organized). Financial Efficiency The project total budget of EURO 2,633,876 was found to be adequate, but funds were slightly under spent under budget line 6 on others which consist of the works, which could be attributed to the fact that HTI (dealt with elsewhere in this report) did not conduct any training in the first year of inception. Table 1 gives the specific percentage budget utilization at the end of the project (EOP) as compared to the original approved budget. Table 2: Comparison between the original (approved) and revised budget (EOP) BL Expenses Original Budget

% Revised Budget % Variance %

1. Human Resources 25 25.8 0.8 2. Travel 1.5 1.2 -0.3 3. Equipment and Supplies 7.6 8.3 0.7 4. Local Office 8.4 8.4 0 5. Other costs 3.8 4.7 0.9 6. Others 47.1 45 -2.1 7. Admin. 6.5 6.5 0 8. Total 100 100 0 The project has trained more than 4,500 technical and vocational skills using institute and enterprise-based training approach as well as more that 3,000 in NFE (literacy and numeracy. This translates to EURO 351 (approx. $526) per beneficiary. The evaluation took note of the fact that UNESCO and UNDP paid a higher amount ($1000 per beneficiary) in demobilization projects which involved training of ex-militia in literacy and vocational skills using an enterprise-based approach in Mogadishu10. The average direct cost per trainee per day was $1.5. According to the tracer studies the TVET graduate earns more than $100 a month or at least $3 a day upon employment. Therefore the $1.5 cost per trainee per day which later translates to more than $3 per person per day upon employment was in the view of the consultants acceptable and cost-efficient.

10 UNESCO PEER Programme Assessment, Internal Oversight Service, Evaluation Section, February 2004.


However the cost per trainee should vary with the investment needed for a particular skill and the length of the course. Indeed, the feasibility study conducted in 2004 indicated that the cost of a typing course is less than a computer course. As well a confectionary course takes a shorter time than a carpentry course. There is a need therefore to review the blanket cost and duration of the various courses to improve cost-efficiency. It has been a cost-effective approach to train the beneficiaries in already established institutions and enterprises (the EBTVET centres did not require any investment), adopt ‘trainer out-sourcing’ for courses without presence of expertise at particular institutions as well as purchase project materials from the local suppliers. Overall the project is considered cost effective because of the long-term benefit of the training and its relevance to the beneficiary communities. Although the rising inflation in Somaliland/Puntland has had negative impacts on the project, the financial efficiency was satisfactory. Implementation Efficiency PETT adopted a participatory approach whereby it provided major external support for investments (mainly infrastructure, training and materials) against a commitment from the government to be the local counterparts and the training centres to be the implementing partners to ensure they would cover future project activities (amortization of investments, operation and maintenance of infrastructure, training, salaries, etc). This was to ensure ownership of the project activities. Initially, the trainee selection process was flawed and it was feared that it might not necessarily adhere to the proposed selection criteria, especially for EBTVET, according to the 1st local labour market survey11 12 conducted in September 2006. It therefore recommended the reconstitution of the selection panels to exclude interested parties to avoid issues of conflict of interest or nepotism arising. It further recommended that the panels must include people with knowledge in certain skills and the implementing institutions must be represented in these panels. The evaluation team notes that this recommendation was undertaken. This is evidenced by the fact that the identification and selection of the 3rd batch of the target group was conducted by a panel consisting of MOE/TVET, IBTVET, EPS project officers, community elders and youth representative with due gender consideration. The youth and community representatives were chosen depending on the location where the selection took place while MOE and the project staff remained consistent. This ensured fairness in the selection exercise and maintained transparency.

11 Local Labour Market Survey, Somaliland, Sept 2006, Practical Action Consulting – Eastern Africa 12 Local Labour Market Survey, Puntland, Sept 2006, Practical Action Consulting – Eastern Africa


The admission guidelines were also reviewed in order to improve targeting of beneficiaries and increase the community participation in the selection process. The selection process involved an oral interview of the personal background, age, educational history, proof of vulnerability, etc. Priority was given to disabled persons and women. The exercise took an average of 3 days, due to large number of applicants. The fact that the selection criteria of the target beneficiaries of the project were revised to include female representation is a clear indicator of the project’s endeavour to ensure equal representation of gender in decision making process13. The local implementing partners (LIPs) assumed a leading role in implementing the project and have performed the role relatively well. They have demonstrated the capacity to promote fairly good partnership and handle funds under this project. However weaknesses were revealed in their organisational capacity to monitor and co-ordinate activities. In this respect, measures are needed to strengthen the institutional, administrative and monitoring resources on the local level. The monitoring and supervision process by the consortium staff was regular. During discussions it was revealed that they visited the training institutions at least once a month to measure their progress of the project. The measuring indicator was based on trainee attendance, training methodology and approach used and at the same time observing the progress with regard to continuous assessment tests. Discussion with the project staff demonstrates the competency in utilizing their skills to monitor progress of the training. The project has carried out, among other activities, local labour market surveys, aimed at advising the project on the capacity of the training centres in imparting relevant and marketable skills in the Somalia context. On employment promotion the project has carried out a Local Economic Development (LED) study focusing on the identification of opportunities that could be utilized in employment promotion through the provision of skills training in TVET centres and review critical gaps existing in sub sectors that deal with TVET. The study also identified principle challenges that stakeholders are likely to face in addressing those gaps and provided recommendations on how to address them. The project has also conducted numerous studies and produced reports some of which are cited in this report in various sections. These include tracer studies, tools and materials assessments, capacity of stakeholders’ assessments, examination guidelines, training needs assessment, etc. All these studies were aimed at improving implementation quality based on emerging issues. The project, in collaboration with all the key players, developed a comprehensive TVET Policy Document for the Ministry of Education which was considered as a milestone in the TVET sector since there was none before. An independent examination board was also established to administer centralized TVET examinations.

13 The European Commission; A Roadmap for equality between women and men, March 2006


The PETT programme has an employment promotion services (EPS) component. The intention is for the EPS officers to capture and use all kinds of information that are available about opportunities and prospects for the target groups, in order to create interest among employers in the ‘products’ of the institutions and advise the trainees. In short, the EPS Centres act as “bridge” or link between the TVET centres and potential employers. Subsequently, the project established an EPS unit in the MOL to assist in linkages between the trainees and potential employers. On the other hand the officers give guidance to the trainees on career choices, CV writing, etc. Management Efficiency The project supported the training institutes from its bases in Hargeisa, Somaliland and Garowe, Puntland. This was planned to ensure ample logistical and technical assistance in the implementation of the project. The project was headed by a Project Manager and supported by IBTVET, EBTVET and NFE project officers. It is important to note that CARE and DS has a wealth of experience in TVET project implementation. The project also supported a TVET unit in MOE and an EPS unit in MOL, though the involvement of the latter came late in the project timeframe. All the partners signed partnership agreements detailing their specific roles and responsibilities. They held monthly steering committee meetings to review project progress, lessons learnt, challenges and the following month’s activities as well as share experiences. Coordination meetings with international agencies in similar intervention e.g. Caritas, UNHCR and UNESCO, etc, where peer review of project activities were undertaken, were held both in Nairobi and at field level. The constant support and advice from SCD and the consortium partners and the donors’ maintained interest played a crucial role in attempting to build the capacity of the relevant ministry’s staff and in encouraging the implementing team to ensure the overall success of the project. The project design was sound, particularly with regard to the organizational support and commitment from the local counterparts. The relationship between the SCD, the consortium partners and EU is good. It was established that the organizations collaborated quite well with regard to financial management and implementation support. The partnership arrangement in the implementation of the project was found to be appropriate and the partners have been very sensitive to changes in the environment in which the project operates. The project staff held weekly meetings to coordinate activities. As well every month the project officers reported their monthly progress reports in a similar format to the summary technical reports. This reporting assisted the officers in compiling the summary and the interim technical reporting at the end of the quarters. The arrangement of financial reporting was also well planned and coordinated. The consortium partners reported the expenditure to SCD which in turn reported to EC in half-year intervals. There was quality implementation and management of the project and the inputs and activities were carried out on time, though enrolment of the first batch in Hargeisa


Technical institute could not be achieved due to delays of the MOE in acquiring the centre which had been leased to Integrated Adult education Centre (IAEC), a local NGO. The implementers have adhered to work-plans as per the logical framework and the budget. The project has also achieved its original objectives against outputs and activities as detailed in the project document - notably the success of completing the project within the proposed timeline, albeit a no-cost extension of 3 months. Overall the efficiency of the project was considered satisfactory.

2.3 EFFECTIVENESS

The effectiveness criterion concerns whether the project’s results achieved the project specific objective i.e. what difference the project made in practice, as measured by how far the intended beneficiaries really benefited from the services it made available. The important outcomes of the project relate to the evaluation findings. The findings herein have been evaluated from the perception of the direct beneficiaries as well as from views of in-direct stakeholders e.g. employers, government, etc in the Somalia context. The findings herein are derived from careful discussions and a well-planned formative and summative evaluation of each of the major planned results of the project, which is also convincingly, a valuable outcome of the project. This survey’s finding is related to the aspects of the four towns of Somaliland namely, Hargeisa, Borame, Berbera, Burco (including Sheikh), as well as Bosasso, Qardho, Garowe and Galkayo in Puntland, revealing situations and highlighting benchmarks and integration approach of the interventions in the project while also highlighting local organization’s involvement in the implementation of the activities. Outlined below is the analysis of the extent to which the results were realised and their contribution to the specific objective. The analysis was based on the indicators under each result in the Log Frame (Annex VI).


2.3.1 Result 1: Access of disadvantaged youth and women to TVET and Employment Promotion Services (EPS) improved. The PETT project had listed the following activities under result 1: 1. Equal enrolment (male/female) of targeted number of youth and women. 2. Five Institute Based Technical, Vocational, Education and Training (IBTVET)

centers strengthened and made operational. 3. Presence of diversified and standardized skill areas with due gender

consideration 4. At least 4,525 disadvantaged youth and women got relevant/demand driven skills

training in IBTVET centres (2,525 trainees) and Enterprise Based Technical, Vocational Education and Training (EBTVET) centers in (2,000 trainees).

5. EPS available in four locations within six months of project start up 6. 5,000 beneficiaries have utilised EPS by the end of the project period 7. 3,000 youth and women have acquired literacy and numeracy relevant to TVET

1. Equal enrolment (male/female) of targeted number of youth and

women. PETT was essentially based on a gender equity approach, which aimed at reducing gender discrimination and promoting social/economic self-reliance of women through policies and programmes that increased access to basic education and training of women. The project design was such that there would be equal enrolment of vulnerable men and women within the age bracket of 16-24 years, who have had no or limited access to education, training and employment opportunities. Gender distribution and age was therefore a factor in the selection criteria. The study established that the national gender representation in TVET training was 52 % male and 48 % female (Fig. 1 & 2). Institute-Based Technical Vocational Education Training (IBTVET) In the first year, a total of 445 males (representing 67%) and 217 females (33%) were enrolled into the IBTVET Component of the project. The female enrolment rate was low due to a number of factors, key among them being: fewer number of women friendly skills, the socio-cultural barrier and poor perception among the community on the importance of the skill training. In the second year, 943 trainees; 559 males & 384 females (41%) were trained. The gender representation being 59% and 41% male and female respectively while in the third year 943 trainees enrolled out of which 542 (57%) were male and 401 were female (43%).


Enterprise- Based Technical Vocational Education Training (EBTVET) In the first year, 701 trainees underwent training in the EBTVET component of the project. The gender representation was 382 males (translating to 54% and 319 females (46%). In the second Year, the project trained 828 (416 males and 412 females) with the gender representation of 50% of each gender. Training in the third year comprised of 310 male and 211 female i.e. 59% and 41% respectively. Non Formal Education (NFE) on Literacy and Numeracy) In the first year, 1,109 trainees underwent training in the NFE component of the project. The gender representation was 471 males (42%) and 638 females (58%). In the second Year, the project trained 1,070 with the gender representation of 41% (439 male) and 59% (631 female). Training in the third year comprised of 209 male and 569 female i.e. 27% and 73% respectively.

Gender Enrolment

01020304050607080

M F M F M F

Batch 1 Batch 2 Batch 3

Year of Study

Per

cent

age

IBTVETEBTVETNFE

Fig. 1: Gender enrolment in Somaliland during the 3-year project duration


05

101520253035404550

BOSASSO QARDO GAROWE GALKAYO

Town

Freq

uenc

y

MaleFemale

Fig. 2: Gender enrolment in the four towns of Puntland

Fig. 3: % of TVET (both IBTVET and EBTVET) trainees desegregated into gender (overall) The evaluation also established that 74% the respondents were aged between 16-24 years while 26% were aged over 25 years (Fig. 4) in Somaliland. Although one of the project’s selection criteria was that trainee enrolment should be aged 16-24 years, the entry age was subsequently raised to 30-35 years especially for EBTVET female trainees who have been made more vulnerable by combination of several factors such as: high divorce rate and polygamy, taking care of those affected by HIV/AIDS, etc.


Age of Respondent

0

20

40

60

80

100

HARGEISA BURAO BORAME BERBERA

Town

Freq

uenc

y 0-15 yrs16-24 yrsOver 25 yrs

Fig 4: Age of respondents interviewed during this study in the four Somaliland towns The study also revealed that 64% of the respondents in Somaliland were enrolled on the basis of limited education, 18% returnee, 5% marginalised women, 2% ex-militia and 12% for other reasons (Fig. 5). This is consistent with the findings of the second tracer study which showed 62%, 17%, 0%, 2% and 19% respectively in the range of selection criteria. In Puntland, 39% had limited education, 27% were returnees from the neighbouring countries of Ethiopia, Djibouti and Kenya and 10% marginalised women, 13% ex-militia while 11% were selected for other reasons (Fig. 6). The number of ex-militia enrolled in Puntland was quite significant as compared to Somaliland. However in both regions the highest percentage was selected on the basis of limited education.

Selection criteria

0 10 20 30 40

Returnee

Marginalised

Limited education

Ex-militia

Other

Frequency

BURAOBORAMEBERBERAHARGEISA

Fig. 5: Selection criteria of beneficiaries in Somaliland


Fig. 6: Selection status of beneficiaries in Puntland disaggregated into gender The gender balance is consistent with the projections in the project document that required that at least a third of either sex should be able to access the vocational training except for the third year for NFE when the number of male reduced to 24% in Somaliland. Therefore, PETT should be given credit in being a gender-sensitized programme. As well the project addressed the age selection criteria with subsequent alteration as a result of other emerging issues of vulnerability. It is notable that the PETT project paid special attention to the critical issue of identifying and supporting opportunities for employment of women, many of whom were illiterate. This included the provision of numeracy/literacy training, appropriate identification and provision of market-driven and women-friendly skills as well as facilitating women to take employment opportunities in male-dominated technical/vocational occupations. 2. Five Institute Based Technical, Vocational, Education and Training

(IBTVET) centers strengthened and made operational. The study team sampled 3 IBTVET institutions in Somaliland, namely GAVO in Berbera and Hargeisa Technical Institute (HTI) and ANDP and 2 in Bosasso and Galkayo in Puntland to assess the quality of infrastructure, type of training equipment and education materials as well as the suitability of the management and training personnel. During the visit, it was observed that the quality of infrastructure was generally good though lack of enough tools and space was evident in some of the institutions.

Selection criteria

0 5 10 15 20 25 30 35

Returnee

Marginalised

Limited education

Ex-militia

Other

Frequency

FemaleMale


Initially before the collapse of

the government,

state owned institutes e.g.

Hargeisa Technical

Institute were convenient

venues for training and

with experienced

trainers who used to

conduct training during the Siad Barre Regime. Due to the protracted war, the assets were looted and the

infrastructure destroyed. The project played instrumental role in rehabilitation of the infrastructure and provision of tools as well as training of the trainers/instructors through workshops and on job training. The IBTVET centres have organisational structures composed of a Board of Governors composed of women and men headed by a Chairperson. An Executive Director runs the day to day activities e.g. ANDP Board of Directors constitutes 7 members, 3 of which are women. The organization has a written vision and mission statement, strategic plan and core values. The quality of practical training in IBTVET Centres was found to be low since the institutes tend to concentrate more on theory. This was attributed to lack of enough training tools and materials for technical skills as well as lack of qualified trainers. PETT has put efforts to address these issues through provision of equipment, training of the managers and on-job mentoring of the trainers. Discussions with the trainees indicated dissatisfaction with their trainers because of the lack of sound pedagogical skills. However, the availability of qualified and experienced vocational trainers in Somalia is a critical issue and more so for newly introduced skills. The project therefore sub-contracted external trainers on a need basis.


The evaluation noted that the project made renovations of existing physical structures in the IBTVET centres following a tools and equipment assessment14. Basic tools, equipment and furniture were procured and provided to the centres. As well, safety materials such as goggles, overalls, fire extinguishers, first-aid kits were supplied. The MOE in both regions assigned trainers to each of the three centres whose selection was based on the following criteria: the level of education; skills previously taught; years of experience and the gender. However getting female trainers was difficult. Training of trainers (TOT) has been conducted thrice within the project timeframe. The project staffs also make regular visits to the institutions and provide on job mentoring to the trainers/instructors. 3. Presence of diversified and standardized skill areas with due gender

consideration A total of 31 different skills were offered within the duration of the project. Initially, the TVET project, with the exception of garment making and agriculture, was training male-dominated trades such as woodwork, metalwork and building construction. The training in the latter areas never had any women participating. The PETT project developed new areas, such as computer and secretarial skills, where women were able to benefit and their participation encouraged. Based on the 1st local labour market survey report15, a total of 11 skills (out of which 9 were female-oriented) were introduced into the TVET centers. The female-friendly skills introduced included audio-visual, fish processing, bee-keeping, beauty therapy, basic accounting & secretarial courses, auxiliary nursing, confectionaries & bread-making, hotel management & front office operations as well as embroidery & cloth designing. Newly introduced male-dominated marketable skills include auto-motive body building (including spray painting & panel beating) as well as arts & design. Based on the advice of Local Economic Development (LED)16 information, the project developed six more skills namely, business development services, automotive mechanics, livestock rearing, fishing, basic telecommunication and cargo handling. Through matching fund from Band-Aid, the project further developed other six skills gender-friendly syllabi, namely, bread-making, kindergarten skills, beekeeping, brick making, beauty arts and arts &design. EBTVET offers a wider choice of training skills. There are 19 skills of which catering, garment making and handicrafts are predominantly female. In computer skills training there is almost a 40% mix of women to men. The study also shows that the demand and popularity of a particular course varies from one region to the other depending on 14 Tools and equipment assessment report on IBTVET centres in Hargeisa and Berbera, Aug. 2006 15 Local Labour Market Survey, Somaliland/Puntland, Sept 2006, Practical Action Consulting – Eastern Africa 16 Local Economic Development (LED) Study Report, Somaliland/Puntland, June 2006, Intermediate Technology Consultants – Eastern Africa


the market needs, with the exception of tailoring, which seemed to cut across all but one town as the most popular.

0

20

40

60

80

100

120

M F M F M F

BATCH 1 BATCH 2 BATCH 3

ElectricityCarpentryPlumbingComputer softwareSolar energyFishingBeauty TherapyAccountingElectronicsCooling SystemsBee-keepingTailoringHennaCloth designOffice managementAudio visual (women)Beauty artsCateringMasonryOffice management

Fig. 8: The various IBTVET skills offered by the PETT project

Fig. 9: Gender enrolment of the skills offered during the PETT project


The graphical representation of skill/gender in Fig. 8 indicates that PETT has gone a long way in diversifying skills training to the target group. The local labour market surveys helped identify courses that were either irrelevant or have produced too many trainees thereby saturating the market. Plumbing, masonry, solar energy & carpentry are most popular with men while beauty therapy was popular with women. Computer courses were popular for both genders. The most marketable skill at self employment category was embroidery & cloth designing while computer skills attracted most of the graduates on formal employment. It was established that it was easier to change the skills in EBTVET since there is little or no capital requirement for equipment. Changing skills for IBTVET is much more difficult since there has to be an initial investment of equipment and a search for qualified trainers. Thus, for IBTVET change of skill was limited to skills with little capital requirement and having trainers available. The 2nd tracer study17 attempted to analyze the usefulness of the skills training as perceived by the graduates. 81% said the skill training was very useful in their lives, while 15% said the skill has some usefulness to them and only 4% felt that the skill was not very useful to them. The main reasons given by those who felt the skill was useful to them were their ability to get employed as a result of the skill training. Some respondents felt that they had gained self esteem as a result of the skill training while others, for example those in food processing felt that their diet and food preparation at home had also changed as a result of the skills they acquired. 4. At least 2,300 disadvantaged youth and women got relevant/demand driven

skills training in IBTVET centres (1,300 trainees) and Enterprise Based Technical, Vocational Education and Training (EBTVET) centers in (1,000 trainees).

In the IBTVET component of the project, 2,548 trainees (1,002 (39%) female and 1,546 (61%) male) underwent a 7.5 months training in the 5 IBTVET centres. A total of 2,050 trainees (942 (50%) female and 1,108 (50%) male) graduated in the EBTVET component of the project (Table 3). This is 102% achievement of this result.

Table 3: TVET trainees in the 3 year duration

Year 1 Year 2 Year 3

IBTVET EBTVET IBTVET EBTVET IBTVET EBTVET M F M F M F M F M F M F

No. of trainees

445 217 382 319 559 384 416 412 542 401 310 211

% 67 33 54 46 59 41 50 50 57 43 59 41 Total 662 701 943 828 943 521 Grand Total 4,598

17 The 2nd Tracer Study Report, Nov 2008, Astrads Management Services Ltd.


The training in EBTVET is overwhelmingly practical, mostly due to a lack of facilities for theoretical classes while in IBTVET, the opposite applies due to lack of enough tools. While the practical orientation is basically good in view of the projects objectives, it could be improved if supplemented with standard theoretical courses. It is important to note that compared to IBTVET, high level of literacy may not be a prerequisite in EBTVET. Generally, the training helped the target groups increase their competency and thereby their employment prospects, both formal and self-employment. The evaluation concludes that the enterprises played a crucial role in the employment of the EBTVET graduates trainees. In most of the cases they were able to absorb not less than 60% of the graduates. However, according to the tracer study some of the employment opportunities were in fact voluntary positions with little or no allowance. But the graduates seemed to value this kind of arrangement as it made them acquire the much experience that was so much cherished by the employers. The project expected to achieve a 75% level of attainment of sustainable employment within six months for the graduates. The first tracer study18 was conducted in February 2007 and indicated an overall employment rate of 42.5% IBTVET and 54.4 % for the EBTVET. The second tracer study indicated the figures as 72.2% overall (57% male and 43% female) in Somaliland and 70% in Puntland. Table 4: Employment status of trainees in Puntland

Batch

Trained Traced Employed Employment type

IBTVET EBTVET IBTVET EBTVET IBTVET EBTVET Self Formal M F M F M F M F M F M F

Batch 1 271 141 210 156 107 70 108 87 56 36 55 58 103 89 Batch 2 277 131 153 163 162 96 138 169 122 77 114 132 280 178 Total 548 272 363 319 269 166 246 256 178 113 169 190 383 267

Source: 2nd Tracer study report, Puntland, Nov 2008 Table 5: Employment status of trainees in Somaliland

Batch

Trained Traced Employed Employment type

IBTVET EBTVET IBTVET EBTVET IBTVET EBTVET Self Formal M F M F M F M F M F M F

Batch 1 174 76 172 163 127 102 83 96 78 88 43 51 221 167 Batch 2 282 253 263 249 202 264 133 108 116 84 151 187 233 177 Total 456 329 435 412 329 366 216 204 194 172 194 238 454 344

Source: 2nd Tracer study report, Somaliland, Nov 2008 Of great significance is the fact that all the interviewed persons confirmed that they are using the skills obtained in their current employment and that their standard of living 18 Somaliland Tracer Study, March 2007, Busilink Development Services


had significantly improved as a result of their participation on the project. However those unemployed reckoned that they required a higher level of skill in order to compete effectively in the market. This was mainly amongst graduates who had done courses on electronics and computer software. Most of the graduates used their relatives/friends as a strategy in securing employment and even the unemployed ones still had faith in the same approach. There was significant distortion of equal opportunities in the job market brought about by the various elements of discrimination; significantly as it pertained to clan bias and on women. However, reasons cited by the unemployed female graduates were that they were not assisted by families and friends to start business because of their gender, employers simply do not employ them because they are women and that employers prefer male graduates. It is evident that gender discrimination is still a hindering factor to employment of women. Lack of practical experience was also a major barrier to access employment for IBTVET graduates as compared to EBTVET. It is therefore recommended that a strong linkage is established between the IBTVET trainees and labour market through apprenticeship programme. This approach would require the cooperation of the employers in agreeing to the arrangement. Other challenges encountered by the employed graduates include: lack of capital for expansion; high rental cost of business premises; lack of market for products and lack of adequate tools and materials.

5. EPS available in four locations within six months of project start up Another objective of PETT was the availability of employment promotion services (EPS) in four locations within six months of project start up and utilized by 5,000 beneficiaries by the end of the project period to give career guidance, business planning, etc to the trainees. 4 EPS centres were established in Hargeisa, Berbera, Garowe and Bosasso. However, this seems to have been over-ambitious to have fully operational EPS units within six months of project. The evaluation indicated that the units established were originally too weak to be effectively operational within six months. The intention was for the EPS officers to capture and use all kinds of information that are available about opportunities and prospects for the target groups, in order to create interest among employers in the trainee services’ capacity of their institutions and advise the trainees towards acquiring gainful employment. This study therefore recommends the establishment of EPS Unit under the Ministry of Labour in order to ensure its sustainability.


6. 5,000 beneficiaries have utilised EPS by the end of the project period A total of 3,436 trainees (Table 4) of which 1,377 females (representing 40.2%) benefited from the EPS activities comprising of trainings on business planning, resume writing, career counselling and interview techniques. Table 6: No. of Trainees who have utilised EPS

No. of trainees Batch 1 Batch 2 Batch 3 M F M F M F

112 67 292 271 1655 1039 Total 179 563 2,694

Grand Total 3,436 (69%) It is evident from the above table that initially the capacities of EPS were very weak, which led to the dismal performance of achievement. This was attributed to poor networking system between the EPS Officers and the relevant stakeholders mainly the municipalities, chamber of commerce and even the trainees. Based on challenges encountered and lessons learned, there was a need for restructuring. Subsequently, the EPS officers participated in a 3-day technical review meeting that was called by the lead agency. The evaluation notes that measures were taken to strengthen the EPS through the formation of an EPS Working Group (EPSWG) which contributed to the success of achievement during the third batch of the trainees. Achievement of the EPS was gauged by the numbers that found employment since the objective of the vocational education was to facilitate employability. According to PETT targets, it was expected that 75% of those trained would be employed within 6 months. In 2007 a high percentage of traced EBTVET male graduates found jobs, but a much lower proportion (31%) of male graduates from IBTVET. Female prospects were particularly poor e.g. only 13% of IBTVET graduates found jobs partly because of gender discrimination and limited marketable skills. 7. 3,000 youth & women have acquired literacy & numeracy relevant to TVET

The project saw the training of 2,957 (1,838 (62%) women and 1,119 (38%) men learners in the twelve NFE literacy and numeracy centres in Somaliland and Puntland (Table 7). This translates to about 99% of the target figure of 3,000.


Table 7: No. of NFE trainees within the project duration No. of trainees Batch 1 Batch 2 Batch 3

M F M F M F

471 638 439 631 209 569

% 42% 58% 41% 59% 27% 73%

Total 1,109 1,070 778

Grand Total 2,957

The number of NFE graduates who joined the TVET component of the project is unknown due to lack of clear established link between the TVET and the NFE section of the training. Consequently, the MOE in collaboration with the consortium members are closely working together to make sure that links are established between the TVET training and NFE centers so that NFE graduates can join the TVET training immediately. The project adopted UNICEF’s adult literacy syllabi as the standard NFE syllabi though the syllabus prepares trainees for formal education as opposed to skills training. This was due to lack of appropriate syllabi tailored for adult learners joining skills training, which was a challenge to the project.

2.3.2 Result 2: Competency in terms of technical and entrepreneurship of target groups increased in accordance with employment promotion

Under this result, The PETT project had listed the following activities:

1. At least 80% of TVET trainers/instructors trained and hence posses needed

skills 2. Trainers/instructors are able to transmit required knowledge, skills and attitudes

to trainees according to established norms 3. Standardized and relevant training packages/ curricular materials used by TVET

centres 4. 90% of trainees passed exam or acquired basic skills 1. At least 80% of TVET trainers/instructors trained and hence posses

needed skills:

The project inducted and trained a total of 106 TVET trainers on how to implement the curriculum materials; lesson planning; management of training; modalities and frequency of conducting continuous assessment tests; safety standards, proper documentation & recordkeeping. This was followed up by regular mentoring by the project staff. While the trainers still have limited capacity, the impact of the training of the instructors is evident from the documentary evidence at the TVET Centres.


2. Trainers/instructors are able to transmit required knowledge, skills and

attitudes to trainees according to established norms A training needs assessment (TNA)19 for IBTVET and EBTVET trainers was conducted within the first year of the project. The trainers underwent a training of trainers (TOT) on pertinent issues such as lesson-planning, safety standards, modes of continuous assessments testing, implementation of the curricular materials, proper documentations, and record keeping of trainees’ social profile, attendance sheets and evaluation records. December 2007 saw the induction and training of more trainers on how to implement the curriculum materials; lesson planning; management of training; modalities and frequency of conducting continuous assessment tests; safety standards, proper documentation and recordkeeping. The regular monitoring carried out by the project staffs and the MOE officials showed that majority of the trainers were implementing trainee-centred, practical-oriented trainings at the TVET centres. However, it was revealed that there was a challenge in getting good quality trainers since most of the trainers have below average English language comprehension, which made it difficult for them to interpret the syllabi utilised by the project. It was also challenging to get proper trainers for the skill areas developed by the project. A training of trainers on skills mastering is required to be able to ensure that all trainers are knowledgeable with all the aspects of the skills they are training. Translation of all training materials into the Somalia language is recommended. TVET trainers were further provided with on-job training by project officers during regular monitoring and follow up in developing daily lesson plans that were learner-centred, as part of ensuring quality training. The project also developed 12 generic lesson plans in order to enhance the training quality and ensure curriculum content coverage. 3. Standardized and relevant training packages/ curricular materials used

by TVET centres At the inception of the project, the TVET centres were using syllabi/curricula developed by UNESCO PEER. These included garment making, masonry, carpentry and joinery, metal fabrication, electrical installation, entrepreneurship, agriculture, plumbing, hospitality services, computer studies, secretarial studies and accounting. Based on the advice of Local Economic Development (LED) information, the project developed six more skills namely, business development services, automotive mechanics, livestock rearing, fishing, basic telecommunication and cargo handling. These syllabi were utilised alongside the syllabi developed by UNESCO.

19 Report on the Training Needs Assessment for PETT Project Skills Trainers, Somaliland, April 2006


Through matching fund from Band-Aid Foundation, the project further developed other six skills gender-friendly syllabi, namely for bread-making, kindergarten skills, beekeeping, brick making, beauty arts and arts &design. The PETT Project has reviewed and translated the 5 skills training packages that had been developed by UNESCO, these being; Garment Making, Electrical Installation, Masonry, Carpentry and Joinery and Metal Fabrication The project developed two examination guidelines. These were; “The Examination Guidelines for the TVET Trainer” 20 aimed at assisting the trainers in preparing internal examinations and the “Examination Guideline for TVET Institutions and Ministry of Education” 21 to iron out the interactions and requirements between training institutions and the Ministry of Education in both regions. 4. 90% of trainees passed exam or acquired basic skills During the first year, examinations were not standardized. The trainers teaching the same skill got together and prepared one exam which was administered in the training centres, with the invigilation of the secondary school teachers assigned by the Regional Education Offices. Results indicated that 93.8% of the trainees passed with over 50% score. In IBTVET, 90.4% of the trainees passed, while 97.2% in EBTVET passed their exams. Subsequently, the established National Examination Board within MOE took over the responsibility for testing the trainees and their qualification accredited by the MOE. The examination centre administered centralized examinations of which the percentage pass was above 90% for all the batches with an average of 4% dropout rate. Table 8 illustrates the percentage pass of the 2nd and 3rd year trainees according to the project interim reports. Table 8: Percentage of trainees who passed exams Initial enrolment Sat for exam Passed exam Dropped out M F M F M F M F Batch 2 IBTVET 282 253 272 244 268 237 10 9 Total 535 516 505 19 % pass = 97.8 EBTVET 263 249 235 248 232 237 28 1 Total 512 483 469 29 % pass = 97.1 Batch 3 IBTVET 275 260 271 254 269 251 4 6 Total 535 525 520 10 % pass = 99 EBTVET 94 111 89 99 82 93 5 12 Total 205 188 175 17 % pass = 93

20 Examination Development Guidelines for the TVET Trainer, Ministry of Education, Republic of Somaliland, and Puntland State of Somalia, May 2007 21 Examination Development Guidelines for TVET Institutions and MOE, Ministry of Education, Somaliland, and Puntland, May 2007


2.3.3 Result 3: Availability and use of information on Local Economic Development (LED) opportunities improved

1. 90% of trainees passed exam or acquired activities are informed by and adopted to

LED opportunities. 2. Networking and linkage system/platform established between TVET centres and

relevant stakeholders including between trainees and potential employers.

1. 90% of trainees passed exam or acquired activities are informed by and adopted to LED opportunities

The PETT project put emphasis on job placement services and availability of information on employment and local economic development opportunities in conjunction with TVET activities. The review of existing LED materials to identify gaps and synthesise issues related to TVET and EPS was carried out in 200622. Based on the advice of the LED study reports, new skills/trades that were considered as high potential in terms of stimulating economic growth were introduced. These skills included: bee-keeping, telecommunication as well as art and design. The 1st Local Labour Market Survey conducted within the first year of the project indicated areas and skills that were marketable and others not. The report was significant in the changes of skills and re-designing of other skills, for the 2nd year of training. Number of trainees for the skills like carpentry were reduced, solar training was amalgamated with electrical installation, while the intake for electronic and beauty therapy course were increased. The aim was to improve the marketability of the skills. The 2nd Local Labour Market Survey23 was, among others, aimed at identifying the trainee absorption capacity of 26 enterprise sectors in the short-term. The findings of the report indicated that some skills such as cooking & tailoring were saturated and hence had to be phased out. The report also identified highly marketable skills such as audio-visual & photo processing, auxiliary nursing, electronics & mobile phone repairs as highly marketable. This information was essential in the allocating the number of trainees for the 3rd year of project implementation, thus improving their marketability. 2. Networking and linkage system/platform established between TVET

centres and relevant stakeholders including between trainees and potential employers

The project saw the formation of an EPS Working Group (EPSWG) to strengthen networking system between the EPS Officers, the municipalities, chamber of commerce, 22 Local Economic Development (LED) Study Report, Somaliland, June 2006, Intermediate Technology Consultants – Eastern Africa 23 2nd Local Labour Market Survey, Somaliland/Puntland, Sept 2007, Astrads Management Services


the trainees and the Ministry of Labour, though this came late in the project timeline. Its mandate was to give guidance on careers, business planning and facilitate interaction with renowned business persons to give lectures to trainees and enhance employment promotion services. The employment promotion services held stakeholders’ workshops and reiterated the need for the project to focus on highly marketable skills which the employers could benefit from and veer from skills that were flooded. Eight stakeholders’ workshops were held with a cross-section of private and public sector players taking part. The workshops focused on the private sectors views on employment promotion. During the workshops, the participants accentuated the need to focus on market-driven short courses that would guarantee the trainees jobs once they completed their training. On the same note, the participants noted that little had been done to develop coherent policies and leadership at the Ministry of Labour as far as the labour market and youth employment were concerned. The participants suggested that institutional weaknesses within the ministries need to be addressed so that the EPS activities could be more effective. It was suggested that the Ministry of Labour in collaboration with other stakeholders need to develop and implement clear and comprehensive employment promotion policy. 2.3.4 Result 4: Capacity of local associate implementing partners and authorities enhanced with respect to TVET and EPS. Under result 4, the following was planned: 1. Networking and linkage system and financial systems for TVET centres

revised/developed and implemented 2. Cost recovery system developed and used by TVET centres 3. MOE staff and TVET managers and supervisors trained in management and

supervision skills 4. Quality and relevance of EPS services provided to beneficiaries 1. Networking and linkage system and financial systems for TVET centres

revised/developed and implemented The project improved the management and operational capacities of the IBTVET and EBTVET centres in collaboration with the Ministry of Education. This is evident from the fact that in October 2007, the Ministry of Education with the support of the project revised the guidelines for assessing the capacities and the experiences of potential EBTVET host-trainers/enterprises in conducting EBTVET training. The major areas of assessment for the partners included: past experience in implementing literacy/ skill training projects, partners capacity to deliver, networking and linkage systems availability of training facilities and financial management skills.


The capacity assessment of the TVET Centre led to the introduction of proper management and financial system including transparent cost-recovery approach. 2. Cost-recovery system developed and used by TVET centres A cost recovery system through the use of production systems was developed for the IBTVET institutions24 whereby the measurement of the effectiveness of the cost recovery system would be indicated in the annual income statements. Capacity training for IBTVET centres focused on institutional operation and cost recovery systems through the use of production systems. Since management of the training centres with regard to cost-recovery system and general centre management have not taken root firmly, the practical start-up of any production systems was not evident. In any case it was difficult to have such systems being operational within the 3 year duration. 3. MOE staff and TVET managers and supervisors trained in management

and supervision skills Following an organizational capacity assessment conducted in April 200625, the evaluation study established that MOE staff training had been conducted covering the following areas: development, implementation, monitoring and evaluation for TVET; Supervision and Quality Assurance; Training of Trainers/mentors; Education Management Information System (EMIS) and financial management. IBTVET Centre Managers and administrators were also trained in resource mobilization/fundraising; financial management, office administration & record keeping. The project thus enhanced the capacity of the MOEs and TVET by providing on-job training to personnel and assisting them to develop and implement management and financial systems (including cost recovery system), though the practical aspects take time to have a measurable impact. 4. Quality and relevance of EPS services provided to beneficiaries The project saw the formation of an EPS Working Group (EPSWG) to strengthen networking system between the EPS Officers, the municipalities, chamber of commerce, the trainees and the Ministry of Labour, though this came late in the project timeline. The main mandate of the EPS Working Group is to adopt strategies to enhance partnership with the private sector in developing TVET through various approaches and mechanisms including: identification of the roles of the private sector and TVET institutions in developing and training work forces to meet labour market requirements 24 Financial (Cost Recovery) & Management Systems Manual, Sept 2006, Acacia Consultants Ltd. 25 Report on the Organisational Capacity Assessment of the Ministry of Education, Somaliland, IBTVET Centres and Local Implementing Partners, April 2006, Acacia Consultants Ltd.


in terms of technological advancements; contribute to employability of the graduate trainees by identifying working mechanisms for partnership between TVET institutions, MOL and Local Authorities and the private sector through setting the bylaws, regulations, and instructions needed to ensure the effectiveness of this cooperation and partnership. The fortified links between the EPS Officers and the members of the EPSWG saw the employment of former trainees by various municipalities and enterprises. As well, the EPS officers provided career counselling, business planning, interview techniques, and entrepreneurship skills to the last batch of trainees. In order to further enhance the EPS activities, renowned/successful business persons were also invited to give lectures to trainees.


Assumptions and risks management Table 9: Management of assumptions and risks

Assumption/Risk Mitigation/Remarks 1. Presence of an enabling policy

environment for TVET. A TVET policy was developed and adopted by the MOEs

2. Support of municipalities and business communities in Employment Promotion Services.

The project put efforts for close contact with industry through formation of an EPS Working Group (EPSWG) to strengthen the networking systems.

3. Acceptance of the project by local communities.

The participatory approach ensured that the project was accepted by the community.

4. Absorption capacity of local economy improves.

The project conducted 2 local labour markets and LED surveys to ascertain the absorption capacity.

5. Security situation in the regions does not deteriorate.

The security situation in Somaliland has remained stable within the project timeline. The recruitment of capable Somali nationals in Punt land ensured project continuity during episodes of insecurity.

6. Target communities support female /women in taking up employment opportunities

Of the 72% trainees who attained employment, 43% were women, though a percentage of gender discrimination still exist

7. Employment opportunities do not decrease due to macro- economic shocks

The camel export ban due to the Rift Valley Fever caused an economic stress

8. Moderate fluctuation of dollar against local currency.

Although this did not affect the project per se, this had implications on procurement of materials.

9. Ownership of sites of IBTVET centres resolved.

Enrolment could not be done for the first batch at HTI. However MOE acquired the institution for the 2nd and 3rd batches.

10. Availability and low turnover of qualified trainer.

The Second Labour Market Survey revealed that it was difficult for institutions to retain good staff due to the fact that there is a substantial gap between the end of one batch and the start of the next (4-5 months), during which the instructors were not paid.

11. Collaboration of TVET centres and trainees with business communities and concerned government authorities.

An EPS Working Group (EPSWG) to strengthen networking systems between stakeholders was established.

12. Concerned stakeholders such as public and private partners are interested in participating in LED initiatives.

The project has put efforts in strengthening linkages between the private sector and the project stakeholders, though still weak.

13. Support and commitment of MOE, MOL and other government authorities.

The participatory approach ensured that the stakeholders remained committed.

14. Availability and low turnover of TVET management personnel.

The TVET management personnel have remained consistent within the project timeline.

The project’s effectiveness was rated highly satisfactory since the results were directly, or indirectly, contributing to the project purpose or have the potential to do so. However, refinement of the EPS activities would further enhance the project effectiveness.


2.4 PROJECT IMPACT

The general picture seems to be that graduates appreciate the training they received. Almost all the employed graduates indicated they are in employment or have started their own business applying skills they had acquired. All indicated that their livelihood has improved. The project has rehabilitated infrastructure in the IBTVET institutions; improved access of disadvantaged youth and women to TVET (4,598 beneficiaries of which 1,944 (42%) are female and 2,654 (58%) male and EPS (3,436 young men and women of which 1,377 (40%) are women and 2,059 (60%) men); enhanced the capacity of local implementing partners with respect to TVET and EPS, trained 1665 women/girls/young men on literacy and numeracy in NFE centres and has improved the capacity of the MOE staff. At least 72% and 78% of those trained are in sustainable employment in Somaliland and Punt land respectively. Rehabilitation of the infrastructure has guaranteed access to vocational education among the beneficiaries thus reducing their vulnerability and increasing their socio-economic well-being. The increased competency in technical and entrepreneurship skills will reduce poverty. The attention given to the women participation was a plus in ensuring gender equity. The ‘on-the-job training’ in EBTVET effectively transferred skills to the beneficiaries due to exposure to the practical aspect making them more competitive in the market and self-employment opportunities. This is further compounded by the fact that majority of the trainees had no formal education to grasp easily the many theoretical aspects in IBTVET centres. Of great significance to the project is the fact that all the interviewed persons confirmed that they are using the skills learned in their current employment and that their standard of living has significantly improved as a result of their participation on the project. From the tracer studies, it is clear that most of the respondents recorded improved education as one of the benefits that they have been able to experience from participating on this project. This is consistent with the fact that the study showed that most of the beneficiaries were chosen on the basis of limited education. It is clearly evident that there is a direct link between access to education/skill and financial stability as most of the respondents indicated that they were experiencing financial bliss from the participation in the project (see Fig. 6).


Fig 10: One of the training graduates at her place of work This project has enhanced the capacity of the implementing staff, especially the local implementing partners. The project ideology, insight and skills obtained by especially the local personnel stemming from and being part of the local communities is bound to have a long term impact on themselves as development workers as well as on the attitude and capacity of the local community to plan and manage development activities on their own. This approach, where the implementing consortium partners and the local partners planned together to improve the livelihoods of the wider community contributed to the success of the project and to the promotion of a more peaceful, equitable and democratic society in the target area. In summary the project has achieved benefits for the target beneficiaries, which will have a wider overall effect on a larger number of people in the different categories, their regions and in the country as a whole. These achievements contributed directly to achieving the overall objective and the impact was therefore found to be highly satisfactory.

Case Study 1 A tale of a successful tailoring PETT graduate In the main market place of Maroodijex in Central Hargeisa, Miss Hodo Ahmed Hassan is sitting with her sewing machine. She is 22 years old and learned the trade in 6 months at ASWA Enterprise-Based Training Centre in 2006. Prior to joining the PETT programme, she had been attending private evening lessons on literacy and numeracy where she learned of the project from her friends. She started her business in 2007 after she was selected by CARE, one of the PETT project consortium partners, as one of the beneficiaries of tools from Band Aid Trust. Miss Hodo is the second born in a family of 10 living in Hargeisa. Her father lost his job due to paralysis, her mother has no job and none of her siblings is working. Today she gets SoSh. 30,000 (1US$ = SoSh. 6,000) per day from sewing clothes for people and selling clothing material. This translates to more than US$ 100 per month. She is now the backbone of the family income. When asked why she opted for the skill, she said that it was out of personal interest. She was full of praise for the PETT project describing it as very beneficial and has helped a lot of people to have their own income. Her ambition is to have a big shopping mall dealing with clothing. Miss. Hodo at her tailoring/clothing shop with her trainer Ms. Nimo


Case Study 2 Maslah Ahmed: An ex-militia with healing hands Age: 24 years

Skill: Auxiliary Nursing

Fig. 11: Case Study 2 Maslah Ahmed Alim was in grade 3 at Omar Samatar Elementary School when heavily armed militiamen loyal to the former powerful warlord, Gen Mohamed Farah Aideed captured Galkayo town in 1991. That was the last time Maslah saw the inside of a formal classroom. It was a horrifying experience for young Maslah, who was then only 10 years old. His father, who was their only breadwinner, was shot dead in his shop in the town centre. The shop was equally looted and thus reduced to state of pauperism.“We lived on handouts from good Samaritans” Said Maslah. At the age of 13 and with no formal education, Maslah was recruited into the local militia.“I hated it but had no option; I was jobless and had no hope in life. I had nothing to lose then. The local warlord was ready to pay us $60 a month” Maslah told the project staffs. “It was at this point in time that I met, PETT project EPS Officer, who encouraged me to join the NFE component of PETT project. At first it was difficult, but the trainers and the project staffs encouraged me and my friends not to give up”. Upon completion of the literacy and numeracy course, he was enrolled into the enterprise-based vocational training as auxiliary nurse at Galkayo Hospital.“The nurses and the doctor at the hospital were very helpful. They gave me a lot of support during the training”. Maslah, who was clad in white lab coat, told our staff. Upon completion of the training, Maslah was employed by the same hospital. He now earns 300 dollars a month. Apart from supporting his elderly mother, Maslah uses his earning to pays fees for his 6 siblings (four sisters and two brothers). Maslah plans to do undertake diploma course in community nursing in order to help his people. “Most of the patients who come here suffer gunshot wounds. Others are mothers with malnourished children” Said Maslah. Apart from treating minor injuries, Maslah also trains mothers on child nutrition, importance of breast-feeding. “I thank PETT project for giving me the opportunity to undergo the training” Said Maslah.“Looking back, I can’t stop to wonder why men use clans, religion and political ideologies to maim, plunder, rape and kill.” Maslah told us.


2.5 SUSTAINABILITY PETT was designed to focus on the long-term capacity development for local partners to continue activities once the external aid ends. Sustainability has been increased by working more in collaboration with local structures. More focus on the development of local infrastructure contributed to an increased sense of local ownership. The project put efforts in reinforcing existing institutions which will contribute to institutional sustainability. The PETT project has put deliberate efforts to ensure the long-term sustainability of the activities and structures established. One such measure was the development of a TVET policy from scratch and the adoption of the same by the governments of Somaliland and Puntland although it is relatively easy to formulate a policy, but to have it become operational seems to be the bigger challenge. On a positive note, the Somaliland MOE has allocated an approximate sum of $14,286 into the current budget to support the TVET activities including payment of salaries to all the TVET instructors at Hargeisa Technical Institute. Another measure is to have the TVET centres raise money through income generating activities (IGAs). The project developed a cost recovery system through the use of production systems for the IBTVET centers. As for the sustainability of the IGAs, it is too early to see any direct results. It would be a good idea for PETT to monitor and report on the viability of the IGAs during the on-going STEO project. PETT has established an EPS unit within the MOHL with the mandate to provide linkages between potential employers and TVET trainees. The unit is still too young and undergoing mentoring by the SCD EPS officers. The sustainability of these units, which are mainly functioning on extra-budgetary funding, is uncertain and there seems to be a need for a sustainability strategy. Improved implementation and reporting by MOE staff can be attributed to the capacity building provided by SCD and its consortium partners. This translates to a greater consciousness on the need to expand TVET activities and implement projects on its own as well as transfer similar skills to other institutions e.g. Municipalities, community education committees (CECs) at the village level, thus increased and strengthened management capabilities for project sustainability. The involvement of MOE in the implementation of the project has created a sense of ownership. It was also established that the project was designed in such a way that it allows flow of benefits to the beneficiaries and the society at large, thence likelihood of this to continue even after the external aid ends. The project’s activities have been integrated into local institutional structures and policies. After analysis of the measures that have been put in place for sustainability, the conclusion is that it was satisfactory.


2.6 CROSS-CUTTING ISSUES Gender equality - The project aimed at achieving a 50% gender composition and to develop training syllabi that promotes women skills and employment. The PETT project was designed against the challenges faced by Somali women according to The EC Strategy for implementation of Special Aid to Somalia (2002-2007), “The Somali women have rather restricted rights, violated by the widespread practice of FGM; they also suffer from parent-induced restrictions to education and from limited availability of targeted health-care services, rendering them highly vulnerable to maternity risk”. In view of the above challenges, the PETT project has had an important and longstanding commitment to promoting gender equality as provided for in various legal and international instruments such as the EC Treaty, the Charter of Fundamental rights of the European Union (2000), the Beijing Platform for Action (1995), the Cairo Programme of Action (1994).

In endeavouring to reduce gender gap, the project identified priority areas of intervention as tabulated below: Priority

intervention areas of the project

Progress made

1. Equal economic independence for women and men;

The project qualified a total of 1944 females (representing 42.3%) in basic skill training. Out of which 64.8% are in (self-) employed. One of the strategies developed by the project in helping women starting business was to network with other agencies in providing start-up toolkits. In light of the above, UNHCR donated toolkits worth USD28, 000 to 494 graduate trainees (out of which 296 were females) as seed capital.

2. Reconciliation of private and professional life;

Considering the fact that many female trainees attend to household chores in the mornings thus making it difficult for the target beneficiaries to reconcile their training and private life, the project introduced flexible timetable for female trainees. This was a key factor that contributed to low dropout rate and high enrolment of the females into various skills

3. Equal representation in decision-making;

The selection criteria of the target beneficiaries of the project were revised to include female representatives.

4. Promotion of gender equality in external and development

Development of Female-Friendly marketable curriculums: Based on the local labour market survey report, a total of 11 skills (out of which 9 were female-oriented skills) were introduced into the TVET centers. The female-friendly skills


Priority intervention areas of the project

Progress made

policies. introduced include audio-visual, fish processing, beekeeping, beauty therapy, basic accounting &secretarial courses, auxiliary nursing, confectionaries & bread-making, hotel management & front office operations as well as embroidery & cloth designing. Minimal or deficient sanitation facilities (no gender-segregated latrines and no bath houses, particularly necessary during menstruation periods) in the TVET Centres was one of the key reason that contributed in a high and relevant dropout rate, and a very significant lower attendance rate (which eventually contributes to increasing the drop-out rate). It was against the above background that the project constructed female-only toilets (complete with sanitation facilities) in all the TVET Centres.

Human Rights Issue: The project trained 452 persons with physical disabilities as well as 393 others from minority groups thus by supporting equality in inclusion of the handicapped and the minority groups. In ensuring and maintaining gender equity equality, the project ensured that not only was women participation encouraged among the trainees, but it also encouraged the panels of TVET interviewers to be based on equal gender proportion. HIV/AIDS and FGM - During the training of the TVET centres instructors/trainers and NFE teachers on lesson planning, pertinent issues in HIV/AIDS, FGM, environment, Peace and Democracy was integrated as part of the training. Environment - GAVO, IBTVET centre offered solar installation as part of the training skills, although the skill was not popular and had to be combined with electrical installation skill during the second intake. The inclusion of these cross-cutting issues in TVET is a good approach of advocating for the same.


2.7 STRENGTHS AND WEAKNESSES Below is a summary of the strengths and weaknesses of the PETT project. Strength Weaknesses

1. Good project design. Lack of clear indicators in some of the results

objectives.

2. Good project planning, co-ordination and implementation.

Understaffed MOE and MOL limited adequate monitoring the training in the TVET centres

3.

Good relationships with the implementing partners, both external and local.

Weak EPS and job placement services

4. Cross-cutting issues have been taken care of in the activities.

Weak dissemination of LED information to beneficiaries

5.

Committed and well trained local staff whose capacity has increased in the course of the project

Relatively short training periods in comparison to syllabi coverage

6.

Capacity building efforts of the local partners and staff.

Lack of training periods flexibility in the towns located next to the sea due to harsh weather conditions

7. CARE and DS as a partner has a wealth of experience in TVET


3.0 CONSTRAINTS AND CHALLENGES Several challenges encountered by the project included: The Hargeisa Technical Institute did not offer any training in the first year as it

had been leased to IACE. Poor Financial accountability in the training centres due to low capacity within

MOE in monitoring and supervision. Procurement of materials was affected in the final stages of the project due to

inflation and currency fluctuations. Selection of new skills was problematic due to non-availability of relevant

curricula and experienced trainers. Availability of qualified and female trainers especially for newly introduced skills Training institutions offering capital intensive skills like metal fabrication and

automotive mechanics suffered from lack of appropriate equipment and tools to offer adequate practical training to trainees.

Most of the trainers have below average English comprehension, which made it difficult for them to interpret the syllabi utilised by the project.

Female trainees were unwilling to venture into male dominated skills. The trainees expressed fear of being accepted and cited the lack of employment opportunities when competing with men for the same skills.

4.0 LESSONS LEARNED The direct involvement of the local counterparts and stakeholders in project

implementation is vital to the success of a project because this provides a direct link of project performance to systems of local control and develops a sense of ownership, which greatly contributes to sustainability.

Private enterprises play a crucial role in absorbing the EBTVET graduates trainees

in their own enterprises. In most of the cases they were able to absorb not less than 60% of the graduates.

Involvement of local enterprises in the PETT project enhanced project’s impact

and cost-effectiveness. If well and carefully supported through financial and technical advice by international NGOs, the participation of local enterprises leads to improved project performance not only on outputs, but also on impacts.

Communities are willing to be partners in development and are willing to

contribute if well mobilized and sensitized. A community taking a role in the local development and management of facilities is empowering and cost-effective. For sustainability, the project should encourage community contributions in supporting TVET activities in their localities especially subsidizing on user fee.

The inclusion of cross cutting issues like HIV/AIDS, peace and democracy, FGM

and environment into the adult literacy syllabus is a good approach of advocating for these issues.


5.0 CONCLUSIONS AND RECOMMENDATIONS

5.1 CONCLUSION RELEVANCE The Consultant found that the project design was appropriate to the problems to be resolved at its design stage and its stated objectives correctly addressed the identified problems or real needs. The project was designed particularly to address widespread unemployment and poverty among disadvantaged groups as well as inadequate capacity for job placement and employment promotion services. PETT activities were directly aimed at reducing these constraints through quality non-formal education and vocational training. The projections for enhancing vocational skills and increasing employability, competence and local capacity to implement long term development projects was adequately designed. This evaluation found that PETT was a relevant programme. The target group lagged behind in education and learning achievements in the regions. The objectives of the project were judged to be substantially relevant throughout the implementation period. The project’s outputs generally scored uniformly well in terms of the relevance of their objectives. The project is also relevant in relation to the Education for All (EFA) Dakar Framework, to the Millennium Development Goals (MDGs) No. 1, 2 and 3, the EC’s strategy for the Implementation of Special Aid to Somalia and the Somaliland/Puntland MOE policies and strategic plans. Due to its relevance, the PETT project has been able to ‘connect’ with a similar project albeit with a new name – Skill Training for Employment Opportunities (STEO) - to reflect inclusion of lessons learned through monitoring and evaluation also implemented by SCD and its consortium partners and funded by the European Commission. The PETT project has also been able to attract extra support from other donors namely UNESCO, UNHCR (through provision of tools) and WFP (through FFT and provision of tools) and Band Aid Trust Foundation (through provision of start-up tools and curricular development activities).

EFFICIENCY The Consultant found that the project activities transformed the available resources fairly efficiently into the intended results in terms of quantity, quality and timeliness. Things were done right and the value for money for the activities implemented were quantified and seemed generally satisfactory in the Somalia context and in comparison to similar initiatives within Somalia. The IBTVET and EBTVET implementing partners assumed the role of project implementation and management. This meant that they received and managed the training funds for the project and co-ordinated activities at field level. In all these roles


they performed fairly well. Moreover, the ability of the institute managers to mobilise community support on the local level assisted in promoting the project to the intended beneficiaries. SCD and its partners support the training institutes from its bases in Hargeisa, Somaliland and Garowe, Puntland. This was planned to ensure that there is ample logistical and technical assistance to the local partners in the implementation of the project in the regions. The project also supports a NFE/TVET unit in MOE and an EPS unit in the Ministry of Labour (MOL) to develop their capacity to manage and activities in their dockets. The planning and co-ordination function was well undertaken by the implementing partners, SCD, CARE, DS and GI. The partners were adaptable and able to resolve problems, provide advice and support and helped to build the capacity of the training institutions, Government ministries through field visits from time to time. The donor, EC, also maintained a close interest in the project. Consistent interactions between all the partners to view project progress helped in achieving project results. EFFECTIVENESS Effectiveness was found to be highly satisfactory as all the results directly contributed to the project overall objective though refinement of the EPS component would further enhance effectiveness. This means that all activities that have been undertaken are leading to increased employment opportunities which will result to improved livelihoods of the beneficiaries. The prime indicator of having achieved this socio-economic objective is that graduates from training courses come into gainful employment, either as wage-workers or as self-employed. Effectiveness has been particularly strong for objectives related to the Enterprise Based Technical Vocational Education Training (EBTVET), physical infrastructure in the training centres and capacity building of the local stakeholders, but fairly effective in EPS and job placement services. Performance with regard to imparting practical knowledge in Institution Based Technical Vocational Education Training (IBTVET) centres is perceived as less effective by the beneficiaries. Qualitative information contained in other project progress reports document tends to confirm a common outcome. PROJECT IMPACT The PETT project achieved benefits for the target beneficiaries on purpose level had, or are likely to have in future, a wider overall effect on a larger numbers of people in the different categories, the region and in the country as a whole. The achievements at purpose level contributed therefore directly to achieving the project’s overall objective


stated as “to improve the livelihood of disadvantaged youth and women in the regions thus contributing to reduce their vulnerability”. The underlying factor of PETT was to address unemployment of the youth. The general picture seems to be that graduates appreciate the training they received. Almost all the employed graduates indicated they are in employment or have started their own businesses applying skills they had acquired. All indicated that their livelihood has improved. Rehabilitation of the infrastructure has guaranteed access to vocational education among the beneficiaries thus reducing their vulnerability and increasing their socio-economic well-being. Many disadvantaged young men and women have been and are likely to be trained in these institutions. The increased competency in technical and entrepreneurship skills results to improved opportunities for gainful employment. The participatory approach of the project has built capacities in TVET management and training as well as developed standardized curricula materials for new skills which have been accepted and adopted by the governments and distributed to training centres. This was cost-effective because the materials will be used by many more beneficiaries in the future. The project has proved its impact with the increased participation of women. It has enhanced women positioning in the socio-economic well-being as some have started their own businesses or have been absorbed in gainful employment as a result of the intervention. These enhanced capacities have promoted their self-reliance and the well-being which will contribute to national development especially to child protection. The attention given to the women participation was a plus in ensuring gender equity. The Director General, MOE in Somaliland, summarised the project as ‘positive and viable from A to Z’. In summary the project has achieved benefits for the target beneficiaries, which will have a wider overall effect on a larger number of people in the different categories, their regions and in the country as a whole. These achievements contributed directly to achieving the overall objective and the impact was therefore found to be highly satisfactory. SUSTAINABILITY The PETT project has put deliberate efforts to ensure the long-term sustainability of the activities and structures established. One such measure was that the governments have adopted the TVET policy although it is relatively easy to formulate a policy, but to have it become operational seems to be the bigger challenge. PETT was designed to focus on the establishment of long-term capacities for local counterparts and partners to continue activities in the delivery of service once the external aid ends. The project put more focus on the development of local


infrastructure and reinforcing existing institutions which contributed to an increased sense of local ownership. Institutional sustainability has been increased by working more in collaboration with local structures. This is evidenced by the adoption of the TVET policy by MOEs, the establishment of EPS units in MOHLs and the establishment of independent examination bodies. The building of capacities within the government and the TVET centres through training and on job mentoring will contribute to long-term sustainability. Considering financial sustainability, SCD has been working in an environment where the Government is fairly in a position to maintain or finance vocational educational support. Therefore, the likelihood of the Government and the TVET centres sustaining activities initiated by PETT beyond external support appears to be substantial from a financial point of view. The development of the financial (cost-recovery) and management systems manual will assist the TVET centres to start income generating activities. The great challenge will be to ensure the government and the TVET centres are moving at the same pace. The education and competency provided to the beneficiaries is life-long allowing the flow of benefits to their families and the society at large which increases the likelihood of continuity of these benefits even after the external aid ends. Overall, the project generally incorporated sustainability as a pre-requisite in its approaches and interventions. Capacity building in the Somali context is known to take considerable time. Hence for the local partners to become capable of continuing the full flow of benefits after the project ends may not be achieved in the short time, but rather on the long time, especially from the financial point of view though it appears substantial. The following table shows the overall performance of the project. Table 10: Overall project scoring Evaluation Parameter Scoring Relevance Highly satisfactory Efficiency Highly satisfactory Effectiveness Satisfactory Impact Highly satisfactory Sustainability Satisfactory


5.2 RECOMMENDATIONS From the evaluation findings, the following is recommended: SCD and its consortium partners should devote more efforts to improve the

managerial capacities of the local implementing partners by participating in hiring of trainers and instructors and closely monitoring and following up on the financial accountability. Similarly, prior to the enrollment, mobilization and awareness campaigns should ensure full participation of the training centres management in the planning, financing, execution and supervision, which has direct impact on project results.

There is need to incorporate an exit strategy in the project design for purposes of

continuity and financial sustainability. This is possible if the project can implement a workable cost-recovery system that facilitates the IBTVET institutions to make money and create a financial base for a reliable exit strategy e.g. engaging the trainees within the institutions to make products which can be sold to the public. A money-making venture would improve the ability of the institutions to manage themselves in a commercial manner and reducing necessity for external support. However it should be noted that for such ventures to be successful, appropriate accounting practices need to be put in place. Similarly, in order to realistically train the institutions on proper cost-recovery system, a practical model institution should be sought as practical example in the local context.

Training in EBTVET is predominantly practical. The centres should be supported with

tools and materials in return for taking in trainees as a kind of incentive. Theoretical instructions should also be strengthened in the centres to offer courses such as quality control and business skills. The evaluation also indicated that the IBTVET centres are still under-equipped for practical lessons and that the centres need to be upgraded in equipment and materials.

Environmental impact assessment (EIA) of TVET projects ought to be included in the

project design to take care of environmental issues (both physical and socio-economic) so as to avoid health and safety of the trainees and other environmental hazards as a result of waste materials e.g. paints, oils, lubricants, metal scrap, etc generated in project activities.

It is proposed (not recommended) that non-vocational activities aimed at achieving

the social objective be integrated in PETT to include the establishment of sports clubs, the organizing of competitions, aimed at occupying youths not interested in vocational skills training with other mind occupying positive activities i.e. the social objective as opposed to the economic objective.

SCD should consider the provision of resources for strengthening local institutions

for sustainability. TVET and EPS units in the Ministries of Education and Labour should be strengthened and possibly converted to full ministerial departments or even the formation of an autonomous authority on vocational training to be dubbed


as Vocational Education and Training Authority (VETA). Future projects need to assist the MOL to formulate and develop an employment policy and strategic plans, which will have an added value for sustainability. However, it is important to note that it is relatively easy to formulate a policy, but to have it become operational seems to be the bigger challenge. Capacity of MOE and MOL staff at all levels should also be strengthened in all aspects of project design, implementation monitoring and evaluation. An identification study for the education sector development support for Somalia conducted in 2008 indicated similar views.

Whereas influx of cheap goods e.g. electronics from other countries e.g. China

promote the employment opportunities for TVET graduates (due to repairs), importation of goods which can be made locally by the TVET graduates e.g. furniture is counter-productive as it depresses the local industry. The Ministry of Industry (chamber of commerce) therefore ought to be highly involved in future TVET projects to regulate the importation of such goods since the influence of foreign market have pros and cons to the local market.

Provision of contingencies should be done during project design to take care of

inflation within the project time frame e.g. at the start of PETT, the exchange rate was 1 Euro to 1.5 US dollar while at the end the exchange rate was 1 Euro to 1 US dollar which has implications on project activities e.g. procurement. Alternatively the agreements between SCD and the local implementing partners and counterparts of the project could be done basing on the currency used in the project proposal i.e. Euro.

Awareness of local economic development (LED), EPS services and labour market

survey information to the trainees should form part of the projects component right from inception. There should be a review of the approach used in the dissemination of such information to make it more effective, efficient and relevant to the intended beneficiaries.

Future projects need to address gender related challenges that face women

graduates regarding employment. These could include campaigns by the implementing partners on affirmative actions to ensure that women constitute a certain percent of the total employees in the work place; form women organizations that will fight for their employment; reduce the cultural stereotypes that hinder women from seeking employment; women engage in self employment and become independent economically.

There is need to establish clear linkages between the TVET and the NFE section of

the training so that NFE graduates can join the TVET training i.e. there should be clear articulate pathways of moving from NFE to TVET.

Future TVET projects should incorporate a short business management course to be

conducted after graduation, especially for IBTVET centres. Such a course would require inclusion of business proposal writing to enable the graduates seek such services competitively. Such training could also focus on group formation,


marketing, and improved production. Complimentary skills also need to be introduced for the trainees. Environmental education (relating to marine, water, land use, conservation, socio-economic, etc) should be integrated as part of the TVET curricula.

IBTVET training do not include attachments for practical work and experience and

graduates without practical attachments usually have difficulty in securing employment, partly because of the employers’ preference for some experience and the applicants’ lack of confidence. TVET projects should encourage the local implementing partners to liaise with potential employers on job placement/attachment with a view to gaining such experience. There is a need to strengthen linkages between vocational centres and private sector employers in skill development.

The LIPs should also seek partnership with micro-financing institutions to enable

trainees to kick-start their own business immediately after training. As has been revealed in this study, many of the graduates lack capital to start their own business while others lack tools and materials to expand their ventures. Without such linkage the trainees may find it difficult to get started and the objective of the training eroded. It should be noted that the PETT objective is to provide skills aimed at promoting employability of its beneficiaries and therefore not in a position to be financiers of the graduates. However provision of tools could have an added value towards self-employment.

A standardized skill grading system should be introduced such that learners

graduate on various categories of grading depending on proficiency levels e.g. Grade 1, 2 or 3 to spread out employment opportunities and reduce flooding of the market with only one competency level. Presently, the graduates are semi-skilled at Grade 3. The country also needs well trained youth to replace the old generation. A centre of excellence to generate trainees at various competency levels to also cater for the training of trainers is a necessity. The long-term goal of upgrading Hargeisa Technical Institute in Somaliland and Galkayo IBTVET Centre in Puntland into national polytechnics or technical universities that will train Somaliland’s technical human resource at certificate, diploma and degree level is strongly recommended. However this requires articulate planning and heavy investment.

It is necessary to review and expand curriculum to cover new skill areas, taking into

consideration the market trends based on regular market surveys and local economic development needs. All TVET institutions, public and private should be harmonised so as to offer standardized curriculum. As well a standardized assessment system to be used by all TVET institutions should be adopted.

User fee and training duration should be dependent on type of skill. The feasibility

study conducted in 2004 indicated a difference in charges for different skills e.g. computing course is more expensive than accounting or typing. In the same breath some skills require more time to learn than others. A study could be done to address this anomaly.


Although the main objective of PETT is to train beneficiaries for the sole purpose of

acquiring employment, the duration of courses of between 6 and 7.5 months regardless of skill area is rather short. Discussions with the beneficiaries and the trainers/instructors for both EBTVET and IBTVET indicated that the duration of the programmes is insufficient to cover the whole syllabus. Most of the trainers recommended a longer period of 8-12 months.

The Somali language is widely used in the training since few students (and some

trainers) have an adequate command of English to understand the curricula. The trainers therefore have to spend considerable time translating from English textbooks to Somali for the benefit of their students. Although some of the curricula materials have been translated into the Somali language, it is recommended that efforts be made to translate all curricula materials into the Somali language.


6.0 ANNEXES ANNEX I: TERMS OF REFERENCE

ANNEX II: THE EVALUATOR

ANNEX III: LIST OF DOCUMENTS CONSULTED

ANNEX III: EVALUATION TOOLS

ANNEX IV: LIST OF PERSONS INTERVIEWED/SAMPLE SIZE

ANNEX IV: LOGICAL FRAMEWORK MATRIX


ANNEX 1: TERMS OF REFERENCE (TOR) FOR TERMINAL EVALUATION OF THE PROMOTION OF EMPLOYMENT THROUGH TRAINING (PETT) PROJECT IN SOMALILAND & PUNTLAND

1. INTRODUCTION

1.1. EDUCATION SITUATION IN PUNTLAND/SOMALILAND Somalia was defined as a Least Developed Country (LDC) and ranks amongst the ten poorest countries in the world. In 2003 UNDP estimated26 that over 43% of the population live in extreme poverty on less than one US dollar a day while 62% of the urban labour force and 41% of the rural/nomadic labour force are reportedly unemployed. During the protracted war that erupted in 1991, majority of the population lost their economic assets, including homes, farms, lands and jobs, and went into exile or became internally displaced. After the war, many people, especially those in the relatively stable North West Somalia (Somaliland) and North East Somalia (Puntland) resettled in their hometowns and villages and are striving hard to rebuild their livelihood and economic bases. The dynamic and expanding private sector that is growing steadily is supplying the majority of jobs and income for the people and appears to be one of the fundamental elements for the country’s reconstruction. However, the sector is challenged by the low supply of qualified and skilled labour force. Most of the educated and skilled labour force has either fled the country or in their twilight years. The majority of the young population is uneducated and unskilled. There is tremendous increase in the number of unemployed youth in major towns and urban centres of Somaliland and Puntland in the last few years. Continued influx of returnees, increased rural-urban migration and worsening economic conditions as a result of the indefinite livestock export ban have contributed to worsening the chances and opportunities of the youth to obtain gainful employment opportunities. The immediate causes of the high rate of unemployment is also exacerbated by the existence of large number of disadvantaged people, lack of government economic structures, loss traditional livelihood systems and limited availability of people with employable skills and basic education. The feasibility study commissioned by EC in June 2004, pointed out that whilst local economies are growing at a rapid rate in many areas, these economies are starting from a very low base. Notably, resources are limited and there are no significant labour market opportunities. At the same time, there are only marginal prospects for local economic development in some areas. As both Puntland and Somaliland face an almost overwhelming demand for more and better jobs, there is a pressing need to provide job placement and employment promotion services. Such services are critical to the success of the project whose core objective is to promote employment through training.

26 Socio-economic Statistics, Somalia, Report No. 1. UNDP/WB Somalia Watching Brief (2003)


In addition to widespread unemployment and poverty, and the inadequate capacity for job placement and EPS, there are other constraints such as: (a) limited capacity and infrastructure to provide appropriate conditions and means for receiving and re-integrating large numbers of militia and returnees; (b) limited availability of job and livelihood opportunities, especially for women and the increasing numbers of school leavers; (c) over-reliance on traditional economic activities; and (d) limited capacity to recognize economic development opportunities and to start and expand business and livelihood activities. 1.2. PROMOTION OF EMPLOYMENT THROUGH TRAINING (PETT) PROJECT: In response to the needs of the target groups and the zones as mentioned above, PETT project was initiated by SCD (lead agency), CARE, DS, and GI and got financed by European Commission (EC). The project aims to sustainably improve the livelihood and employability opportunities of the disadvantaged youth and women thus contributing to reduce their vulnerability in ways that are consistent with the Millennium Development Goals, the Dakar Framework for Action (DFA) as well as the EC’s Strategy for the Implementation of Special Aid to Somalia (SISAS) through provision of technical and vocational education and training (TVET) and by availing information on employment opportunities, and thus reducing their vulnerability. The project primarily targets 4,525 disadvantaged youth, ex-militia, minority groups, out-of-school-youth (of which 2262 (representing 40% of the target group will be women and girls) within the age bracket of 16-24 years, who have had no/limited access to education, training, and employment opportunities. The activities being implemented in the main towns of Somaliland and Puntland, seeks to: (a) deliver quality training and opportunities through Institution Based Technical Vocational Education Training (IBTVET) to 2,525 people and Enterprise Based Technical Vocational Education Training (EBTVET) to at least 2,000 people; (b) provide employment promotion services to at least 5,000 people; (c) provide job placement services; (d) avail information on employment and local economic development opportunities; and, (e) build the capacity of local stakeholders. It was planned that 75% of those trained will be in sustainable employment by the end of the project period. The 3-year PETT project is implemented by a consortium of INGOs (SCD, DS, GI, and CARE) with SCD as the lead agency. The project is due to complete its operation by end of August 2008, though there could be extension for three more months to complete certain activities. The purpose of conducting the evaluation before completion period is basically to draw necessary lessons and critical issues that need to be addressed in the next phase of the project. 1.2.1. OBJECTIVES OF PETT: The overall objective of the project is to improve the livelihood of disadvantaged youth and women in both Somaliland and Puntland thus contributing to reduce their vulnerability. Consequently, the origin of the project, its preparation, its objectives, its activities, and structures are geared to sustainably improving the livelihood and employability opportunities of the target group through provision of technical and vocational education and training (TVET) and by availing information on employment opportunities, and thus reducing their vulnerability.


1.1.3. PURPOSE OF PETT:

The project purpose is to create sustainable employment opportunities for youth and other disadvantaged groups with equal gender composition in Somaliland and Puntland. The project therefore focuses on improving access for disadvantaged groups to TVET and EPS that will in turn enhance the technical and entrepreneurship competencies of target groups. The project also endeavours to improve the availability and use of information on Local Economic Development (LED) opportunities. Additionally, the project strives to enhance the capacity of local authorities and local implementing partners and build national institutional arrangements as well as realigning, reforming and strengthening the TVET centres for the running TVET activities and EPS.

1.1.4. PLANNED RESULTS OF PETT: The project has four key expected results: Result 1. Access of disadvantaged youth and women to TVET and Employment Promotion Services (EPS) improved Result 2. Competency in terms of technical and entrepreneurship skills of target groups increased in accordance with employment promotion Result 3. Availability and use of information on Local Economic Development (LED) opportunities improved Result 4. Capacity of local associate implementing partners and authorities enhanced with respect to TVET and EPS

1.1.5. BENEFICIARIES OF PETT: The project seeks to directly benefit 2525 and 2000 disadvantaged youth, internally displaced persons (IDPs), returnees, out-of-school youth, ex-militia, persons with disability, and minority groups within the age bracket of 16-24 years through access to quality IBTVET and EBTVET trainings respectively. It is also projected that at least 5,000 people will benefit from EPS. The indirect beneficiaries include: the MOE, family members of the trainees through improved income; enterprise owners/host-training institutions where the target groups are receiving the training, local authority and general public through spill over effect of the availability of skilled labour force.

1.6. IMPLEMENTING PARTNERS: The PETT project is implemented in Somaliland and Puntland by a consortium comprising SCD, CARE SSS, Diakonia Sweden, and Gutenberg Initiative under the leadership of SCD. These agencies work closely and in partnership with the Somaliland/Puntland Ministry of Education and six IBTVET Centres namely: – Hargeisa, GAVO, ANDP, Garowe, and Galkayo IBTVET Centres as well as a host of EBTVET Centres.


2. PURPOSE AND OBJECTIVES OF THE EVALUATION:

The evaluation is intended principally for learning and accountability purposes. It is expected to generate relevant findings, lessons, and recommendations which will be shared with key stakeholders of the project and used by the implementing agencies to guide and inform future programming of skill training and EPS in Puntland and Somaliland. The evaluation will also assess the performance of the project against key parameters including the project’s relevance, effectiveness, efficiency, sustainability, timelines of activity implementation, and its strengths and weaknesses. This information will also be shared with key stakeholders of the project, importantly the donor (EC), MOE, local education authorities, TVET centres and other actors in Puntland and Somaliland. 2.1. OBJECTIVES 3.1 To assess the performance of the project along the following

dimensions; Relevance (how appropriate has been the project design? To what extent did the

stated objectives correctly address the problems and real needs of the target groups?)

Efficiency (how economically were project inputs (funds, expertise, time, etc) converted into results in required quantity and quality and in good time? Was the use of the project resources cost-effective? Could they have been done more cost efficiently?

Effectiveness (to what extent have the stated project results and purpose been achieved in a sustainable way?)

Impacts (what sustainable changes – positive/negative, intended/un-intended – did the project produce on the target groups – (past and present)as well as MOE, TVET Centres and other stakeholders?).

Sustainability (what is the likelihood of the project to continue after end of the project? What sustainability measures have been put in place?).

Implementation processes (To what extent were the activities of the project implemented in participatory and empowering manner? How involved were the key stakeholders of the project in planning and execution of activities, and steering the project?)

2.1. 2 To identify the strengths and weaknesses of the project and come up with

findings, lessons and recommendations to guide and inform future programme work with special focus on the delivery of TVET and EPS.

2.1. 3 Examine and give recommendations on the future expansion of the TVET (with specific focus on the levels and skill areas) vis-à-vis the employment prospects/needs of the Somaliland/Puntland economy.


2. SCOPE OF THE EVALUATION The evaluation will cover the accomplishment of all the expected results as outlined in PETT project document and detailed in the annual work plans during the period of September 2005 – May 2008. All the different project components and activities stated in the original log frame will be assessed. In view of the fact that the project is ongoing, the evaluation will also cover the links between the different phases of the project such as: (a) current institutional and organizational set up at the MOE (b) establishment of network and linkage system/platform between TVET centers and relevant stakeholders including trainees and potential employers (c) the development and implementation of policy guideline; (c) presence of operational management, financial and cost-recovery systems at the TVET centers; and to answer to what extent these environments enables survival of the TVET and EPS activities after the project ends. 3.0. EVALUATION CRITERIA: The evaluation will concentrate on the five evaluation criteria elaborated below, namely; relevance, efficiency, effectiveness, impacts and sustainability of the project. 3.0.1 . Relevance: The main focus will be on the appropriateness of the project’s

concept and design to the overall situation of Somaliland and Puntland Education System. In particular: Extent to which the stated objectives correctly address the problems and real needs

of the target groups (past and present trainees, MOE, and TVET centres, entrepreneurs, host-trainers, instructors/trainers).

Relevance of project design within the framework of Ministries of Education programmes and policy guidelines.

3.0.2 . Efficiency: The main focus will be on how well the project activities transferred

the available resources into the intended results in terms of quantity, quality, and timeliness through sustainable and participatory processes. In particular:

Whether similar results could have been achieved more or by other means at lower cost in the same time.

Whether project activities were done right i.e. on time, in expected quantity and quality, and through participatory processes.

General implementation and management of the project in terms of quantity, quality, and timeliness of inputs and activities, adherence to work-plans, action-plans, logical framework, and budgets.

Adequacy of management arrangements as well as monitoring and evaluation, technical backstopping support given to the project by all parties concerned coordination and communication.

Institutional set-up through the Central Ministry, the EPS Office, and the TVET Centres, and the degree to which the project has encouraged full involvement of all the stakeholders;

Responsiveness of project management to changes in the environment in which the project operates;

Co-operation among project partners (SCD, CARE, GI, DS and the MOE in Somaliland and Puntland and other key stakeholders in achieving project results.


3.0.3 Effectiveness: The main focus will be on extent to which the project achieved its

stated results and purpose in a sustainable way. In particular; The progress made in achieving the results of the project at all levels. The project

Logical Framework will be used as the basis for analysis and assessment. Efforts made in capacity building of the MOE, TVET Centres and other local project

stakeholders and whether the strategies are working or not working and why. Project’s management of risks taking into view the stated risks and assumptions.

3.0.4 Impact: The main focus will be on whether the project’s overall objectives

have been achieved or are likely to be achieved, specific changes that the project has brought about in the lives of target groups (teachers, children, etc), and impacts realized or likely to be realized in relation to cross cutting issues such as gender, environment and child protection. In particular;

Achievements of the project against the original results (outcomes), outputs, and activities specified in the project log frame.

Level of awareness of the stakeholders, particularly the MOE, TVETs, trainees, trainers, and entrepreneurs regarding the project outcomes.

Level of ownership of the project by the education Authorities (MOE) and their commitment to support the project after the expiry of the project period.

Degree of support given by the consortium partners and the Ministry of Education in integrating the project objectives and goals into the National Education Policy, and how well the project fits into national goals of education

Extent to which the project has impacted on teaching and learning. Project impact on enhancing co-operation among education sector and project partners. 3.0.5 Sustainability: The main focus will be on whether the outcomes and changes

brought about by the project are likely to continue after the end of the project and whether they can be sustained at the sector level, and in each implementation region (Somaliland and Puntland). In particular, Extent to which the MOE, local education authorities and key education stakeholders

own the objectives and achievement of the project. Policy support available to the project from the MOE and governments of Somaliland

and Puntland Institutional capacity of the MOE, the TVET, and NFE Centres to continue the project

activities after the end of the project, including good will and financial and technical capacities.

Conduciveness of the socio-cultural factors to the continuity of the project activities Possibility of advancing and expanding the TVET training vis-à-vis the employment

prospects and the needs of the economy. 3.2 Evaluation sites: The evaluation will encompass visit to: Hargeisa, GAVO, and ANDP

IBTVET centres in Somaliland; Garowe, Bosasso, and Galkayo IBTVET Centres in Puntland; a number of EBTVET and NFE centres in all the major towns of Somaliland and Puntland as well as visit to offices/places of various stakeholders identified as sources of information.


3.3 SPECIFIC TASKS: Working closely with the Evaluation Team, the evaluators will undertake the following tasks:

Develop evaluation framework and methodology and refine these with the project team. Develop the evaluation plan; Undertake extensive document review to familiarize with the project; Develop the evaluation tools and conduct validation and field testing of the same; Assess the demand for TVET by the target groups with particular reference to women

enrolment; the demand for TVET graduate by the labour market, appropriateness of content of the TVET curricula developed and utility of the same by the TVET Centres; establishment and implementation of cost-recovery systems in TVET Centres, quality and relevance of EPS provided to the beneficiaries, and make recommendations for improvement; the extent in which the TVET and EPS activities are informed by and adopted to LED opportunities;

Assess the practicability for, and give recommendations on the future establishment and upgrading of one Technical Institute that will act as a “hub” in each zone (i.e. PL and SL) for training instructors at the vocational centres in the Zone;

Assess the capacity development interventions the project has undertaken with the MOE and TVET Centres;

Prepare the evaluation draft reports and present the findings in workshops in Hargeisa (Somaliland), and in Garowe (Puntland) for key national and international agencies operating in the zones; and relevant actors in Somali Support Secretariat (SSS) in Nairobi.

Prepare and submit final report to the Lead Agency in three hard copies (bounded) and CD-ROM.

4. EVALUATION APPROACH The evaluation will be undertaken through a combination of qualitative and quantitative techniques that will be used to gather data and information from main stakeholders of the project mainly target groups, MOEs and Labor authorities, actors in the private and public sectors including municipalities, chamber of commerce, TVET and IBTVET centers, enterprises, project officers, other actors in similar projects etc. The data and information gathered will be systematically analyzed and presented following the five evaluation criteria mentioned above. 4.1. The methodology for the study is envisaged to cover the following: Data Collection: I) Documents Review: Desk study review of all relevant PETT Project documentation,

including but not confined to: 1st and 2nd Local Labour market Survey

Reports; EC Strategy for Implementations of Special

Aid to Somalia (2000-2007); 1st Tracer study report; Joint Needs Assessment Report (draft Sept.

‘06); Local Economic Development Reports; MOE Education policies/Strategic Plans; TVET Policy Documents; Past audit reports, quarterly and interim

reports TVET curricula materials TVET Examination and Certification


Guidelines; PETT Project Documents including

proposal/budgets ; EC Feasibility study (2004);

PETT Project technical and financial reports;

Partnership agreements (MOUs);

II) Qualitative TechniquesFocus Group Discussions: Focus Group Discussions will be held with TVET graduates and those who are still undergoing training for each gender category separately.

a) Semi-Structured Interview (SSI): SSI will be carried out with selected individual male and female graduate and present TVET trainees, trainers, TVET managers, concerned government authorities, project staff, partners etc.

b) Observations: Careful and systematic observation regarding the operations of TVET, EPS and NFE Centres, and MOEs will be carried to capture primary information. The evaluation team will also observe TVET trainees and trainers conducting sessions in selected sample centres. Observation at workplaces (employment or self employment) of TVET graduates is also required.

c) Key Informant Interviews: This method is important to capture the views and professional opinions of people who know about the circumstances on the ground and PETT project in particular. Such persons should be identified and consulted from agencies like UNESCO, UNHCR, MOL, World Food Programme (WFP), and CARE, members of NFE Working Groups, employers from both the public and private sector.

d) Most significant change stories: It is suggested that the consultants should collect stories of change from identified beneficiaries of the project. The stories will focus on what the project beneficiaries consider to be the most significant change brought by the project in their lives, capacities, or way of operating. Stories should be collected especially from former trainees, MOE, TVET Centres and enterprises.

III) Quantitative data collection: The would-be consultant is also expected to use and

apply quantitative research techniques such as questionnaire technique (structured way of data collection).

4.2 Data Analysis: Data gathered using various methods and tools have to be collated, analyzed, and interpreted systematically. The evaluators will be expected to come up with detailed data/information analysis methods. It is also expected that the analysis and interpretation shall be made along with the key issues of the evaluation. It is suggested that the evaluators employ triangulated data analysis procedures. It would be possible to utilize social science data analysis software such as the Statistical package for the social sciences (SPPS) as appropriate and to supplement this with qualitative techniques of data analysis. This will help to triangulate and support the information/data that will be gathered by other methods. This is particularly important for gathering information from Trainees, instructors, training managers etc.


5. DELIVERABLES The consultants should deliver the following outputs and services: Detailed evaluation framework/design and implementation plan agreed with the Evaluation

team. Development and finalization of the evaluation tools. Orientation /training of the evaluation team. Separate draft reports for Puntland and Somaliland. Facilitate validation workshops in Hargeisa and Garowe on the basis of the draft reports. Produce final draft evaluation reports per Zone Present the findings and recommendations of the evaluation to selected actors of Somali

Support Secretariat (SSS) Finalise the reports of the zones and submit merged reports for the two to the lead agency.

The final report shall be submitted in three hard copies and CD-ROM to the Lead Agency in Nairobi.

6. EXPERTISE REQUIRED The following attributes are requirements for the selection of the evaluation team: 6.1 Lead Consultant Postgraduate qualification (Masters or above) in Education or in a discipline relevant to this

assignment with a minimum of 15 years experience. Experience in the evaluation of technical assistance project, preferably EC-funded projects,

particularly those under the education portfolio; Proven capacity in working across the levels of institutions from policy, to legislation and

organizations and ability to assess institutional capacity and incentives; Excellent report writing skills

6.2 Co-consultant Academic / professional background in TVET and EPS with extensive experience in

sustainable development – preferably in post-conflict environments with a minimum of 10 years of working experience;

Experience in implementation or evaluation of technical assistance projects; An understanding of TVET activities and broad-spectrum knowledge of operational

programmes in the Horn of Africa region; Adequate knowledge of the current political and socio economic situations of

Puntland/Somaliland and ability or possibility for the consultant to travel and undertake the evaluation particularly in Puntland without major problem;

Skills, experience and knowledge of Project cycle management. Excellent report writing and facilitation skills.

7. TIME FRAME The consultancy will run for 30 effective days. The dates/month for executing the evaluation will be jointly agreed between the consultants and SCD. The following are indicative number of days for executing the activities involved:-


S. No. Activity No. of

days Location Person responsible

1. Document review 2 Nairobi /Hargeisa) Lead consultant 2. Development & revision of

evaluation tools 2 Nairobi/Hargeisa Lead consultant

3. Training of data collectors on evaluation tools

2 Nairobi & field (Hargeisa/Garowe)

Both consultant

4. Pre-testing/revision of evaluation tools

1 Nairobi & field (Hargeisa/Garowe)

Both consultant

5. Fieldwork 11 Field (Somaliland/Puntland)

Both consultant

6. Data analysis & report writing (Separate for SL & PL)

5 Field Both consultant (3 days for consultant)

7. Validation & debriefing meeting

2 Hargeisa/Garowe Both consultant

8. Revision & submission of separate reports for PL & SL

1 Nairobi Lead consultant

9. Presentation of the report to SSS


10. Produce & submission of the zonal and consolidated evaluation reports


11. Revision & submission of the reports to the lead agency based on the feedback


12. Total 30 Lead consultant=30 days; Co-consultant=19days

8. SUPPORT AVAILABLE: On behalf of the implementing agencies, SCD will: Facilitate and coordinate the with consortium members to avail their project officers to be part of the evaluation team and to assist in the collection of the required data; pay the professional fee and other arising costs from the contract; provide logistical support (transport, accommodation, food, etc during the assignment), and review/comment on methods and other outputs of the evaluation.

9. SUPERVISION AND MANAGEMENT: The overall supervision of the evaluators will rest with SCD, the lead agency for the project. The consultants will report to the Deputy Country Director (or a designated officer) based in Nairobi. Whilst the consultants will be expected to work closely with, and interact a lot with PETT implementing agencies and the MOE, the overall management will be provided by the lead agency. As such, the both the Programme Manager and the PETT Project Manager based in Hargeisa will provide day-to-day supervision and support to the consultants.


10. TERMS AND CONDITIONS: Code of conduct: Because Save the Children's work is based on deeply held values and

principles, it is essential that our commitment to children's rights and humanitarian principles is supported and demonstrated by all members of staff. Save the Children's Code of Conduct sets out the standards which all staff members must adhere to. The consultant is bound by the principles and conditions of Save the Children’s Code of Conducts.

A contract will be signed by the evaluators upon commencement of the evaluation which will detail additional terms and conditions of service, aspects on inputs and deliverables.

11. APPLICATION REQUIREMENTS: All expressions of interest should include: i) Cover letter: A short (maximum three pages) letter addressing the evaluation criteria. ii) Technical Proposal (maximum eight pages) highlighting: brief explanation about the

Consultant/s with particular emphasis on previous experience in this kind of work; profile of the Consultant/s to be involved in undertaking the evaluation; understanding of the TOR, the task to be accomplished as well as draft evaluation framework and plan. Taking into consideration the current deteriorating security situation in Puntland, the would-be consultant is expected to come up with modalities of conducting the final evaluation of the project. It is suggested that either of the following approaches shall be given serious consideration in designing of the technical proposal and subsequent undertaking of the assignment:

Teaming up of consultants of which either the lead or co-consultant to be deployed in Puntland would be of Somali origin;

Identifying and engaging professional Somali resource person within Puntland as associate consultant who will carry out the evaluation with support of the project staffs based in Puntland.

iii) Financial Proposal: The financial proposal should provide cost estimates for services

rendered including daily consultancy fees excluding: accommodation and living costs; transport cost; stationeries, and supplies needed for data collection; costs related to persons that will take part from consortium partners and government authorities during evaluation process, enrichment workshop. The financial proposal should also provide cost estimates for services rendered including daily consultancy fees related to the consultant and/or associate consultant who will take part in the final evaluation of the project in Puntland. Data collection and data processing costs are on the account of the Consultant. The Consultant should also use his own computer.

Applications with non returnable copies of CVs, testimonials and with at least two sample of previous work should be submitted on or before May 30, 2008 and via email to [email protected] and [email protected]

Applications for Lead consultant and co-consultant can be applied separately or as a team or by a firm.




12. ANNEXES:

Annex A: The Presentation of the Evaluation Report The format of the evaluation report shall be as below:

1. Title page with title and nature of evaluation 2. List of acronyms and abbreviations 3. Table of contents, including list of annexes 4. Executive Summary (Should not be more than five pages) 5. Main Text

5.1. Introduction 5.2. Evaluation Objectives 5.3. Methods of Evaluation 5.4. Evaluation Findings (based on the five evaluation criteria)

6. Conclusion and Recommendations 6.1. Relevance 6.2. Efficiency 6.3. Effectiveness 6.4. Impact 6.5. Sustainability

7 List of annexes :

The report should generally include the following annexes: 7.1 The Terms of Reference of the evaluation 7.2 The names & companies of the evaluators(summarized CVs, limited to 1 page

per person should be shown) 7.3 Methodology applied for the study (phases, methods of data collection, sampling

etc) 7.4 Logical Framework matrices (original and improved/updated) 7.5 List of persons/organizations consulted 7.6 Literature and documentation consulted 7.7 Other technical annexes (e.g. statistical analyses, etc)


ANNEX II EVALUATORS 1. Lead Consultant This evaluation has been conducted by Morris W. Mathenge as the Lead Consultant. He is a community development specialist with over 18 years post qualification experience in undertaking resources needs assessment, project impact assessments, project evaluations as well as environmental impact assessments.

The Consultant holds a Bsc degree and an Msc degree from the University of Nairobi, a post graduated advanced certificate from the University of Tubingen (Germany) and an Environmental Impact Assessment and Audit certificate from Jomo Kenyatta University of Agriculture and Technology.

The Consultant has worked with the Ministry of Water & Irrigation as well as the Ministry of Energy (Kenya); University of Nairobi and Jomo Kenyatta University of Agriculture and Technology as a lecturer; the Adventist Development and Relief Agency (ADRA) and Project Coordinator both in Kosovo and Somalia for humanitarian assistance; the Church Ecumenical Action in Sudan (CEAS) in Kurmuk, Sudan and Cooperazione Internationale (COOPI) as a Project Manager in Somalia as well as for Save the Children Denmark in Somaliland and Puntland. The Consultant works under the Company name of Hydro Tech Consult.

The Consultant has over 15 years professional experience in many projects related to project needs assessment, implementation and evaluation. He has over ten years NGO practical experience in the East Africa, horn of Africa and Eastern Europe in implementation of rural and urban community based projects including community mobilisation, participatory training methods and implementation approaches as well as co-ordination and networking with partners and has worked in different environments with people of varied cultural diversity.

Mr. Mathenge has a wide knowledge of Somalia both in the North and Central and Southern regions having conducted several assignments there. In particular, he has conducted a similar project for ADRA Somalia which involved an end of project evaluation for a water project with a component of vocational training in Mudug Somalia. 2. Co-consultant The Co-consultant was Abdi Osman Abdullahi, a graduate of Kenyatta University in Kenya with an upper class degree in education and has deep knowledge of the Somali culture. He is a curriculum development specialist and has worked with various organizations in Somaliland on issues related to the education development of the child.

He has worked as a community development advisor in the SOS Family Strengthening Program in Hargeisa in facilitating program development, providing overall coordination and strategic directions, monitoring and evaluation including capacity building.


He was also the head of social studies for SOS and assisted the Somali counterparts in developing and implementing curricula as well as guiding departmental activities. He has also been a consultant of the terminal evaluation of community based activity project to advance the rights of the child/youth for SCD in Somaliland which involved the assessment of the achievements of the project against the project objectives and analyzing lessons learned for best future practices. He is well versed with community organization and mobilization, sensitization on child rights, youth mobilization, HIV-AIDS awareness, etc.


ANNEX III LIST OF DOCUMENTS CONSULTED 1. PETT project proposal document (including the log frame)

2. 1st and 2nd Local Labour Market Survey Reports

3. 1st and 2nd Tracer Study Reports

4. Local Economic Development Reports

5. EC Strategy for Implementations of Special Aid to Somalia (2000-2007)

6. Joint Needs Assessment Report (draft Sept ’06)

7. Somaliland/Puntland MOE Education Policies/ Education Sector Strategic Plan 2007-

2011

8. Somaliland/Puntland TVET Policy Document –August 2007

9. TVET Curricula materials

10. PETT Project Technical and Financial Reports

11. Past Audit Reports, Quarterly and Interim Reports

12. PETT Project Capacity Assessment Reports

13. TVET Examination and Certification Guidelines

14. EC Support to Employment Promotion in Somalia Feasibility Study (2004)

15. Partnership Agreements

16. EC Evaluation Procedures and Guidelines structures (2001)

17. Identification Study for the Education Sector Development Support Programme for

Somalia (2008).

18. UNESCO PEER Programme Assessment (2004)


ANNEX IV:

TERMINAL EVALUATION OF THE PETT PROJECT FOR SAVE THE CHILDREN, DENMARK (SCD)

Informed consent & cover page

Hello. My name is _______________________. I am engaged by Safe the Children Alliance and I am/we are here to get information about the Promotion of Employment through Training (PETT) project funded by the European Union (EC). You/Your institution/business/government ministry have been selected to be surveyed. I would like to ask you some questions related to the project.

The information you provide will be treated with confidentiality. It will be useful in establishing the impact of PETT, and will be beneficial for planning future development programs within the community in this area. The information will be compiled in an evaluation report. *Participation in the survey is voluntary, and you can choose not to take part. If you have any questions about the survey, you can ask me. At this time do you have any questions about the survey? Thank you for availing your time. Signature of Enumerator: Date:

Respondent number.

Date of interview. Day: Month: Year: Time interview commenced. Time interview ended.

General information: Name of Region and Town…………….…………………………………………… Name and Type of Ministry/Institution/NGO/Business/etc…………………………………… Status of respondent: No Partner Name of respondent Designation Any comments 1 Ministry of Education 2 Ministry of Labour 3 Partner Organisation 4 Opinion Leader 5 Training Institution

(IB/EB/NFE)

6 Employer 7 Parent/Guardian 8 Employed Beneficiary 9 Un-employed Beneficiary 10 Other (specify)


QUESTIONNAIRE FOR PROJECT MANAGEMENT (SCD & PARTNERS)

1. What is your perception about design, implementation and management of the project in terms of: Quality Timeliness of inputs and activities Adherence to work-plans Logical framework Budget and cost-effectiveness

2. What is your perception on achievements of the project against: Original objectives Outputs and activities as detailed in the project document, including logical

framework.

3. What is your opinion on appropriateness and relevance of the project’s design and implementation strategies to the identified needs? Was the design process participatory?

4. What is your perception about (i) adequacy of management arrangements as well as (ii)

monitoring and (iii) support given to the project by all parties as established in the project proposal and other unforeseen support?

5. What is your perception about appropriateness of the partnership arrangement in the

implementation of the project? Has it been effective? 6. What can you say about responsiveness of project management to changes in the

environment in which the project operates? 7. Are the original objectives currently relevant to target group’s aspirations/needs? 8. How were the risks managed taking into view the stated risks and assumptions as stated in

the project document? 9. Do you think the project objectives have addressed the beneficiaries’ problems taking into

account the prevailing political and social economic situation? 10. In your opinion, what can you say about the project being relevant? Effective? Efficient? 11. What can you say about the selection critera for all target towns and beneficiaries? 12. Do you have identified lessons learned to be used in future projects? Yes/no, if yes which

ones?


TECHNICAL/VOCATIONAL TRAINING INSTITUTIONS 1) Training institutions in Somaliland/Puntland?

a) How many? And in which towns? b) How many beneficiaries have completed vocation skills training, in both regions?

No Name of Institution

Region Town Graduated beneficiaries

Year (batch)

Gender (M/F)

Employed Un-employed

(Use separate sheet) 2) What are the various skills offered in both the IBTVET and EBTVET?

IBTVET -……………………………………………………………………………………… EBTVET -…………………………………………………………………………………….

3) What is your perception about appropriateness of the vocational skills training provided? 4) What is your perception about the level of skills of the vocational trainers at the established

centers measured against the vocational skills taught in the center? 5) What is your perception about the relevance of the vocational skills training to the local

economic and social environment? EMPLOYMENT PROMOTION SERVICES 1. List the services you offer to the trainees 2. Do you think the services have been beneficial to the trainees? If yes how? 3. What challenges do the trainees face when looking for employment? 4. Have you managed to deal with the challenges? If yes, how? 5. What can you say about the level of commitment and support from the Government and

local authorities in promoting employment? 6. In your opinion do you think the promotion services you offer have helped the trainees?


IMPACT AND SUSTAINABILITY 1. Do you think the flow of benefits to the beneficiaries and society as a whole is likely to

continue after the external aid ends? If yes how? 2. As a stakeholder what can you say about participation and ownership in the project

interventions? 3. What measures have you taken to ensure continuity of the services after the project ends?

Do you have an exit strategy? 4. What can you say about the level of commitment and support from the Government and

local authorities and to what extent the project’s activities been integrated into local institutional structures

5. What is your opinion concerning the level of commitment from the government about

continuity of the TVET activities


GROUP DISCUSSIONS GUIDE QUESTIONS FOR THE EMPLOYED BENEFICIARIES

Date…………………………………Time started………………Time ended………….

Enumerator………………………………………………..

Region………………………… Town: ……………………….

NAME OF RESPONDENT

TYPE OF COURSE (also indicate whether IB or EB)

TYPE OF EMPLOYMENT

BATCH NO

SEX AGE

(Use separate sheets)

1. What were you doing before you enrolled in the PETT training?

2. Give reason for having chosen the course that you have taken

3. Comment on the duration of the course

4. How did you get your job?

5. In what ways have the course helped you to find employment?

6. Do you think that the skill makes it easier to perform your work?

7. In what ways were the Employment promotion Services useful in looking for a job and/or effectively performing your duties?

8. What challenges do you encounter while looking for employment/ performing your

duties?

9. Is there any course/subject that you think would have been more useful to you in acquiring/retaining or getting promotion on your job?

10. In what ways do you consider that your life has changed because of participating in the

PETT project?


QUESTIONAIRE FOR EMPLOYED BENEFICIARY

Name of respondent………………………………………….M/F………Age………...

Vocational/Technical Institution………….. …………………………………Batch No…..

Employer…………………………………… Designation……………………..

Enumerator………………………………… Date…………

Time started……………..Time ended……………….

Region………………………… Town: ……………………….

1. What criteria made you to be accepted into the PETT project?

a. Returnee from another area b. Marginalised girl/woman c. Little or no education d. Ex-militia e. Other (please specify)

2. How did you know about the interventions of the PETT project? Through:

a) NGO official …b) Neighbour…c) Relative…d) Community leader…. e) Other (specify)……….

3. What skills did you learn under the PETT program? …………………………..

4. Did you find the training useful?

a) Very useful………… b) Useful…………… c) Not useful…………… 5. Was the time allocated for this training adequate?

b) Short………………….b) Adequate…………c) Long…………………… 6. What was your level of education before enrolling in PETT training?

a) None…. b) Primary…… c) Secondary….. d) Post-secondary………… e) NFE…………………

7. How long after graduation did you stay to get a job?

a. Immediate b. 1-3 months c. 3-6 months d. Six-12 months e. Over one year

8. Have you ever been employed elsewhere since you acquired the skills and then left? a. Yes b. No

9. If yes, why did you leave?


a. I resigned because of………………………….. b. I was sacked because of………………………. c. The contract expired d. The job got finished

10. Please list the various ways that you used to look for employment

a. Have asked my relatives to look for a job for me b. Have walked from potential employer to another c. Have sent my CV/applications to various potential employers d. Have approached my friends e. Have done absolutely nothing f. Other ways (please specify)…………………….

11. If given another chance, would you still go for the same skills that you acquired at the PETT project?

a. Yes b. No

12. If yes please give reasons:

a. It is still exciting for me b. I believe it still has the best chance for employment c. My friends are also in it d. It is not difficult to learn e. Other reasons (please specify)…………………

13. If no, please give reasons:

a. It is not in demand b. It requires a more advanced learning c. Other reasons (please specify)……………………………….

14. What challenges did you encounter while looking for a job?

15. In what ways could some of the above challenges have been addressed by the PETT project?

16. What is the impact of the training in your socio-economic life?

a. High b. Moderate

c. Low

17. What other skill would you rather have combined with the current one to make you

more effective?


GROUP DISCUSSIONS GUIDE QUESTIONS FOR UNEMPLOYED BENEFICIARIES

Date of interview……………………… Time started………………Time ended………….

Enumerator ……………………………………………..

Region………………………… Town: ……………………….

NAME OF RESPONDENTS

TYPE OF COURSE (also indicate whether its IB or EB)

BATCH NO. SEX AGE

(use separate sheets)

1. What were you doing before you enrolled for the PETT?

2. Give reason for having chosen the course that you have taken

3. Do you consider those reasons to be still relevant now and why?

4. What do you think are the main reasons why you are not currently employed?

5. What ways have you used so far to look for employment?

6. Have you ever considered self employment? If no why; if yes, why have you not started already?

7. Are there other persons that you know of that have gotten employed as a result of

participating on the PETT training. What do they say made them succeed?

8. Do you think there is discrimination in getting employed? If yes, explain.

9. Do you consider the PETT skills and EPS to have been uselful?


QUESTIONAIRE FOR UNEMPLOYED BENEFICIARY

Name of respondent………………………………………….M/F………Age………

Vocational/Technical Institution………….. …………………………Batch No………

Enumerator………………………………………… Date………………………..

Time started………………Time ended………………..

Region………………………… Town: ……………………….

1. What criteria made you to be accepted into the PETT project?

b. Returnee from another area c. Marginalised girl/woman d. Little or no education e. Ex-militia f. Other (please specify)

2. How did you know about the interventions of the PETT project? Through

a. NGO official …b) Neighbour…c) Relative…d) Community leader… e) Other (specify)……….

3. What skills did you learn under the PETT program? ……………………………

4. Did you find the training useful?

a) Very useful………… b) Useful…………… c) Not useful……………

5. What was your level of education before enrolling in PETT training?

a) None…. b) Primary…… c) Secondary….. d) Post-secondary………… e) NFE……………….

6. Please list for me the various ways that you have used to look for employment

a. Have asked my relatives to look for a job for me b. Have walked from potential employer to another c. Have sent my CV/applications to various potential employers d. Have approached my friends e. Have done absolutely nothing f. Other ways (please specify)…………………….

7. If given another chance, would you still go for the same skills that you acquired at the PETT project?

a. Yes b. No

8. If yes please give reasons:

a. It is still exciting for me b. I believe it still has the best chance for employment c. My friends are also in it


d. It is not difficult to learn e. Other reasons (please specify)…………………

9. If no, please give reasons:

a. It is not in demand b. It requires a more advance learning c. Other reasons (please specify)……………………………….

10. What challenges have you encountered while looking for a job?

11. Which of the above challenges do you consider to be as a result of some form of

discrimination?

12. In what ways could some of the above challenges have been addressed by the PETT project?

13. What other course would you rather have combined with the current one to make you

more marketable?


QUESTIONAIRE FOR THE EMPLOYER

Name of respondent………………………………………………………….M/F……

Name company/organization/etc……………………………………………………..

Enumerator ……………………………………………………..

Region………………………… Town: ……………………….

Date of interview…………… Time started………………Time ended………….

1. Please give us the skills’ areas for ALL of your present employees who have been through the PETT project

Skills area No of employees in the skill category Male (No) Female (No) (Use separate sheet)

2. From which Institutions did you get your PETT employees?

Name of Institution

IB or EB or Don’t Know Existing (yes or No or don’t know)

(Use separate sheet)

3. In your opinion, how would you rate the performance of the PETT employees?

a) Very Good b) Good c) Average d) Poor

4. What are the specific reasons that make you rate them as above?

5. Please state ways to improve the effectiveness of those employees’ performance 6. How would you rate the performance of the PETT graduates if compared with your other

employees from other institutions holding similar positions? PETT graduates are:

a. Better than others b. Same as others c. Poorer than others

7. What forms of employment terms are the PETT employees operating under

a. Casual (paid every week/day) b. Fixed Contract (paid in instalments for a fixed amount of time) c. Permanent employment (paid every month for a long time)

8. How did you acquire the PETT employees?


QUESTIONAIRE FOR THE TRAINING INSTITUTION

Name of respondent………………………………......................................... M/F……

Designation …………………………………………………………

Name institution……………………………………………NFE/IBTVET/EBTVET…….

Enumerator ……………………………………………………………

Region………………………… Town: ……………………….

Date of interview…………………… Time started………………Time ended………

1. What are the total number of PETT trainees who have graduated through your institution:

(Use separate sheet)

2. Which of the above courses were most popular with the graduates?

3. What made them popular as compared to the others?

4. Do you have a system of tracing whether your graduates have gained employment?

Yes/no

5. If yes, how many of your total graduates have received employment………….

6. What are the key reasons that have helped the graduates acquire employment

7. What post graduation services have you offered to the beneficiaries?

8. Which of the above services do you think were most helpful to the graduates in acquiring employment?

9. How has your institution benefited in partnering in the implementation of PETT?

10. What challenges do you think that the graduates encounter while seeking employment.

11. Of the above challenges, which ones do you think the PETT project could have

addressed.

12. List the main challenges facing you in assisting the PETT trainees

13. In what ways do you think they can be addressed?

Type of skill Total number of trainees

Training duration

Male (No)

Female (No)


14. What curriculum/syllabus is offered in the institution?

15. How many trainers/Instructors has the institute assigned to the PETT program

16. In your opinion how do your rate their suitability/efficiency in transmitting knowledge,

skills and attitudes to the trainees.

a) High………… b) Moderate…………. c) Low…………

17. Are the present managers/trainers the same ones who started with PETT?


QUESTIONAIRE FOR THE PARENT/GUARDIAN

Name of respondent…………………………………………………………….M/F……

Name of beneficiary……………………………………………………..

Enumerator …………………………………………………………….

Region………………………… Town: ……………………….

Date of interview……………… Time started………………Time ended………….

1. How did your child know about the PETT project?

a. I told him/her about it b. She/he learnt about it from a friend/relative c. Other (specify)

2. What criteria made your child become accepted into the PETT project?

a. We are returnees from another area b. She is a girl and needed a chance c. The child had little or no education d. The child is an ex-militia

3. What did you have to give up in order to contribute for the participation on the PETT project?

4. Do you think it was worth it?

a. Yes (explain)……………………………………………… b. No (explain)………………………………………………

5. Is your child earning some income now?

a. Yes b. No

6. If yes, is the income as a result of the skills training obtained from the PETT project? a. Yes b. No

7. In what ways has the additional income assisted in improving your living conditions? 8. What challenges has your child encountered while looking for employment?

9. How have you overcome (or intend to overcome them if child is still unemployed)?


QUESTIONAIRE FOR THE MOE and MOL

Name of respondent..……………………………………………………..M/F……

Designation …………………………………………………………….

Name of department………………………………………………….

Enumerator …………………………………………… Date………………….

Region………………………… Town: ……………………….

1. What were the original expectations of the Ministry on the PETT project? Was it in line with Ministries policy guidelines on TVET, if any existed? Has there been a review of policy?

2. What have been the roles/responsibilities of the ministry during the implementation

period of the PETT project?

3. How has the Ministry benefited from partnering on the implementation of PETT?

4. What policies has the Ministry implemented to help such graduates like those of the PETT project to acquire employment?

5. What strategies/organizational set up are in place for continuity of the PETT set-up after

external aid ends? What are the exit strategies?

6. What have been the challenges of the PETT graduates acquiring employment?

7. What challenges did you face as a ministry and how were they addressed?


Most Significant Change Case Study Checklist

1. Kindly narrate your life story before you joined the PETT training. Concentrate on:

Family background Education Socio-economic life

2. Comment on the following regarding your training

What skills did you take? What made you join the skills training? How have you used the skill after the training?

3. Narrate how the training has changed your life.


ANNEX V: LIST OF DIFFERENT CATEGORIES OF RESPONDENTS A: REPUBLIC OF SOMALILAND

1. Local Implementing Partners (LIPs)

Town No. Name Position/Institution Hargeisa 1. Eng. Abdullah Essa Director – Hargeisa Technical Institute (HTI) Berbera 2. Farhan Adam Haibe Executive Director- GAVO 3. Osman Ahmed Abdi Chairperson – Activist Network for Disabled People (ANDP) 4. Mukhtar Hassan

Ibrahim Executive Director - Activist Network for Disabled People (ANDP)

5. Rogasian M. Massue HIV/AIDS Youth Advisor - Progressio 6. Dr. Hussein Omar Alrazai Medical Centre 7. Adan Nur Ali Maroodhi Jeex Metal Fabrication 8. Nimao Yusuf Trainer – Aswa Tailoring 9. Abdillahi Ahmed Trainer - Hargeisa Technical Institute (HTI) 10. Ummal Kheyr Nur

Mohamed Soywa Tailoring

11. Kadija Elmi Principal – Worda Tailoring 12. Mohamed Hassan Manager - Allamini Upholstery 13. Hassan Abdullahi Trainee Teacher – Children Development Organisation 14. Intasara Ahmed Bilicsan Beautician 15. Mohamed Khan Awde Electrician – Cawale Workshop 16. Ahmed Hassan Yusuf Manager – Sowda Tailoring 17. Khalid Adan Manager – Mandeeq Confectionaries 1. Abdi Khan Mohamed VTC Manager -GAVO 2. Mohamed Dahir

Hameri Marketing Manager - Red Sea Fishing Co-operative

3. Omar Aden Osman Vice-Chairman - Red Sea Fishing Co-operative 4. Jama Ahmed Principal Trainer “ “ 5. Khadra Abdi Yusuf Family Life Education Centre 6. Mohamed Abdi Trainer Borame 1. Hassan Heif Manager - Dubai Tailoring 2. Ismail Yasin Ahmed Daldhis Aluminium 3. Idris Farah Centre Manager -Africa Youth Develop. Association

(AYODA) 4. Abdullahi Haji

Abdirahman Trainee Administrator - Babunaas

Burao 1. Abdihakim Mohamed

Abdillahi Sitco College

2. Yahya Mohamed Director- Alfa Audio


Town No. Name Position/Institution 3. Mohamed Said Manager – Faroole Electronics 4. Jama Diiriye Farah Beder Electricity Sheikh 1. Hado Mohamed Sheikh School of Training

2. Direct Beneficiaries

Town Total Number Interviewed

Present Status

Male

Female

Employed Un-employed Male Female Male Female

Hargeisa 114 26 88 21 50 5 38 Berbera 80 40 40 30 24 10 16 Borame 33 19 14 15 5 4 9 Burao 41 30 11 17 8 13 3 Sheikh 10 0 10 0 8 0 2 Total 278 115 163 83 95 32 68

3. Indirect Beneficiaries

No. Category Number Interviewed Hargeisa Berbera Borame Burao Sheikh Tot

al 1. Parents/Guardians 11 8 5 6 1 31 2. Employers 8 10 6 5 1 30 3. Training Institutions 15 6 4 4 1 30 4. SCD Staff, Hargeisa

No. Name Position Location 1. Hasheem Simba Programme Manager Hargeisa 2. Farhia Kassim Ibrahim Acting Project Coordinator “ 3. Abdillahi Hussein EPS Officer “ 4. Adan Yusuf Hussein IBTVET Coordinator “ 5. Abdirahman Nur PETT project Accountant “


5. Consortium Implementing Partners

No. Name Position Partner 1. Jael Shisanya Education Centre Coordinator CARE 2. Abdihakim Ahmed Project Manager (Education) CARE 3. Ahmed Abdi Project Officer, PETT CARE 4. Fatuma Shahib Project Officer, NFE CARE 6. Local Counterparts/Other stakeholders

No. Name Position Partner

1. Eng. Ali Abdi Odowaa Director General MOE 2. Ibrahim Diriye Haid Director General MOHL 3. Geraldo Romey Programme Officer WFP 4. Isaie Habimana Associate Programme Officer UNHCR 5. Paul Crook Technical advisor ILO


B: PUNTLAND STATE OF SOMALIA

I. Institutions:

NAME OF INSTITUTE TYPE TOWN PERSON CONTACTED

DESIGNATION

1. Gottenborg Initiative IBTVET Galkayo Mr. Dahir Haji Issack Manager 2. Galkayo Centre for Business

Studies EBTVET Galkayo Mr. Mohamed Hussein

Abdi Manager

3. Cosvo Intitute EBTVET Galkayo Amina Haji Duale Manager 4. General Medical Centre EBTVET Galkayo Saeed Suleiman Manager 5. Mudug Development

Organization EBTVET Galkayo Hassan Hussein Manager

6. Nugal Electric Company EBTVET Garowe Abdirizak Haji Isse Technical Head 7. Puntland State University IBTVET Garowe Peter Mwangi Teacher 8. Garowe Vocational Training

Institute IBTVET Garowe Abdifatah Osman Manager

9. Gollis Telecommunication EBTVET Bossasso Mohamed Mohamud Head of Training 10. Business Women

Organization IBTVET Garowe Asho Hirsi Ali Chairlady

11. Qardo Women Network EBTVET Qardo Halima Mohamed Jafar

Manager

12. Somali Institute of Business Studies

IBTVET Qardo Abdihakim Mohamed Osman

Manager

13. Towhiid Capacity Building Intern centre

Qardo Eng. Alasis Sheikh Hussein

Owner

14. Bossaso Technical Vocational Institute

IBTVET Bossasso Jama Osman Shibir Manager

15. School of Professional Studies and Service

IBTVET Bossasso Zeynab Haji Ayan Manager

16. Somali Acra Action Institute EBTVET Garowe Farhia Mohamed Ismail

Manager

17. Dandor Institute EBTVET Bossasso Mohamed Khalif Manager


II. GOVERNMENT OFFICIALS CONTACTED PERSON CONTACTED MINISTRY DESIGNATION

1. Abdirahman Ali Dakanne MoE REO Muddug 2. Mohamed Ceynab Mohamud MoE REO Bossasso 3. Abdirahman Sharriff MoE REO Qardo 4. Mohamed Jibril MoE Director General 5. Mohamed Gobe MoE Vice Minister 6. Awes Hirsi Jamac MoL Director General

III. OTHER PERSONS CONTACTED

PERSON CONTACTED ORGANIZATION DESIGNATION 1. Mukhtar Alasow UNHCR Field Officer 2. Jawahir Mohamed Cilmi CARE International Project officer 3. Olad Mohamed Yussuf Gottenborg Initiative Project Officer 4. Ahmed Mohamed Shire Diakonia Sweden Project Coordinator 5. Fatuma Dahir Diakonia Sweden EPS officer 6. Abdirahman Dakane ILO Field Officer


ANNEX VI: LOGICAL FRAMEWORK MATRIX (Original)

Overall objective

Intervention logic Objectively verifiable indicators of achievement

Sources and means of verification

Assumptions

Improved livelihood of disadvantaged youth and women in Somaliland and Puntland

1. Increased employment rate of disadvantaged youth and women by 10%.

2. Increased income of trained disadvantaged

youth and women.

Tracer study report of the project Project final review/evaluation report Ministry of planning annual statistics

Specific objective

Sustainable Employment opportunities created for 4,525 youth and women with equal gender composition in Somaliland and Puntland by the end of the project

1. 75% of trainees got employment opportunities after completion of training

2. Demand for Technical, Vocational, Education

and Training (TVET) by target groups increased

3. Demand for TVET graduates by the labour

market increased 4. Improved capacity of TVET management

including implementation of cost recovery system by IBTVET centers

5. Application of entrepreneurship skills through

creation of self-employment schemes

Monitoring and progress reports Tracer study report of the project Training enquiry (register) maintained by partners EPS reports Project final review/evaluation report

Presence of enabling policy environment for TVET Support of municipalities and business communities in Employment Promotion Services Acceptance of the project by local communities Absorption capacity of local economy improves Security situation in Somaliland and Puntland does not deteriorate. Target communities support female/women in taking up employment opportunities (EO) EO do not decrease due to macro-economic shocks

__________________________________ PETT TERMINAL EVALUATION REPORT

Expected results

1. Access of disadvantaged* youth and women to TVET and Employment Promotion Services (EPS) improved

1.1 Equal enrolment (male/female) of targeted number of youth and women

1.2 Five Enterprise Based Technical, Vocational, Education and Training (IBTVET) centers strengthened and made operational

1.3 Presence of diversified and standardized skill areas with due gender consideration

1.4 At least 4,525 disadvantaged youth and women got relevant/demand driven skills training in IBTVET centers (2,525 trainees) and Enterprise Based Technical, Vocational Education and Training (EBTVET) centers in (2,000 trainees)

1.5 EPS available in four locations within six months of project start up

1.6 5,000 beneficiaries have utilized EPS by the end of the project period

1.7 3,000 youth and women have acquired literacy and numeracy relevant to TVET

Monitoring and progress reports Field visit and record of IBTVET EPS records

2. Competency in terms of technical and entrepreneurship skills of target groups increased in accordance with employment promotion

2.1 At least 80% of TVET trainers/instructors trained and hence posses needed skills.

2.2 Trainers/instructors are able to transmit required knowledge, skills and attitudes to trainees according to established norms

2.3 Standardized and relevant training packages/ curricular materials used by TVET centers

2.4 90% of trainees passed exam or acquired basic skills

Monitoring and progress reports Field visit and records of IBTVET Training quality supervision reports

3. Availability and use of information on Local Economic Development (LED) opportunities improved

3.1 The extent in which the TVET and EPS activities are informed by and adopted to LED opportunities

3.2 Networking and linkage system/platform established between TVET centers and relevant stakeholders including between trainees and potential employers.

Study reports Project progress reports Minutes of meetings


4. Capacity of local associate implementing partners and authorities enhanced with respect to TVET and EPS

4.1 Policy guideline, operational, management and financial systems for TVET centers revised/developed and implemented

4.2 Cost recovery system developed and used by TVET centers

4.3 TVET managers and supervisors trained in management and supervision skills

4.4 Quality and relevance of EPS services provided to beneficiaries

Monitoring and progress reports

Field visit and records of IBTVET

Increase in the level of funding from local sources for provision of TVET and EPS

Activities Means: Cost

Result One: Access of disadvantaged youth and women to TVET and EPS improved

1,133,870

a. Assess technical and management capacity of local implementing partners

8,400

b. Identify and select local implementing partners

c. Develop and sign project agreements with local implementing partners

d. Induct partners on the project and how to run and manage TVET activities

2,700

e. Prepare detailed operation plan with partners

f. Develop a protocol/guidelines for selecting enterprises for undertaking EBTVET

g. Conduct a survey of the 4,800


enterprises offering EBTVET

h. Select enterprises for training EBTVET as per preset protocol

i. Develop and sign training agreement between partners and EBTVET training enterprises

j. Induct EBTVET enterprises on running and managing training activities

3,200

k. Upgrade five IBTVET centers' working facilities as required

29,000

l. Supplement basic tools, equipment and furniture for IBTVET centers as required

78,000

m. Provide identified safety materials and training tools to TVET centres based on agreed procedures

4,000

n. Select and assign trainers as per established qualification for IBTVET centres

o. Select trainees based on preset criteria and procedure for both IBTVET and EBTVET

p. Support user fee of IBTVET trainees

473,450


q. Support use fee of EBTVET trainees

468,000

r. Implement TVET to selected trainees

s. Establish and operate four EPS centers within IBTVET

7,200

t. Conduct local labour market study to inform the subsequent selection and focus of TVET trades/skills

12,000

u. Implement NFE in numeracy and literacy

43,120

Result Two: Competency in terms of technical and entrepreneurship skills of target groups increased in accordance with employment opportunities

186,755

a. Translate, adapt and reproduce UNESCO's TVET training materials for both IBETVET & EBTVET

10,500

b. Identify, review and develop standardized training package for skill areas not included in UNESCO curriculum package

48,600

c. Conduct training needs assessment for TVET trainers/instructors

12,200


d. Develop training manual and conduct training of trainers/ instructors (methodology)

4,000

e. Conduct upgrading training (technical) for trainers/instructors

f. Provide reference training materials (books etc.) to IBTVET centers and selected EBTVET with rules of their utilization

10,000

g. Develop/implement quality control/supervision system

26,760

h. Develop and implement assessment examination/ certification guidelines for both IBETVET and EBTVET

19,575

i. Engage enterprises and organize practical/ apprenticeship training (outside of IBTEVET centers)

10,000

j. Review, adapt and make available existing NFE mat. and conduct NFE

45,120

Result Three: Availability and use of information on Local Economic Development opportunities (LED) improved

54,000


a. Review existing LED materials, identify existing gaps and synthesize for use in TVET and EPS` activities

5,000

b. Conduct LED study to fill in the identified gap

11,000

c. Provide information about employment opportunities

d. Establish linkage between trainees and potential employers

e. Establish networking forums with municipalities chambers of commerce, enterprises and Ministry of Labour etc- a process of engaging stakeholders

12,000

f. Conduct tracer study and follow up of trainees after graduation

16,000

g. Document project best practices and disseminate

10,000

Result Four: Capacity of local associate implementing partners and authorities enhanced with respect to TVET and EPS

123,725

a. Conduct capacity assessment of IBTVET centers and local

7,200


partners

b. Assist the MOE to review TVET policy guideline

19,200

c. Support development and implementation of effective finance (cost-recovery) and management systems for IBTVET centers

d. Provide capacity building training in management and supervision of IBTVET activities to MOE and local partners

12,500

e. Provide capacity building training to IBTVET centres managers

4,825

f. Provide implementation/coordination support to local partners

37,000

Monitoring, Review and Evaluation of the Project

g. Undertake quarterly monitoring of project progress

h. Conduct half annual review exercise with local partners

20,000

i. Conduct half annual review exercise with EBTVET enterprises

10,000


j. Carry out final project review six months before completion

13,000

Common Activities Means Costs a. Project Staff salaries 599,040 b. Per diem -monitoring 81,300 c. Travels 36,900 d. Office equipment,

furniture and supply including EPS

18,300

e. Local project office costs 220,680 f. Auditing 12,000 g. Financial charges - bank

charges 32,000

h. Administration 7% 169,978 2,598,228 x The disadvantaged include women and girls, internally displaced persons, returnees, out-of-schools youth, ex-militia, the handicapped and minority groups