end of term evaluation democracy & human rights … … · pepp puntland education policy paper...

1

Delta Partnership: Evaluation of the Democracy & Human Rights Programme – Phase IV

END OF TERM

EVALUATION –

DEMOCRACY &

HUMAN RIGHTS

PROGRAMME –

PHASE IV

FINAL REPORT

Produced by Delta

Partnership for

DIAKONIA

Date: 13th January 2014

2


Contents

Acronyms

Contents ............................................................................................................................... 2

Acronyms ...................................................................................................................................................... 2

Acronyms .............................................................................................................................. 4

1. Executive Summary ...................................................................................................... 5

1.1. The Programme .................................................................................................................................. 5

1.2. Findings ............................................................................................................................................... 5

1.3. Conclusion and recommendations ..................................................................................................... 8

2. Introduction and methodology ..................................................................................... 9

2.1 Analytical framework .......................................................................................................................... 9

2.2 Report structure ................................................................................................................................. 9

2.3 Evaluation objectives ........................................................................................................................ 10

2.4 Methodology and processes............................................................................................................. 10

3. Context and background ............................................................................................. 12

3.1 DIAKONIA .......................................................................................................................................... 13

3.2 The Democracy & Human Rights Programme, Phase IV .................................................................. 14

3.3 Evaluation of phase 3 of the DHRP programme ............................................................................... 15

4. Findings ...................................................................................................................... 18

4.1 Programme design ............................................................................................................................ 18

4.2 Partnership with Diakonia ................................................................................................................ 18

4.3 Relevance .......................................................................................................................................... 19

4.4 Portfolio performance - Activity completion .................................................................................... 19

4.5 Efficiency ........................................................................................................................................... 20

4.6 Sustainability ..................................................................................................................................... 21

4.7 Innovation, replication and scaling up.............................................................................................. 22

4.8 Result areas - Outcomes ................................................................................................................... 22

4.9 Diakonia as the facilitator ................................................................................................................. 33

3


5. Conclusions and recommendations ............................................................................. 34

5.1 Short term ......................................................................................................................................... 34

5.2 Future programming opportunities .................................................................................................. 34

6. Annexes ..................................................................................................................... 38

Annex 1 - ToRs ..................................................................................................................... 39

Annex 2 – Self Assessment questionnaire guideline ............................................................. 43

Annex 3 - Partners meeting Agenda (Garowe) ...................................................................... 45

Annex 4 - Partner interview template .................................................................................. 48

Annex 5 – Staff questionnaire template ............................................................................... 51

4


Acronyms

ARDC Abyan Research Development Centre ARS Achievement Rating Scale CBO Community-Based Organisation DEO District Education Officer CEC Community Education Committee CPD Centre for Peace and Development CV Curriculum Vitae FAWESOM Forum for African Women Educationists Somalia FECNET Female Education Centres Network FGDs Focus Group Discussions FGM Female Genital Mutilation GARWONET Gardo Women Network GECPD Galkayo Education Center for Peace and Development GGSS Gardo Girls Secondary School GRC Garowe Resource Centre GTEC Garowe Teachers’ Education College HIV/AIDs Human Immune Virus/Acquired Immune Deficiency Syndrome IDP Internally displaced Person KAALO Local NGO MoE Ministry of Education MoWDAFA Ministry of Women’s Development and Family Affairs NFE Non-Formal Education NGO Non-Governmental Organisation OECD –DAC (Organisation for Economic Cooperation and Development - Development

Assistance Committee OVI Objectively Verifiable Indicators PCC Puntland Community College PEPP Puntland Education Policy Paper PETT Promotion of Employment through Training PSU Puntland State University Puntland Regional State of Somalia REO Regional Education Officer SAACOM Somali Agro Action Community SCG Somali Consulting Group SDO Somali Development Association SIDA Swedish International Development Agency SFS Somali Family Service (Garowe Resource Centre) ToR Terms of Reference UNICEF United Nation’s Children Fund WAWA We Are Women Activists

5


1. Executive Summary

This evaluation was commissioned for learning and accountability purposes and to assist in charting

the future of the programme. The evaluative process generated relevant findings, lessons and

recommendations that will guide and inform the future of Diakonia’s Somalia Country programme,

especially in relation to the new Swedish Results Strategy, and the Democracy and Human Rights

programme, in particular.

Delta Partnership1, the independent assessor for this evaluation, used a variety of techniques to

carry out the evaluation including a stakeholder self-assessment exercise, literature review, key

informant interviews, focus group discussions as well as partner meetings. An analytical framework

was devised, which examined the progress made so far, successes, challenges as well as capturing

recommendations for future programming. The full results are highlighted in this report.

1.1. The Programme

The Democracy and human rights programme has an overall objective of promoting democracy,

human rights and gender equality in North-East and Central-South Somalia; with the specific

objective of empowering the community through the civil society to qualitatively participate in the

democratisation process in Sool, Sanaag, Puntland and Galmudug. The SIDA-funded Democracy and

Human Rights Programme – Phase IV, is a 3-year programme that started in April 2011 through

March 2014, implemented by Diakonia through its partners in Somalia.

This fourth phase of the Democracy and human rights programme builds up on the previous three

phases of the programme, which have focused on promoting democratic principles and values,

culture, gender equality and human rights by increasing access to formal and non-formal education

and training, vocational skills development, promotion of peace and civic education, awareness

raising on emerging issues including HIV/AIDS, female genital mutilation (FGM) and gender equality

and strengthening the capacity of civil society organisations and key local institutions involved in

education and training, human rights and democracy initiatives in North East and Central South

Somalia.

1.2. Findings

Relevance - Over the period that this programme was implemented, great strides have been made

on the democracy and human rights fronts and the interventions chosen were proven to have been

highly relevant. There was no standardisation of approaches across the different geographic settings

and this helped encourage innovative and localised ideas that not only ensured buy in but also

helped spread the messages clearer and faster.

Diakonia’s strategy for change has been a three step process with the first step being imparting

knowledge then helping the boundary partners to organise themselves and finally encouraging them

to lobby and advocate for their rights with the duty bearers. This logical sequence has been

successful with one of the unexpected results of the programme being the fact that partners are

1 www.deltapartnership.com

6


voluntarily organising themselves into committees in order to better address pressing issues

together rather than individually.

Whereas strategies for working directly with partner organisations are key, future programmes will

need to make explicit strategies for influencing the environment of the partner organisations. One

approach could be having causal, persuasive and supportive strategies for dealing with the partners

as well as their supportive environment

The programme’s geographic focus was in Puntland and some regions of South Central Somalia and

this was informed by consultations between Diakonia and the CSO’s. These areas are a mixed bag

with some regions being more stable than others. The appetite and tolerance for the

democratization and human rights agenda was higher in some places and lower in others and this

was reflected in the success stories coming from different areas.

Effectiveness – The programme has been a success in attaining its expected outcomes. A summary

of the main accomplishments of the programme is given below:

Democratic Changes - Since the beginning of the democratisation process in Puntland, there has

been a change in the attitude of the people in the embracing of institutionalised democracy. The

community expressed increased faith and confidence in the system and this was demonstrated by

the relatively peaceful acceptance of the extension of the presidential term which in the past has

been met by violent opposition.

Education sector transformation - The Ministry of Education has improved in its provision of services

to Puntland citizens. This is a culmination of numerous and resolute interventions from the different

actors (including Diakonia) across the sector. In phase IV, apart from being able to lobby for an

increase in the funding allocation to the sector, the ministry has been able to conduct its own census

of schools, develop a coherent strategic plan and enact an education act. The ministry is also

producing data and information booklets about its work, therefore enabling ease of design of

programming by interested actors.

Diakonia’s history of engagement with this sector has been impressive and interventions have

progressed from building the physical structures to supporting development of policies, processes

and skill sets. In phase III, the programme supported the human resources department to learn how

best to utilise the printing press that had been unused for 2 years. There is now a functional printing

press is operational at the ministry.

Increased security and tolerance - Through the dialogue processes of getting the police to engage

with the community that they serve, the programme has seen the formation of security committees

in 3 towns in Puntland. These committees are now playing a key role in enabling the police deal with

security situations. Through the school based peace committees, the programme is influencing

future citizens of Puntland in using non-violence means to resolve conflict.

Government /Media relations strengthened – One of the results has been the relationship between

the government and MAP has improved greatly. This was attributed to the fact that the resource

person has been acting as an impartial go-between between the two parties.

7


Some of the outcomes from this process are - The media code of conduct was developed in

collaboration between the media fraternity, government and members of the Somali Media Support

Group2 (SMSG), in addition, a specific code of conduct for the election processes in Puntland was

developed. The Puntland media law was reviewed in collaboration between the government and the

media fraternity and was approved by the Puntland cabinet in December 2013. It is now pending to

be approved by the Puntland parliament (which had not been constituted by December 2013)

Legal and Regulatory Framework on Property Rights - The participants of the dialogues on land

issues in Puntland have resolved to establish a task force that will continue to advocate for the issues

of land rights. Currently, the taskforce which is in its infancy stage will be formally constituted as a

pressure group on land issues.

Taxation and budgeting - After dialogue processes on taxation issues, an Accountability Committee

was formed by the participants to push for the establishment of proper channels for national

resource mobilisation; budgeting of resources and utilisation of the same resources. The pressure

group is looking at working closely and supporting the government in raising the resources required

for nation building, as well as being a watchdog for the utilisation of the resources within the

country.

Gender inclusiveness - Women’s inclusion and participation especially in politics has been on the

increase and this ultimately led to the establishment of the ministry of women development and

family affairs (MoWDAFA). The programme contributed towards this through its initiatives of

establishing women’s centres that have become focal points for implementation of the literacy

programmes.

Value for Money - In a similar way to the increased rigour required for indicators and evidence, the

requirement for analysis around VFM has also grown. The data required for this level of analysis is

therefore not always available. It is noted that there is a clear focus on economy in the country

offices. An analysis of spent budget shows that more than two-thirds of the expenditure directly

funded activities.

Management -The programme has been well-managed at the country level. Diakonia is perceived as

a strong partner with skilled staff. This has enabled the programme to play a vital role in bringing

together the different stakeholders and has increased the speed of implementation. Areas for

improvement include communications and strengthening the Planning, Monitoring and Evaluation

(PM&E) systems to provide better data for planning and analysing performance.

Sustainability and replicability - The programme has been successful in building institutions such as

the GTEC and the PSU, in building capacity of partner organisations across the Puntland region to

effectively engage with the citizens, local authorities and government. Networks have been created

and strengthened, materials such as handbooks and guides have been customised, shared and are in

use. The young population have also been involved in some of the interventions.

2 The Somali Media Support Group (SMSG) is a coordination body that brings together local and international

organisation and donors working in Somalia on media issues. The group meets once a month in Nairobi.

8


One caveat is that Diakonia’s partnership approach is characterised by providing partners with both

technical and financial support. While this approach has worked reasonably well, it has also created

overdependence on Diakonia by some of the partners. This was expressed by some of the partners

who were hugely concerned about SIDA’s impending shift of support from the education sector

thematic area, which will inevitably impact the level of funds available in the future.

1.3. Conclusion and recommendations

The programme worked: it delivered significant successes and leaves a legacy of individuals and

institutions that can continue to improve democracy and human rights in the region and beyond.

The partners are appropriate and have an ongoing capacity to deliver further programmes. The high-

quality staff has ensured that local ownership is high and that Diakonia has an excellent reputation.

The programme did the right things, most often in the right way. Recommendations therefore

mainly relate to improvements for future programmes. Such programmes should be designed to be

able to take advantage of emerging opportunities in a politicised environment, there needs to be

good quality information available on current events, decisions need to be able to be made quickly

and resources available to follow through. A stronger culture of focusing on results, of gathering

evidence to underpin the achievements, supported by stronger systems (including financial) across

the organisation will enable Diakonia to meet the increasing demands for rigour.

Force field analysis3

We used the tool below to analyse some of the forces for and against the desired change. You can

use it for two purposes: to decide whether to go ahead with the change; and to increase your chances

of success, by strengthening the forces supporting change and weakening those against it.

3 http://www.mindtools.com/pages/article/newTED_06.htm

Achieving Diakonia’s objectives and mandate.

Political and public goodwill and unique mandate

Competent staff

Unique and relevant approaches and interventions.

Boldness – venturing where others fear to go

Using local partners

Funding short-falls

Partner capacity

Skills gap – such as M&E skills

Partner overdependence on Diakonia’s support

Inhibitors Facilitators

http://www.mindtools.com/pages/article/newTED_06.htm

9


2. Introduction and methodology

2.1 Analytical framework

To understand how the programme has been performing, the framework below was used to inform

our interview structure and analysis of relevant documents. The framework depicts interaction

between the two key stakeholder groups. The working assumption underpinning this is that in order

to bring about the desired change in policies, practice and power relations our strategies for

collaboration between the groups and with the external environment will play a crucial role.

Diakonia in this instance entails both the Nairobi office as well as their Punt-land office who are

directly involved in the day to day running of the programme. The partnership between the two

offices has made the implementation process not only efficient but has increased the overall

effectiveness of the programme. The key areas under scrutiny were the organisational practices,

systems and structures.

The evaluation also looked at Diakonia’s partnership approach and its strategy for change.

2.2 Report structure

Coming after this introductory section, Chapter 2 provides an analysis of the context within which

the programme was operating, analysing the political environment, Diakonia’s strategic focus, and

lastly the DHRP programme overview. The overall key findings are summarised in chapter 3 and are

broken down into the OECD -DAC (Organisation for Economic Cooperation and Development -

Development Assistance Committee criteria) (Impact, effectiveness, efficiency, sustainability and

innovation). Chapter 3 also includes an analysis of the five key result areas of the programme.

Finally, Chapter 4 uses the contextual analysis and findings as the basis for highlighting opportunities

for improving future programmes and partnership arrangements.

Diakonia Strategies

(Interventions)

Outcomes

Changes in:

Policy

Practice

Improved

relations

between –

Citizens,

CSO’s and

Government

Partners

10


2.3 Evaluation objectives

The evaluation was intended principally for learning purposes, accountability purposes and to assist

in charting the future of the programme. The evaluative process generated relevant findings, lessons

and recommendations that are expected to guide and inform the future of Diakonia’s Somalia

Country programme, in relation to the new Swedish Results Strategy, and the Democracy and

Human Rights programme, in particular.

The overall objective of the evaluation was to provide Sida, Diakonia, local partners, stakeholders

and project beneficiaries with necessary data and information for making informed decisions

regarding the performance of the Democracy and Human Rights Phase IV programme in terms of its

efficiency, effectiveness, impact and sustainability, and also to take appropriate action on any

required changes in the programme design, development and implementation strategies for the

future.

2.4 Methodology and processes

The Delta Partnership approach is to work in partnership with our clients, combining their

understanding of the programme, the environment and context, and pressing challenges with our

monitoring and evaluation (M&E) understanding and good practices in development.

We used an inclusive assignment management approach where we came to a common

understanding of the assignment’s terms of reference during an initial meeting with the client upon

award of the contract. From this meeting, we were able to plan and design an inception report

which detailed the nuts and bolts of the job for submission and approval from the client. During this

inception phase, we researched and reviewed the relevant documentation and monitoring &

evaluation framework(s) in place. While implementing the assignment, we used a creative and

supportive approach ensuring that our client’s needs were met, and that programme stakeholders

were involved. We used the most up to date technical knowledge on the assignments, and made

concrete suggestions along the way to improve the process.

The framework below was informed by our experience gathered from conducting similar

assignments with similar programmes and within the same or similar geographic context.

Initial briefing/Fact Finding – This was to confirm our shared understanding of the context of the

work, what has been done previously and what is expected from us. The meeting helped us to

understand expectations and further help the client understand our approach and this also gave

both parties at an early stage an opportunity to raise any pertinent issues relating to the assignment

at hand.

Internal Self assessments – We believe the experts on the implementation of a certain project or

programme is the team doing the actual implementation. They have the most extensive knowledge

and understanding of what is being done and with what consequences and as such they possess a

treasure chest of information.

11


We allocated the team an appropriate length of time to answer three key questions:

Did we do what we said we would do? – Internal validity.

Did it make a difference – Key successes? Can this be attributed to our interventions? What

evidence do we have? – Impact assessment.

Were these the right things to do? What should we stop/ continue doing going forwards? –

Strategy and learning.

In preparation for the self-assessments, members re-read relevant documents that are related to

their areas of responsibility. The team then comes together and are facilitated by one of the

participants. They take each of the three questions and come up with an agreed answer for each.

We have found this initial step to be very important in bringing out a more wholesome/robust

overview of the successes, challenges and proposed recommendations going forwards. The fact that

these recommendations are internal also helps in ownership and ease of implementation going

forwards.

Desk review – This entailed going through the relevant programme documentation. For this

assignment, this included the project design documents and log-frame, amongst other documents.

This served to bolster the self-assessment report. We identified any patterns and used this

opportunity to go through project management information in general paying special attention to

the balance between cost (monitoring the money), time and quality/quantity.

Key Informant Interviews – This was done once we had gone through the team self-reviews and

desk reviews and was used as a follow up exercise to fill in any gaps left out by the two previous

stages. We also took this opportunity to talk to key stakeholders such as the steering committee

and donor partners to gauge their view on the performance of the organisation as a whole as well as

that of the relevant programmes or projects. Focus Group Discussions with identified key

stakeholders including partners and beneficiaries was also undertaken.

Analysis and report writing - The findings from the interviews, questionnaires, observations and desk research were collated and synthesised. We also believe that it was vital to use clear and simple language for clear communication reinforced by clear diagrams and pictures. Validation Workshop - A one day workshop with Diakonia staff/ stakeholders was undertaken to disseminate the preliminary report and to capture their comments.

Final comprehensive report - This final report was be put together incorporating the feedback from the validation session.

12


3. Context and background

SOMALI REPUBLIC: The Somali Republic is in the Horn of Africa, bounded to the north by the Gulf of

Aden and in the east by the Indian Ocean. It shares land borders with Djibouti to the north-west,

Kenya to the south-west, and Ethiopia to the west. The Somali population is one of the most

homogeneous in the whole of Africa, in terms of culture, language and religion. Somalia is an arid

country, having only 1.64% of its land arable. 80% of the population is nomadic or semi-nomadic

pastoralists, whose mainstay is rearing goats, sheep, camels and cattle.

At the beginning of 2011, the population of the Somali Republic was estimated at 10 million, with

85% of the people being ethnic Somalis, and the remaining 15% being non-Somali ethnic minority

groups comprising: Banadiri, Bravanese, Bantus, Bajuni, and others. The annual population growth is

estimated at 2.809, and the birth rate is 43.33 births per 1,000 people. Most of the population is

young; the gender ratio is roughly balanced. There is an increasing urbanization trend, with 37% of

the population currently in towns and cities. Despite the collapse of the Somali state in January

1990, despite the civil war, the economy of the country has continued to grow and trade has

flourished – even in the most insecure of places, Mogadishu.

PUNTLAND STATE: The Puntland State, where the projects supported by Diakonia are mainly

concentrated, is an offshoot of the protracted civil war in Somalia. When the country disintegrated

into warring clan-based fiefdoms, the people from the former five regions of the Somali Republic in

the north-eastern part of the country formed the Puntland State of Somalia in August 1998. A three-

year Transitional Charter guaranteed separation of powers and envisaged a federal Somalia. In 2001,

when the Transitional Charter expired, the Puntland Parliament approved a transitional constitution,

which had gone through a number of review stages. The current Parliament approved the final

version of the Constitution in 2010. The different versions and drafts of the Puntland Constitution

had all stipulated and called for democratization, the formation of political parties, decentralization

of powers to the districts, and convening of elections at the earliest possible time.

The Puntland political profile consists of the three branches of a modern state4: executive, legislative

and judiciary. A clan-nominated 66-member parliament acts as a legislative body and elects a

president, who presents his cabinet to parliament for approval. Abdirahman Mohamed Mohamoud

heads the incumbent administration, which was installed in January 2009. Until the end of 2007, the

Puntland region remained peaceful and relatively stable. However, since the start of 2008, the

region has experienced a spill-over from the armed struggle raging in the South-Central regions.

Religious groups fighting there are also establishing footholds in Puntland and, intermittently,

executing organized political assassinations, which often target the administration’s officials. There

have also been bombings and ambushes of security forces. An increasing sea piracy trend is another

security threat and a serious de-stabilizing factor in Puntland. Despite all of this, the Puntland

administration has recently taken drastic steps to strengthen internal security, curb piracy, and act

against organized crime.

4 DHRP phase IV – Mid-term review report

13


In spite of the security concerns, the economy of the Puntland is quite vibrant and has been growing

since 1991. Puntland’s own business and commercial enterprises are thriving. Also, the region has an

outlet port to the Gulf states and it handles a massive goods transit movement to and from the

central regions and large parts of the 5th region of Ethiopia. Historically, the region was known for

trade and animal husbandry of a nomadic pastoralist kind. The marine resources of the region are

also vast, but little has been tapped to-date. A small community using artisanal fishing crafts

traditionally lived on fishing but, since the 1990s, lobster fishing has attracted a considerable portion

of the population to earn from this new industry. Livestock export is still the economic mainstay, but

the growing lobster fishing industry and frankincense exports are also major hard currency earning

enterprises.

International assistance plays a significant part in the development of Puntland. A sizeable number

of UN and other international organisations are active and operating in the region, having impacts in

different sectors. Among these agencies are UNDP, UN-HABITAT, UNHCR, UNICEF and WFP. As well

as Diakonia, there are a number of other international NGOs such as ADRA, DRC, Mercy Corps, NRC,

and Save the Children. Together, they employ a considerable local workforce, reducing

unemployment and bringing income to the region. They fund numerous development projects, while

supporting the maintenance of infrastructure and the provision of social services such as education,

health, water and sanitation, and urban planning.

3.1 DIAKONIA

Diakonia, a Swedish NGO promoting international cooperation with civil societies in Africa, South

and Central America, the Middle East and Asia, aims in its humanitarian and development work to

help establish a more just world where all human beings have a right to a life of dignity. This entails

people having their basic needs for food, shelter, education and health satisfied; having a cultural

and spiritual identity; and having opportunities for exercising the rights and responsibilities of

citizenship.

In Somalia, Diakonia has been working since 1994 not only in close co-operation with civil society but

also with local and traditional authorities. Here, the primary focus of Diakonia’s work is to promote

democratic values, to achieve gender equality and to protect human rights through improving the

quality of education and rising awareness on a range of social issues. It also supports and

strengthens the capacity of civil society organisations, which are involved in educational and

democratisation initiatives in Puntland and central/south Somalia.

Diakonia’s thematic areas of intervention in Somalia – democracy, gender equality, human rights,

social and economic justice, peace and conflict management – are all in line with the Swedish

government’s policy and strategy for development cooperation with Somalia, which is to promote

lasting reconciliation, enhance stability and re-establish good governance in order to reach the

overall goal of poverty eradication. The approach is also in line with the Millennium Development

Goals of eradicating extreme poverty and hunger, promoting gender equity and empowerment,

achieving universal primary education, ensuring environmental sustainability and developing a

global partnership for development. It also resonates well with EC support to Somalia in the

14


education and training sector, which aims to improve the quality and relevance of available

secondary and tertiary education, vis-à-vis employment opportunities; to reduce youth

unemployment by at least 50% for both genders.

Diakonia’s Role in Implementation - Diakonia prides itself in building the capacity of local partner

organisations in order to enable them function better in the implementation of projects, as well as

build a strong civil society. In addition, Diakonia is part of the national (and international) advocacy

processes for changes in various sectors and issues.

During the Phase IV programme, Diakonia opted to an “accompaniment” process of building partner

capacity. Accompaniment process is seen by Diakonia as an overarching process that reinforces

other capacity building components. It is possible for people to attend workshops, trainings and

other forums, learn many new things and, for different reasons, do nothing about them. It is also

possible for organizations to have systems and structures and fail to operationalize them. However

through effective accompaniment, such partners would be identified, supported, encouraged and

walked through the difficult phases of adopting change. It is envisaged that there are different ways

of various processes of providing accompaniment for capacity building of partners; some might

include continuous contact with the partner for a consecutive duration of time, while others might

be sporadic engagements with clear achievements set out. However, the process of engagement,

the principle of accompaniment remains the same.

3.2 The Democracy & Human Rights Programme, Phase IV

The SIDA-funded Democracy and Human Rights Programme – Phase IV, titled “An integrated

approach to enhance democracy and human rights in Sool, Sanaag, Puntland and Galmudug”, is a 3-

year programme that started in April 2011 through March 2014, implemented by Diakonia through

its partners in Somalia.

The programme has an overall objective of promoting democracy, human rights and gender equality

in North-East and Central-South Somalia; with the specific objective of empowering the community

through the civil society to qualitatively participate in the democratisation process in Sool, Sanaag,

Puntland and Galmudug.

The programme, which is in its fourth phase, has the following results;

1. Result 1: Local partners influencing the rights holders and duty bearers in adopting and

applying the principles, practice and values of good citizenship and governance,

2. Result 2: Local partner promoting peaceful co-existence,

3. Result 3: Empowered community (both genders) participating in the democratisation

process in Somalia,

4. Result 4: Local partners effectively engaging in the democratisation process and

advancement of human rights,

5. Result 5: Improved programme development for Diakonia.

This fourth phase of the programme builds up on the previous three phases of the programme,

which have focused on promoting democratic principles and values, culture, gender equality and

15


human rights by increasing access to formal and non-formal education and training, vocational skills

development, promotion of peace and civic education, awareness raising on emerging issues

including HIV/AIDS, female genital mutilation (FGM) and gender equality and strengthening the

capacity of civil society organisations and key local institutions involved in education and training,

human rights and democracy initiatives in North East and Central South Somalia.

The programme has been implemented through 12 local partner organisations, namely; Kaalo Relief

and Development Organisation, Somalia Family Services (SFS), Youth Employment Summit – Somalia

(YES Somalia), Samo Development Organisation (SDO), Puntland State University (PSU), We Are

Women Activists (WAWA), Forum for African Women Educationists – Somalia Chapter (FAWESOM),

Garowe Teacher Education College (GTEC), Gardho Women Network (Garwonet), Ilsan Women

Coalition, Abyan Women Organisaionr and Female Education Centres Network (FECNET), and

Ministry of Education (MoE) and Ministry of Women Development & Family Affairs (MoWDaFA).

3.3 Evaluation of phase 3 of the DHRP programme

The phase 3 evaluation provided an opportunity to learn from the implementation experience to

date and to stand back from the day-to-day operation of the programme and assess both the

direction and the ways of working of the programme. This opportunity was well taken and the

evaluation perceived as having had a strong and positive influence on the programme. It

summarised key lessons with supporting recommendations to adjust the focus, activities and

increase effectiveness. The table below outlines the recommendations, together with the resulting

action taken, as noted by WFD staff.

Key Achievement in previous Phases

Identified Gaps How phase IV planned to intervene

Supported the development of credible civil society organisations since the inception of the programme in 1994, which has led to a civil society with a recognised role by government and other actors

The civil society is more evident because of its role in service delivery.

Several consultative workshops will be supported aiming at strengthening interaction between government and civil society.

Establishment of women centres that have become focal point for the implementation of the literacy programme. The beneficiaries have engaged government for policy change in women participation and recognition in politics and finally led to the establishment of the Ministry of Women Development and Family Affairs

The women centres formed umbrella groups and networks which were to add value to their work, by linking them to markets and resources. This value addition element has not taken root yet

The programme has proposed to build the capacity of the umbrella groups in the value addition for their members.

When Diakonia started working in Puntland, the level of conflict was very high. Through peace building interventions, including the do-no-harm approach to programming

Because of the strong clan affiliations and traditional methods of resolving conflicts (which sometimes aggravate conflict) there is need to continue with

Support interventions geared towards realization of a functional legal system.

16


Key Achievement in previous Phases

Identified Gaps How phase IV planned to intervene

Puntland is now relatively peaceful. The key local partners are spearheading the do-no-harm approach and are propagating the same.

peace building and conflict prevention interventions within a legal framework.

The government of Puntland is young and Diakonia has contributed significantly in its development e.g. human resource development, systems development and structures.

The government is still not in a position to deliver on all of its obligations to its populace.

Diakonia recognizes the government as the main duty bearer and will therefore continue to support its institutional development for effective delivery of services.

Diakonia is credited with the development of physical infrastructure during the re-building of the state after the war. These have provided the facilities required for the implementation of social services such as education.

Maintenance of the infrastructure and education financing are a challenged because of government’s inability.

Advocate for budgetary allocation for maintenance and capacity building in regards to the same.

Increased awareness on the international conventions and instruments for measuring development progress

Lack of resources to support implementation by the ultimate duty bearer – the government

Diakonia will support dialogue on taxation and resource mobilization policy

Reduction of poverty at the household level through skills training which have led to gainful employment and enterprise development

The training courses have not been well linked to the market demands.

Diakonia will promote market driven skills training programmes.

Diakonia has supported development of community and other local structures e.g. Boards, Community education committees

Because of high turnover of members, capacity building has been a constant need.

Diakonia will adopt a more sustainable approach whereby focus will be on institutional development.

Diakonia has supported promotion of gender as a concern that is directly linked to democracy and development. This has resulted in increased number and acceptance of women leaders in positions of leadership

Although there is an increase, the numbers are still low.

The programme will deliberately target promotion of women participation in all programme areas.

The analysis demonstrates the high up-take of recommendations and the positive response to the

phase 3 evaluation which reshaped the programme. Further analysis in the findings section that

follows shows the benefits of these changes.

17


18


4. Findings

This section analyses the key findings from the evaluation exercise following the DAC criteria of

impact, relevance, effectiveness, efficiency and sustainability. In addition to this, the evaluation also

looked at the programme design process, partnership arrangements between Diakonia and the

partner organisations and lastly highlights the key successes and challenges in each of the result

areas that the programme was focussing on.

4.1 Programme design

In assessing the programme design and strategy adopted by the programme it is important to

mention that majority of the implementing partners like Kaalo Aid, SDO, YES, SFS, WAWA were not

part of the programme design. WAWA for example stated that they submitted an integrated

proposal to Diakonia which according to them was addressing the community needs while both

Kaalo and SFS mentioned that they participated only in reviewing Diakonia’s strategic document for

Somalia. It was the respondent’s opinion that this programme was to a large extent designed

centrally by Diakonia staff with most of the key local partners having little role to play in project

design. The partners highlighted that this phase was different compared to the other programme

phases where each organisation selected thematic areas and subsequently prepared a proposal.

Although the partners have indicated that they not been involved in the process of developing the

programme, it should be noted that they took part in four separate meetings – strategic planning

(which fed into the Phase IV programme development), three other partner consultation meeting

directly related to the development and refinement of the programme. However, what can be learnt

from this is that the partners felt that the consultation was not enough, therefore, participation

through-out all the programme development stages would be required in the future. In addition, the

initial draft of the programme and the final draft of the programme differed greatly in terms of

design – while the initial draft was based on core funding (where partner write proposals and are

funded according to their choice of activities), the final draft had pre-determined activities for

partners to implement. This was as a result of requirements by the donor. This change in design was

not readily welcomed by the partners.

4.2 Partnership with Diakonia

The implementing partners were of the opinion that the success of the previous and the current

program implementation was associated with a close working relationship characterised by a long

term nature of the engagement which has fostered closer collaboration, cooperation, coordination

and ownership. This has boosted the level of information sharing among the organisations which as a

result has enhanced confidence and trust. The partners pointed out to the financial and technical

support provided by Diakonia as particularly key to the achievement of the programme outcomes.

However, the partners indicated that in future we should put both program and institutional capacity

on the same footing as institutional capacity will have a huge bearing on programmatic success. They

felt that more support had been channelled towards the programmatic activities with lesser support

directed towards their internal institutional strengthening.

19


During the partners meeting there was consensus that Diakonia officers have been assessing the

capacity of the partners and ascertain that the above mentioned trainings were tailored to match

the needs identified. The partner organisations acknowledged that their capacity to engage,

encourage and educate the local citizens has been significantly increased as a result. For future

interventions, they recommended that Diakonia should focus more on partner programmatic needs

in relation to M&E / ToC, report writing, strategic planning, proposal writing, SIDA / Diakonia

guidelines as the organisations felt they need more support in these areas. They also suggested that

partner exchange visits should also be considered to enhance the cross learning experiences.

4.3 Relevance

Over the years, Diakonia’s presence and initiatives within the Puntland region have been very timely,

highly needed and have matched both Government’s and citizen’s priorities. Their long term

approach has made them a popular and much liked development partner. One of the partners

interviewed pointed out how Diakonia has been working in the country since the 90’ and how their

programmes and interventions were relevant as they factored in the contextual situation and

focused on the most pressing needs.

The work undertaken had a good fit with the policies and projects undertaken by government and as

a result there has been no friction between the two but more of a collaborative partnership working

together for the betterment of the common citizen. The partnership with the ministry of education

has especially been very successful and has realised a lot of positive outcomes. That said, with SIDA

shifting focus from the education sector there is a lot of concern especially by the partners who have

been working together with Diakonia in this sector on how funding needs will be managed going

forwards.

4.4 Portfolio performance - Activity completion

This section looked at the completion rates for the set activities as per the results framework. The

overall conclusion is that the programme has performed very well at both the output and outcome

levels especially bearing in mind the initial delay in contract signing and funds disbursement as well

as the fact that the programme is yet to get to full conclusion with 5 months to go.

The Achievement Rating Scale (ARS) analysis below uses the log-frame structure to summarise

performance, scoring at each level between 1 (target achieved) and 5 (no achievement). This was

based on the indicators and targets set. The overall rating was 2 (Very good), this was based on

completion of set activities and achieving the expected outputs and outcomes. That said, there is

“From 1994 to 1997 – Diakonia focused on emergency initiatives; 1998 to 2002 – A lot of

the focus was on recovery initiatives. The focus then shifted from recovery to education

and this was informed by a survey which revealed the need to focus on this key sector.

Some of the successes from this has been the rehabilitation of education institutions as

well as setting up of new ones such as GTEC and PSU. Diakonia got a thumbs up from the

president of Puntland for the great work it has done especially in the education sector.”

20


still more that can be done before we can attain a perfect rating score and this was agreed upon by

all the stakeholders interviewed. It should be noted that during the period of implementation of the

programme, the standard and expectations of rigour around indicators and evidence have increased

dramatically. This assessment is against this higher standard.

Log-frame extracts Targets ARS

Goal: Promote democracy, human rights and

gender equality in Sool, Sanaag, Puntland and

Galmudug.

• Increased literacy rates in Sool, Sanaag,

Puntland and Galmudug

• Increased community awareness on human

rights, gender issues and equality

• Improved good governance and democracy in

Sool, Sanaag, Puntland and Galmudug

2

Purpose: To empower the community

through the civil society to qualitatively

participate in the democratisation process in

Sool, Sanaag, Puntland and Galmudug

• 3600 adult learners have acquired functional

literacy and numeracy

• 2000 youth and women have acquired

vocational skills training

• 60% of trained youth and women acquire

formal and self-employment for improved

livelihood

• At least one existing policy implementation

plan in place every year in the Ministries of

Planning, Education and Gender

• Key Local Partners produce 5 thematic context

documents in the areas of peace and conflict,

gender, democracy, social economic justice and

human rights.

• 3 regions in Puntland, produce Regional

Development Plans

2

Re

sult

s

1. Local partners influencing the rights holders and duty bearers in adopting and applying the

principles, practice and values of good citizenship and governance;

2. Local partner promoting peaceful co-existence;

3. Empowered community (both genders) participating in the democratisation process in Sool,

Sanaag, Puntland and Galmudug;

4. Local partners effectively engaging in the democratisation process and advancement of

human rights;

5. Improved programme development for Diakonia.

2

1

2

2

2

4.5 Efficiency

On the cost ratio of inputs to outputs the programme has performed very well as compared

especially to regional benchmarks. The underlying strategy for change and especially the first two

steps – awareness creation and helping them to organise themselves – has made a significant

21


difference in encouraging the communities to take the lead and as such work to make sure change is

not only realised but will also be sustainable.

On budget use the absorption rate currently stands at 85% although there was some funds brought

over from the previous financial year thus the average rate would be around 75 %. The variance was

initially high at 52 % and this was attributed to the initial delays in disbursements which led to loss of

a 6 months period. As per the period of this evaluation, we still have 5 months left before the

conclusion of the programme so some funds are still yet to be utilised.

The percentages of administrative costs as compared to the overall budget stood at 5% which is an

acceptable level benchmarked on the general international standards of 10% and the fundraising

costs stood at 2 %.

One of the challenges experience as highlighted earlier was the long period it took before the funds

were disbursed. The contract signing took place in December 2011 but the start period got back

tracked to March 2011 which led to a time loss of around 7 months which ended up creating a

budget variance of around 52%. That said, the partners in collaboration with Diakonia have managed

to catch up and hit the targets as was expressed in the case of the girls receiving scholarships. In

future, it was suggested that programme and donor financial years be synchronised to ease both

funds disbursement as well as the programmatic reporting.

4.6 Sustainability

In this area the evaluation was assessing the likelihood that the benefits stream generated by the

interventions will continue once the support from Diakonia is phased out. The realised and

anticipated results so far will in all likelihood be resilient to risks beyond the life of this project and

the main reason behind that has to do with the change strategy adopted which has not only

transferred knowledge to the partners and communities but has also encouraged them to organise

themselves and actively participate in the various interventions. This has brought with it a sense of

ownership which will go beyond the life of this programme.

That said, the financial dependency on Diakonia is quite high amongst the partners and this has been

evidenced by the huge concern brought about by SIDA’s planned exit from focussing on the

education sector. Whereas the partner’s concern is genuine and justifiable there is little action in

the form of resource diversification. One of the biggest concerns raised by the partners was that due

to their local nature it is at times difficult for them to raise funds from international NGO’s as there is

always some worry on their internal capacity. Going forwards, even as Diakonia looks for other

sources of revenue to be able to continue their work in especially the education sector, Diakonia

could exploit other mechanisms such as linking and acting as referees for the partners with other

INGO’s as well as increasing their fundraising skill sets and capabilities to mitigate against these

shocks.

In addition to this, an exit strategy developed by partnering stakeholders will go a long way into

prompting them into purposeful action as well as preparing partners for eventual phase out.

22


4.7 Innovation, replication and scaling up

A number of innovative approaches was adopted by the programme partners in the activities they

were to undertake. One of those included the use of poetry to pass on messages as well as utilising

mass media to generate debate on key issues. The programme has also been collaborating with

universities in Kenya & Uganda and this has strengthened their capacity. The sports for peace

initiative which brought together two separate entities in Galkacyo to deliberate on security issues

also entailed great creativity and has resulted in increased levels of tolerance amongst different

communities.

One of the unexpected results was the formation of committees to deal with issues such as

insecurity within the community. Whereas organising community groups forms part of Diakonia’s

change strategy, this approach was driven largely by the citizens and has yielded great results. The

process added value through increased participation and ownership as well as being cost effective

and from the results seen so far there is talk of having this approach replicated and scaled up in

other areas such as the south central region where Diakonia will venturing into in the new phase.

Due to Diakonia’s relatively small size, the total impact it can have by drawing on its own resources is

likely to be limited. It should therefore aim to increase the outreach of its activities by playing a

catalytic role seeking to influence other development partners to replicate and scale up the already

successful initiatives. One such opportunity could be with the “Act Alliance” that Diakonia is already

a member of.

4.8 Result areas - Outcomes

This phase of the programme focused on five thematic results:

Result 1: Local partners influencing the rights holders and duty bearers in adopting and applying the principles, practice and values of good citizenship and governance

Result 2: Local partners promoting peaceful co-existence

Result 3: Empowered community (both genders) participating in the democratization process in Somalia Result 4: Local partners effectively engaging in the democratisation process and advancement of

human rights,

Result 5: Improved programme development for Diakonia.

Diakonia has been working through local partners who carry out the project activities in order to

achieve the above results. Consequently each organisation had been assigned specific themes and

underlined a list of activities to undertake in the period under review. As in all such work, the

findings provide the basis for recommendations which follow. The main purpose of this section is

therefore to review progress towards the outcomes objectives achieved by the implementing

partners contracted by Diakonia. In this regard, the evaluation assessed the impact, efficiency,

effectiveness and sustainability of the Programme; progress towards achieving its objectives and

23


outcomes, identified strengths and weaknesses in the Programme design and implementation, and

provided specific recommendations per thematic area on sustaining implementation of the activities

initiated within the programme.

The current programme implementing partners and the subjects of this evaluation were: Kaalo

Relief and Development Organisation, Somalia Family Services (SFS), Youth Employment Summit –

Somalia (YES Somalia), Samo Development Organisation (SDO), Puntland State University (PSU), We

Are Women Activists (WAWA), Forum for African Women Educationists – Somalia Chapter

(FAWESOM), Garowe Teacher Education College (GTEC), Gardho Women Network (Garwonet), Ilsan

Women Coalition, Abyan Women Centre and Female Education Centres Network (FECNET).

Theme: Democracy and Human rights

Result One: Local partners influencing the rights holders

and duty bearers in adopting and applying the principles,

practice and values of good citizenship and governance

The ratification of the Puntland state constitution in June

2009 and it's consequent adoption in April 18, 2012 by

the Puntland Constituent Assembly has been viewed as

one of the remarkable achievements of the five year

Puntland government. Other realizations has been the

development of necessary state laws, institutionalization

of regulations and commissions to advance the

democratization process as well as the establishment of

the Transitional Puntland Electoral Commission (TPEC) in

July, 2011. In line with these democratic provisions there has been some evident changes and

transformation on both rights holders and duty bearers in upholding and promoting good citizenship

and governance. The evaluation established that this particular programme component has indeed

been designed and anchored in line with the national priorities of the Puntland government and the

programme design and strategy has effectively invested resources around most of these priority

areas such as; Constitution Implementation Dialogues, Government Budgeting and Spending;

Taxation and Resource Mobilisation, Consultative Forums between CSO and the Puntland

Government and Voter education awareness campaign.

This realization can be attributed to the needs assessment undertaken by Diakonia which ensured

that the action took into account local context considerations in the design and planning stage. Thus

the action was able to achieve relevance as both the community and the government expressed

their need for it in addition it was able to adopt a local driven strategy from the inception fostering

ownership. The activities in this programme component were undertaken by Somali Family Services

(SFS) and Kaalo Relief and Development Organisation (Kaalo).

To foster security and better rule of law in Puntland, the programme undertook to improve the

relations between the police and the community by conducting dialogue forums between the

community and the police. Findings indicate that there are some improved levels of integration

An example of the citizens working

in partnership with the police was in

the case where the killer of Sheikh

Abdulkadir was apprehended by the

citizens and handed over to the

police

SFS and Kaalo have so far facilitated

6 forums which have increased the

interaction and trust levels between

the police and the citizens

24


between the police and the community such as the police being friendlier to the community now

than before the intervention. During the field interviews it was evident that this programme has

resulted in increased interaction and higher trust levels between the citizens and the police. These

forums have been able to foster linkages between the community and the police as a result

community led neighbourhood watchdog committees have been formed and operationalized which

are instrumental in assisting the police in ensuring security (The committees have equal participation

amongst the different genders and youth and as a result this has increased the sense of ownership

and participation). The committees have been involved in

undertaking community awareness initiatives in their

localities in a bid to build trust with the police as well

enhance community policing.

Another key output of this component has been the

creation of a police hotline number which has been

advertised in public realm to ensure the public know about

it and are encouraged to use it. The hotline has been

particularly effective in reducing crime as the community is

now able to report illegal activities in their neighbourhoods.

The Diakonia partners were able to use some innovative

approaches such as art, drama, radio, billboards and

television to sensitize the community on good citizenship and governance. The partners in this

component reported recognising the role played by the media and exploited the potential inherent

in enrolling it as a partner in their campaign. The genesis of a variety of media- especially the

flourishing FM radio stations were such a boon to the campaigns with the messages packaged for

effective delivery via local radio.

Other notable finding positively contributing to the achievement of program result is that Kaalo Aid

and SFS were able to pick up an advocacy issue in an attempt to address community and civil society

concerns of feeling excluded in the budgeting processes within the government. Both CSOs and the

community raised concerns relating to budget transparency, expenditure accountability and

inclusion in budget prioritization. Mr. Mahamoud Ali, the

Diakonia programme officer in-charge commented that

partner and community knowledge on tax, budget

planning and budget utilisation has increased significantly

which resulted to the setting up of a committee (in

Bosaso) to act as a pressure group to the local authorities

to ensure budget spending is done based on local

priorities. There are plans to have this replicated

elsewhere within Puntland.

During the FGDs both partners and community shared

similar views and ascertained that there has been an increased and better understanding by CSO’s

and the community at large on the importance of revenue collection, budget planning, spending,

tracking, transparency and accountability. This led to CSO’s organising themselves to establish a

“Events are aired through topical

discussion programmes at prime

time and in the announcement

sections which is an added

advantage in that the message

reached the intended audience

without any alterations” – Lul Jama,

community member.

On budget planning, spending and

tracking, the level of transparency

and inclusion has risen as evidenced

by the recommendations from the

CSO’s being prioritised – there was

increased budgets allocated to

social affairs and infrastructure

after the lobbying and advocacy

initiatives.

25


committee whose role is to ensure the budget is planned for and utilised according to the citizen’s

priorities.

The evaluation ascertained that Kaalo Aid has specifically been instrumental in promoting informal

peace and civic education among the community. The organisation effectively identified poetry and

sports as the main avenues for passing on messages of peace and civic education. Major outputs

under this has been the identification of 48 poets from the districts in Puntland who recited poems

that relate to peace and civic education at a major peace event organized that resulted in

development of a booklet “Barashada Madaniga iyo Nabadda” - Peace And Civic Education Booklet

which was later distributed to 400 participants. The action was also able to utilize sports for peace

by organizing football matches where an estimated 3,000 people attended and received documents

with peace and civic education messages.

The programme activities have also been able to promote democracy and human rights as well

enhance peace and reconciliation especially in Galkacyo where both North and South were united by

poetry and the event ended up receiving a lot of media attention especially generating a lot of

debate on radio. The evaluation concludes that this methodology illustrated great creativity and

innovation and going forwards the partners should be encouraged to experiment and develop new

approaches which can greatly increase the general effectiveness of the interventions.

Theme: Peace & Conflict and Human Rights

Result 2: Local partners promoting peaceful co-existence

The activities under this result area were implemented by two organisations; Youth Employment

Summit – Somalia (YES Somalia) and Samo Development Organisation (SDO). SDO focussed on

implementing 'Do No Harm' workshops and provide support in the commemoration of International

days that enhances promotion of democracy and human rights while YES implemented

establishment of peace clubs in schools to enhance peaceful co-existence. The myriad of activities

undertaken towards delivering this result have been lauded for their effectiveness. The study found

out that the 28 peace clubs formed in schools were active and collaborating and were organizing

morning assemblies in which they disseminate peace messages to the students who further take the

messages to the community members.

The clubs are also able to identify and implement peaceful solutions for resolving conflicts as well

preventing crime in their communities by collaborating with Community Education Committees. In

addition, the clubs have been advocating peace related messages such as the respect for diversity,

the ills of negative media, corruption and environmental degradation and ways community can deal

with these problems. The students interviewed stated that before the formation of peace clubs all

student conflicts were reported to the school administration but now the peace clubs handles all the

cases acting as mediators which has proved to be a more effective way in resolving conflicts. The

relationship between teachers and students was also usually unfriendly and there were some

reported cases of students stabbing their teachers or attacking them after school hours as well as

boys intimidating girls who ended up feeling unsafe in schools.

26


This intervention has been able to address such cases by addressing conflict in school spaces and

therefore enhancing general tolerance and peace. The evaluation observed that one of the clubs

formed were active and functional and that YES was able to provide capacity building and training on

leadership, peace and conflict resolutions to not only students but extended it to teachers, parents,

school staff and Community Education Committees. It was clear that there is improved co-existence

and integration between the community, among the students and between teachers and students.

The students interviewed seemed enthusiastic about the peace clubs and findings suggest that this is

a sustainable system that has been institutionalised which will/should continue even after YES’s

support is withdrawn. To ensure sustainability, YES set up a

sustainable programme frame work during implementation

phase. The organisation suggested mainstreaming of peace

education in the school's curriculum as well increasing

coverage to access more schools with similar intervention.

The evaluation noted that the action has positively influenced

the students by providing them with an opportunity to act as

leaders, be confident and make informed choices as well as

keeping them occupied to avoid them resorting to drug abuse

and violence or be lured into piracy. All respondents indicated

that there now exists a marked improvement in human rights

and conflict resolution capacities among CSO’s, the community

and schools.

The focus on peace clubs is deemed relevant and timely especially because it targets schools in

Puntland which constantly face high crime rates that tends to spill over to the communities. Our

findings show there has been remarkable improvements in peace and conflict resolutions in the

schools targeted and the surrounding community. Peace education given to children in their

formative stage has proved to be the building block upon which they can transfer the skills and

knowledge acquired in school down to their parents and peers.

“Do no harm”

In line with this result has been the Do No Harm workshops conducted by SDO that targeted local

organisations operating in humanitarian and aid development programs to help them understand

how aid development interventions may negatively or positively affect conflict settings, and to

generate some ideas of how to use this experience in future project planning. In this regard,

according to SDO, 1000 Do No Harm booklets in Somali version were distributed to the trainees to

use as a tool in the field and to apply the principle into their programs. The evaluation points out to

SDO's ability to also conduct a peace dialogue between CSOs of North Galkacyo - Puntland and South

Galkacyo - Galmudug as a major achievement owing to the difficulties that exists in bringing

together groups from these two conflict zones especially because it is under two different

administrations and there exists a lot of tension.

“After the do no harm training, we

have been linked with civil societies

from Galmudug to think

strategically of ways to connect the

two villages. Working in a conflict

prone area is hard but the do no

harm training has positively

changed our outlook and way of

thinking.”

Said of Disability Centre.

27


The production and dissemination of 10,000 copies of the Somali version of Do No Harm booklet

among CSO's enhanced effectiveness as it assisted members in the implementation of conflict

sensitive programme over a longer period of time of engagement. The evaluation sees this strategy

as having set up new standards among CSOs operating in conflict and complex humanitarian settings

like Somalia where the line between aid and conflict has often been thin. The organisation asserted

that the booklets are currently in high demand among the local and international organisations

working in Somalia as they are being used for training of local organisations and the community at

large.

A key challenge has been the fact that many organisations who attended Do No Harm workshops did

not have the capacity to incorporate the Do No Harm approach in their programming. The

evaluation suggests a follow up by both Diakonia and SDO to see how they can further strengthen

this aspect and possibly include as a critical need in future program interventions. Further

observations show that relevance of this project has been more on the thematic, programme and

technical level. Our assessment confirms that the project accordingly scores highly on relevance in

terms of the thematic focus on peace clubs, conflict resolution and changing traditional way of

implementing projects. Among the most significant changes is that the people of Mudug region in

both Puntland and Galmudug have started to speak about peace and now there exists better

chances of reconciliation as communication channels and integration between the CSOs in Puntland

and Galmudug is functional.

Theme: Social Justice and Gender

Result 3: Empowered community (both genders) participating in the democratisation process in Sool,

Sanaag, Puntland and Galmudug

To improve the quality of education as well as ensure teachers in Somalia can access higher

education Garowe Teacher Education College (GTEC) has been implementing a pre-service teacher

education program to strengthen primary and secondary education. The activity is commendable

especially because it adopted gender responsiveness strategies by specifically targeting the

participation of women teachers in higher education. In the evaluation GTEC stated that they

successfully advocated for more female members to be elected to the Board of Governors (BOG) of

the college and in this regard 40 % of BOG are now women including the chairperson of the board.

They also undertook some activities that supported girls secondary school education to help them

access boarding schools. However, one of the downsides as pertains the intervention with GTEC has

been the over-reliance on the financial support provided by Diakonia. Diakonia has been providing

core funding to GTEC and with the move away from the focus on the education sector this is likely to

have huge implications for the likes of GTEC who have not diversified their funding sources.

''I joined GTEC out of necessity after I noted there were few female teachers in schools where 70%

of the teachers were male. I graduated out of the program, now I support female girls come into

terms with adolescence through counselling'' – Zahra (FGD respondent)

28


In Darwish school, visited during the evaluation, the number of female teachers was found to be at

16 compared to male teachers which was at 19. While the number of girls in schools was found to be

at 46% compared to 54% among boys. This findings point out that there is a gradual reduction in the

gap between male and female teachers and directly correlated the difference between boys and

girls in schools as a result of the intervention. The schools are also able to retain the girls in school

more than before due to the increased numbers of female teachers who provide support to girls and

encourage them to stay in school. Mainstreaming gender in the curriculum as well as advocating

against female genital mutilation has also played a significant role in reducing turnover. As a result

provision of teachers especially to Primary and secondary schools has led to an improved quality of

education for the community.

Still on the same wavelength of support to education institutions, Diakonia has been working closely

with Puntland State University. who not only provide quality education have also able to mainstream

human rights education, democratization and social change in its faculty courses and provided civic

education to community especially relating to the constitutional and democratization processes. The

evaluation shows that improvement in quality of education through setting up of institutions like

GTEC and PSU is having a huge role to play in the current and future democratization process of

Puntland.

Another initiative kick-started by Diakonia is the provision of scholarships to girls from especially the

less fortunate families. The investment in the education of girls came at a time when there were few

girls attending school. Majority of the girls who have accessed the scholarships are either orphans,

IDP’s or those from very poor families. This initiative has been spearheaded by Forum for Africa

Women Educationalists – Somalia Chapter (FAWESOM).

To evaluate the outcomes of this initiative to girl education, we

visited a school where 50 girls were supported by the program

and coincidentally discovered among the girls supported two

had epilepsy. When interviewed the girls indicated that female

teachers provided support to the girls more compared to male

teachers and also gave more care to the two epileptic

students. The findings draws a correlation between the

increase in the number of female teachers in schools to an

improvement in retention of girls in schools as a result of

provision of scholarships to girls, support to adolescent girls to cope with maturity and

mainstreaming of gender in the curriculum which has more empowered girls to stay in schools and

not drop out due to early marriage.

Still under this result area, We are women Activists (WAWA) was assigned to develop a human rights

manual and train both teachers and civil society organisations so that they would apply and

mainstream human rights principles in both school syllabus and programmes implemented by civil

society organisation. It was clear during the evaluation exercise that WAWA is still in the output level

as they indicated that no significant change attributable to the programme has been felt. The only

notable achievement is that WAWA has been able to produce a Human Rights manual which has

“On scholarships, the target number

of recipients was not achieved in

year 1 due to late funds

disbursements but this was made

up in the consecutive years – we

had 300 in year1 but for year 2 and

3 we had 1250” - FAWESOM

29


been approved by the Ministry of Education and other

stakeholders. The tangible changes will most probably be

realized when the other remaining activities are carried

out. Worth noting, WAWA was supposed to train teachers

and civil society members but this had not been done by

the time of this evaluation.

The reason for this was that the development of manual

took longer than expected. WAWA hired an external

consultant to develop the manual, paid 40% of the budget

allocated for manual development, but they were

dissatisfied with the product produced so they had to

recruit a new consultant and this took longer than

expected. It was only until in late February 2013 that the

manual received validation. Consequently, training 100

teachers on use of the HR manual as well training CSOs as

above could not be realized since they depended on the

availability of the manual. At the time of the evaluation WAWA was in the process of holding

workshops with the teachers and CSOs since the production of the manual had been complete. As a

lesson learnt, WAWA underlined the importance of recruiting local consultants on sensitive issues

like HR. This they suggested after the manual produced has been designed by local consultant who

performed equally well if not better than external consultants as local consultants have better

understanding of the local scenario and context and culture as well.

WAWA intends to widely circulate the human right manual through the different HR forums that

WAWA holds from time to time. They expect it to be a resource manual for future HR trainings in

schools and as a source of reference for CSOs in addressing cultural and cross cutting issues that face

the Somali society particularly in Puntland. To further empower the communities the intervention

was able to support several centres with vocational and literacy training among them; Gardho

Women Network (Garwonet), Ilsan Women Coalition (Ilsan), Abyan Women Centre (Abyan), and

Female Education Centres Network (FECNET). The key informants interviewed clearly felt that the

project had helped increase adult literacy, provided youth employment and generally improved the

status of the community.

This multi-pronged approach

adopted by Diakonia was judged by

the evaluation as having been both

relevant as well as very effective as

it not only worked to give access to

education to the girls but also

incorporated a measure

(Increasing/encouraging the

number of female teachers in

schools) to ensure there was

retention throughout the schooling

period and will go a long way into

bringing about sustainability.

Statistics

Human rights manual was developed and training 100 teachers from primary

schools on human rights has already taken place;

Quality teacher education training provided to 485 trainees;

School fees for 900 girls/ year accomplished (Target number of recipients not

achieved in the first year but this was made up in the consecutive years (300 in

year 1 but 1250 in year 2 & 3). This has increased the girls access to education

and with the additional free services such as reference materials as well as the

quality teacher education training, this has increased education performance

levels;

drop-out rates reduced by more than 50%;

20 districts in Puntland are developing strategic plans on education which will

be tied to the MoE’s education plans and initiatives;

30


Result 4: Local partners effectively engaging in the

democratisation process and advancement of human

rights.

In order to effectively engage, the intention in this

result area was to enhance the capacity of the partner

organisations by providing them with requisite skills

to be able to carry out their functions effectively.

Technical assistance and training was provided in the

following areas.

Continuous Monitoring: Through the continuous monitoring of the partners, the programme staff

provided technical support to partners by jointly developing approached to be used in the

implementation of the programme. This was instrumental in ensuring that the tight timeframes were

met and that not only the rights things were done, but that they were done in the right way.

.

Audit Training: The finance staff took part in the training of partner organisations in audit issues. This

training was geared towards reading and understanding audit report, preparation of management

letters and responding to management issues, assets registration and disposal, financial reporting

planning and the role of the Board of Governors in audit. This training has provided made the local

partner organisations much more credible to other potential donors, giving them a better chance at

external fund raising.

Tax Justice Advocacy Training: This training was geared towards enlightening the civil society on tax

issues and the role of tax on development. The training sparked off a huge interest after the

realisation that resource mobilisation and utilisation are at the core of the stability of a state.

Lobbying and Advocacy: this training was offered by a Diakonia staff member after receiving similar

training in Nairobi. The training centred on how to craft and advocacy agenda and implement it. The

training has led the partners to advise the local communities on how to mobilise themselves and

advocate for their issues. The community policing committees and the accountability committees on

tax are spinoffs from this work.

Emerging issue – One of the

programme staff based in the

Puntland office, was not performing

as per the requirements of the

organisation and as a result the

contract for his services was

terminated. The activities he was

responsible for (result 2 and 4) were

distributed to the rest of the

programme staff. These activities

are expected to be completed in

time albeit they will be taking place

much later than the anticipated

time of completion.

31


Reporting Training: The reporting training was carried out by the programme staff after receiving the

training in Nairobi. The training centred on the formulation of goals, objectives, results and indicators.

In addition, the actualisation of the strategy for change within the programme, with a view to enable

partners and subsequently Diakonia report better on the results of the programme. The outcome of

this training will be seen in the next reporting period.

Ministry of Education Training for the Inspectorate Department: The proposed support to the MoE

was amended to become training for the inspectorate department. This activity is not completed yet.

Review of 5 year plan for Ministry of Women Development and Family Affairs (MoWDaFA): With

the help of WAWA, MoWDaFA is currently reviewing its 5year plan. This activity is not completed yet.

Capacity Bulding for Women Umbrella Groups: The activity was geared towards assisting umbrella

groups realise value addition for their member organisations. An assessment carried out with 2013

indicated that resource mobilisation was a key challenge for umbrella organisations. This led to the

planning of a resource mobilisation training for the umbrella organisations. The training has not been

carried out yet – it is envisaged to be implemented in 2 phases in December 2013 and January 2014.

Emerging Issue: The resource mobilisation training will include the education partners that

are being phased out. This is to allow the partners mobilise funds from other donors in order

to cater for the shortfall that will be felt with the exit of Diakonia from their funding

portfolios.

Partner Accompaniment Process: This is a capacity development for partners in the areas of human

rights, good governance, peace & conflict and media development. The programme has placed

resource persons within partners organisations to assist them in the development of these areas. For

the human rights, good governance and peace & conflict

were placed in October 2013 and are expected to complete

their assignment in March 2014. The media expert was

posted in Garowe in May 2013 and will complete the

assignment in December 2013.

The media expert has been able to accomplish the following;

Training the media on election reporting, in

preparation for the aborted local council elections in

Puntland in July 2013. The training was carried out in

conjunction with Media Association of Puntland

(MAP) and InterPeace (an international organisation).

Since the elections were not carried out, the

outcome of this training has not been measured.

However, it wil crucial to see the reporting of the

January 8th presidential selection.

Development of a code of conduct for journalists – a separate training was done for

journalists under MAP on the development of a code of conduct for elections reporting. This

was carried out and also led to the development of a general code of conduct for journalist in

everyday work. The code was developed, accepted and signed by journalists.

Emerging issue - The activities

related to government ministries

face long and bureaucratic

processes that has led them to take

longer than expected. An example

of this was planned work between

MoWDAFA and WAWA. Despite the

issues being sorted out after lengthy

negotiations between Diakonia and

the ministry, the strategy for

engagement might need to be

revised.

32


Review of the Puntland Media Law – the resource person was able to bridge the lack of trust

between government (Ministry on Information) and MAP, leading to an agreement between

the parties on the process of review of the Puntland media law. Later, the law was reviewed

and drafted with both parties accepting it. On 4th December 2013, the law was approved by

the Cabinet and will be presented to the newly selected Parliament for enactment into law.

The resource person has assisted MAP and government to realise the formation of a media

council consisting of both members from the government, media and private sector.

Although the media council has not been formed (it can only be formed after the media law is

enacted), discussions are underway on its constitution.

The programme is looking at assisting MAP in improving its internal structures and to develop

a coherent strategic plan, in line with the Somali Compact and the needs of Puntland. This is

expected to be completed between January and February 2014.

Result 5

This result areas focussed on training provided to the Diakonia staff in order to enable them carry out

their work more efficiently and effectively. The following activities took place during phase 4 of the

programme:

One staff of the programme attended environmental mainstreaming training in Nairobi in

April 2011, in which aspects of how the environment affects and is affected by the

programme were explored, using an “environmental lens”.

Two staff attended leadership training in Nairobi that looked into aspects of proper

leadership and management of staff and programmes

Two staff attended an anti-corruption training in Nairobi alongside other Swedish

organisations, to tackle the issues of corruption within non-governmental organisations

Two staff attended a resource mobilization training in Nairobi, which was the second part of

an earlier training offered in Johannesburg. The training looked at the current and future

resource mobilization trends as well as how to strategically position Diakonia to access funds

from various donors. This training is in line with Diakonia’s global quest to diversify its

funding.

One staff attended the Accountability Framework Training in Uganda in Jan 2013, which has

led to the development of a complaints and responses mechanism at the country level. This

will also be rolled out to partners, so as to improve their accountability to the right holders.

2 staff attended the ATHA Core Training on Humanitarian Action in Harnosand, Sweden. The

training was crucial in allowing Diakonia access SIDA’s humanitarian frame funds (and also

other humanitarian donor funds) for Somalia. In the past, it has been noted that if Diakonia

did not respond to emergencies in the areas where it works, then it loses credibility with the

beneficiaries and therefore finds it hard to implement long term interventions in those

areas. With an improved capacity of staff to develop and implement humanitarian projects,

this potential risk can be mitigated.

Finance staff at the Regional office has attending various trainings that include project

management, corporate governance and NGO post-budget briefings. This is over and above

the Institute of Certified Public Accountants – Kenya (ICPAK) professional trainings.

33


Whereas it’s commendable that the above trainings took place, the outcomes of the training are not

very clear and it will be important going forwards to have some baseline information that will enable

us to pinpoint the situation prior to training and the subsequent changes after the training has taken

place. Another issue was the lack of mechanisms for assessing the extent of knowledge transfer from

the training recipient to the rest of the staff as well as the accrued benefits.

4.9 Diakonia as the facilitator

During the programme period, the work of one of the staff was found wanting. This was the

staff in charge of the result 2 and 4 of the programme. Despite this hitch, result 2 progressed

well because the partner organisations took initiative to push the activities forward.

However, on result 4 which was under the direct implementation of that staff member, little

traction was made. The contract for that individual was terminated in August 2013 and the

activities allocated to the 2 other programme officers. Already, the re-allocation of the

activities has borne fruit with all the activities already near completion. Diakonia will be

looking at a skills assessment of the country programme staff, as part of the internal review

towards becoming a Model 25 country programme.

Financial reporting especially to the donor was slightly problematic, more specifically on

having real time knowledge on funds utilisation (absorption and spending). It is expected

that the new financial and programme handling systems being rolled out in January 2014

will be able to resolve this.

5 In Diakonia, a Model 2 Country Programme operates semi-autonomously with a some-what higher

delegation in decision making.

34


5. Conclusions and recommendations

The analysis in outcomes and effectiveness sections above shows that the programme has an

impressive list of achievement to its credit.

The partnership but it brought together the right partners. The strategy contained important and

pertinent themes which delivered quality support that made a difference. The credibility of the

Diakonia brand and the in-country staff meant that relationships of trust were established.

Ownership, relationships and support were all done well. A model has been established – and

proven – and that is a major achievement and product of the programme. This is a work in progress

and will continue to be improved as we continue to learn and adapt to different micro and macro

conditions.

Progress has stemmed from governance and accountability decisions made by government, local

authorities as well as citizens and non-governmental organisations. The strategy and

implementation approach also played a critical role in the success of the programme.

Given this conclusion, the next section deals with specific short-term recommendations /actions

that can be taken in the course of the next few months and the others to future programming.

5.1 Short term

The programme closes in a few months. In that time there is an opportunity to extract further

learning. Key areas of focus over the next few months could be on:

Ways to improve efficiency – this could focus on reviewing and revamping the systems

especially communication systems as well as the required skill-sets necessary to ensure

programme success.

It would also be productive to assess the use of the risk assessment tool, particularly in the

context of future programmes.

Plan on ways of build on the strengths identified in this report for the next programme.

Another short term goal could be to gather more ideas from the end beneficiaries on future

programming especially as we plan for the next phase

5.2 Future programming opportunities

Strategic approach - Whereas strategies for working directly with partner organisations are

key, future programmes will need to make explicit strategies for influencing the

environment of the partner organisations. One approach could be having causal, persuasive

and supportive strategies for dealing with the partners as well as their supportive

environment as shown below.

35


Design process – As earlier highlighted, the design process of this phase was not fully

participatory. Whilst some of the partners were consulted and actively participated through

brainstorming sessions, consultative meetings and workshop sessions other had a lesser role to

play and either submitted proposals or only participated in developing Diakonia’s strategic

document for Somalia. The partners expressed the opinion that Diakonia should consider them

during the designing of projects planning so that they can fully understand the design,

implementation and monitoring of the project in order to fully respond to the needs of the

community. This will achieve greater impacts and ensure the project sustainability. Another

area where the partners should be fully involved is on selection of thematic areas of focus as

they are more in touch with people in the grass-roots.

Programme management - This should be strengthened with more pro-active management

and thought leadership for this complex programme. Increased technical focus on capacity

development would be part of this. So, for example, each local partner would have a capacity

development plan, leading to their “graduation”. This has been done to some degree within the

life of this programme, with responsibility passing more to local training partners and materials

being customised to local contexts. But deepening this approach to be more explicit, longer-

term and to move beyond training is required for a more thoughtful and comprehensive plan

that includes building networks, Communities of Practice, that uses more techniques such

mentoring and coaching and exploits the technologies available for sharing of learning.

Funds disbursement - the most pressing issue was on timely disbursement of funds. This

was especially a big issue initially but that improved thereafter. One of the suggested remedy

for this would be to synchronise the programme and donor financial years in order to make

the process less cumbersome especially to the donors.

Another area of concern for the partners was what they describe as ‘a disconnect’ between

the funds allocated to them and the expected outcomes they were to deliver. With donors

tightening their budgets especially due to the economic recession, “value for money” and

“bang for the buck” are terms which have become increasingly common especially in the

international development arena. Whereas the expectations are not outrageous, it will be

important to communicate clearly with the partners on what’s really expected and

brainstorm together with each stakeholders on their roles and responsibilities with a view

Causal Persuasive Supportive

Individual Partner (I)

What will be done to produce an immediate output?

What capacities require to be built among the partners to create new thinking and new ways of doing things?

How will sustained support, guidance and mentoring be provided?

Environment of the Partner (E)

What will be done to change the physical and policy environment under which the Partners operate?

What will be done to create wider access to information services, media, dissemination, exposure, publications?

What networks, relationships will be established or utilised?

36


determining the best way forward. This will foster deeper collaborations and increase the

chances of attaining our collective outcomes and impact.

Internal communication - Communication is key to the smooth running of the programme

needs to be streamlined. Internally, there is a need to have proper communication

mechanisms with agreed timelines for responding to certain requests. Another aspect of

communication is the messages going out to the larger public – the programme must develop

a coherent communication strategy to ensure certain targeted messages are constantly being

disseminated to both the partners as well as the general Puntland citizens.

PM&E - Future programme require a stronger results-based, learning culture across

programmes with decisions based on good quality and time information. More specifically the

PM&E systems should include:

o Needs assessment to be completed before designing a country specific programme. The

design process should involve key stakeholders, partners and beneficiary target groups

o Needs assessment of partners to show progress in building capacity

o Expand focus of M&E from inputs and activities to outcomes and impact and support

this change across the programme ensuring staff have the competencies and incentives

to deliver. One simple example is to measure changed behaviour after training.

o Greater use of secondary data from surveys, national statistics, other programmes, the

media to understand the environment and to begin to analysis the contribution of the

programme to national changes.

o An explicit learning strategy and system for more focused and robust learning, with clear

division of responsibilities. All staff should be aware of their role in gathering, collating,

analysing, sharing and learning from the programme

Marketing and branding – With the anticipated / ongoing shift from “hardware” – construction

type interventions to the more “softer” democracy and governance type of work, the

programme’s and Diakonia’s visibility especially to the larger public is like to be affected.

Already, there are some beneficiaries and partners who have been asking whether Diakonia is

phasing out.

In order to curb this kind of talk and messages spreading further out, we will need to become

more intentional in our marketing and branding activities and develop a strategy to ensure

the public has top of mind awareness of the kind of work we are doing and that we have

every intention of remaining active in Puntland as well as moving further south in the next

programme. Some of the immediate activities could include encouraging the partners to

display the Diakonia logo on all materials they will be using as well as having pull up banners

displayed in these sessions. A fine example of this is done by USAID who ensure all materials

used are branded with their logo and the message “From the American people”.

Capacity building – Whereas programmatic capacity building is of great importance, in equal

measure Diakonia will need to factor in developing the capacities of the partner organisations

37


they will be working with. Despite the fact that this has already been taking place in the past

programmes, the partners feel they could use with additional support in developing their

internal capacities in areas such as strategic planning, proposal writing, M&E, report writing

and lobbying and advocacy. These skills will enhance long term sustainability and enable the

partners handle larger scope and scale of programmes.

Sustainability – The evaluation was of the opinion that the benefits stream generated by the

interventions undertaken by Diakonia and its partners are likely to continue even in the event

of the support getting phased out. That said, whilst some partners have undertaken measures

to guarantee their individual sustainability, some are still lagging behind and going forwards are

still expecting a lot of support from Diakonia and are lacking a “plan B” for revenue

diversification and generation.

To prevent this from getting out of hand, one suggestion would be to create exit plan with

the partners from the onset of the partnership arrangement. This exit plan would be tied to a

strategic plan underpinned by a robust theory of change. This could be a living document

which both stakeholders would review periodically to access the current state and assess

whether there is any need to change tact or course based on the contextual environment.

Another complementary intervention is for Diakonia to assist the partners find alternative

sources of funding through developing their capacities to fundraise and also acting as their

references when they have approached various donors or INGO’s. This will reduce partners’

dependence whilst ensuring achievements of the programme are sustained.

38


6. Annexes

39


Annex 1 - ToRs

TERMS OF REFERENCE – END OF TERM EVALUATION

DEMOCRACY AND HUMAN RIGHTS PROGRAMME – PHASE IV

An Integrated Approach to Enhance Democracy and Human Rights in Sool, Sanaag, Puntland and

Galmudug

1.0 INTRODUCTION

The SIDA-funded Democracy and Human Rights Programme – Phase IV, titled “An integrated

approach to enhance democracy and human rights in Sool, Sanaag, Puntland and Galmudug”, is a 3-

year programme that started in April 2011 through March 2014, implemented by Diakonia through

its partners in Somalia.

The programme has an overall objective of promoting democracy, human rights and gender equality

in North-East and Central-South Somalia; with the specific objective of empowering the community

through the civil society to qualitatively participate in the democratisation process in Sool, Sanaag,

Puntland and Galmudug.

The programme, which is in its fourth phase, has the following results;

1. Result 1: Local partners influencing the rights holders and duty bearers in adopting and

applying the principles, practice and values of good citizenship and governance,

2. Result 2: Local partner promoting peaceful co-existence,

3. Result 3: Empowered community (both genders) participating in the democratisation

process in Somalia,

4. Result 4: Local partners effectively engaging in the democratisation process and

advancement of human rights,

5. Result 5: Improved programme development for Diakonia.

This fourth phase of the programme builds up on the previous three phases of the programme,

which have focused on promoting democratic principles and values, culture, gender equality and

human rights by increasing access to formal and non-formal education and training, vocational skills

development, promotion of peace and civic education, awareness raising on emerging issues

including HIV/AIDS, female genital mutilation (FGM) and gender equality and strengthening the

capacity of civil society organizations and key local institutions involved in education and training,

human rights and democracy initiatives in North East and Central South Somalia.

The programme has been implemented through 12 local partner organizations, namely; Kaalo Relief

and Development Organisation, Somalia Family Services (SFS), Youth Employment Summit – Somalia

(YES Somalia), Samo Development Organisation (SDO), Puntland State University (PSU), We Are

Women Activists (WAWA), Forum for African Women Educationists – Somalia Chapter (FAWESOM),

Garowe Teacher Education College (GTEC), Gardho Women Network (Garwonet), Ilsan Women

40


Coalition, Abyan Women Organisaionr and Female Education Centres Network (FECNET), and

Ministry of Education (MoE) and Ministry of Women Development & Family Affairs (MoWDaFA).

2.0 PURPOSE OF THE END OF TERM EVALUATION

The evaluation is intended principally for learning purposes, accountability purposes and assist in

charting the future of the programme. It is expected to generate relevant findings, lessons and

recommendations that will guide and inform the future of the Somalia Country programme, in

relation to the new Swedish Results Strategy, and the Democracy and Human Rights programme, in

particular.

3.0 OBJECTIVES OF THE END OF TERM EVALUATION

The overall objective of the evaluation is to provide Sweden, Diakonia, local partners, stakeholders

and project beneficiaries with necessary data and information for making informed decisions

regarding the performance of the Democracy and Human Rights Phase IV programme in terms of its

efficiency, effectiveness, impact and sustainability, and also to take appropriate action on any

required changes in the programme design, development and implementation strategies for the

future.

4.0 SCOPE OF WORK

The end of term evaluation will cover the Phase IV period (April 2011 – March 2014) of the

programme with reference to the previous three phases. All the different project components and

activities stated in the original logical framework will be assessed. The standard criteria on

evaluation namely: relevance, efficiency, effectiveness, sustainability and impact will be used to

structure the evaluation. The evaluation will have to take into account that it takes place before the

end of Phase IV - September-October 2013. This will have an impact on measurement of the results

of the programme.

5.0 METHODOLOGY

The methodology of the evaluation will use a mix of qualitative and quantitative approaches;

Documents Review – These will be myriad of documents deemed relevant to the evaluation

of the programme and will include; proposal documents, technical reports, partner reports

financial reports, audits, sector plans from government, contracts, secondary document

from other stakeholders relevant to the programme.

Focus Group Discussions - Focus Group Discussions will be held at partner level and

beneficiary level.

Semi-Structured Interview (SSI) - SSI will be carried out with selected partners and

beneficiaries and government institutions.

Observations - Careful and systematic observation in places where the evaluation team will

engage beneficiaries and partners organizations.

41


Key Informant Interviews - This method is important to capture the views and professional

opinions of people who know about the circumstances on the ground. This will include

project officers, heads of partner organizations, government officials etc.

Most significant change stories - It is suggested that the consultants should collect stories of

change from identified beneficiaries of the project. The stories will focus on what the project

beneficiaries consider to be the most significant change brought by the project in their lives,

capacities or way of operating.

Primary Data analysis - Data gathered using various methods and tools have to be collated,

analyzed and interpreted systematically. The evaluators will be expected to come up with

detailed data/information analysis methods. It would be possible to utilize social science

data analysis software such as the Statistical Package for the Social Sciences

6.0 ROLES AND RESPONSIBILITIES

The consultant(s) will be responsible for supporting the evaluation through tool application,

data collection, and assisting in the production the evaluation report. Diakonia and Delta

staff will support the consultant(s) to mobilize and coordinate the logistics and data

collection process.

Diakonia will organize the verification workshops/meetings. The consultant(s) will report to

the Delta Regional Manager in Nairobi.

The Diakonia Country Representative in Garowe, Puntland, will provide management and

day supervision and support to the evaluation team whilst in Puntland.

Diakonia Monitoring, Learning and Evaluation Manager and Delta Regional Manager in

Nairobi, will provide overall quality assurance of the evaluation.

7.0 OUTPUTS/DELIVERABLES

The consultant(s) should deliver the following outputs and services;

Support Delta in carrying out the evaluation process, including tool development and

application in the field

Produce and submit to Delta Key Interview reports, completed Questionnaires, list of

persons interviewed,

Produce a draft report of the evaluation,

Support in the facilitation of a validation workshop in Garowe on the basis of the draft

report,

42


43


Annex 2 – Self Assessment questionnaire guideline

Our experience undertaking similar assignments has shown the need for undertaking a retrospective

facilitation exercise. The aim of this exercise is to bring out the key achievements, challenges and

recommendations from the clients perspective and helps the client team look back and recall the

original thinking. Experience has also shown that the output of these sessions is usually more

comprehensive and robust and serves to complement information got from other forms of data

collection such as interviews, questionnaires, talking to stakeholders e.t.c.

Did we do what we said we were going to do?

To answer this question we will need to refer to our log-frame with the main focus being at the

activity level. The rating system for this will have three points: no, partial and yes. The three ratings

reflect the degree to which the targets have been met. This serves as a proxy assessment of how

successful an organizational unit has been in achieving its outputs. The “partially achieved” category

is meant to capture those en route or particularly ambitious ones that may take considerable inputs

and time to come to fruition.

No—not achieved;

Partial—only if two-thirds or more of a quantitative target is achieved;

Yes—achieved.

Did this make a difference?

This will be zoning in on impact assessment. This will also be looking for information on how we are

attributing the difference/success to our interventions. The methodology will include the following

rating systems.

Positive change—positive movement from baseline to target as measured by the outcome

indicator(s);

Negative change—reversal to a level below the baseline as measured by the outcome

indicator(s);

Unchanged—no perceptible change between baseline and target as measured by the

outcome indicator(s).

For every change identified, we should try to show a causal link between the change and our

intervention.

Are/Were these the right things to do?

This will look at the strategic relevance of our interventions and key learning from them. This will

help in the design of future programmes.

List key achievements

List key challenges and recommendations.

44


45


Annex 3 - Partners meeting Agenda (Garowe)

Democracy & Human Rights Programme Phase IV

14th November 2013

Garowe, Puntland.

Agenda

14th

Nov

7:30-

8:00

Participants arrive

8:00-

8:45

Welcome Address and Participant Introductions:

Purpose, Objectives and Expected Outcomes of the meeting; and

Conduct of Business

9:00-

10:00

Partner organisation focus group discussions:

Thematic area of focus – how and why?

Were you involved in project design, how participatory was the process?

How was the partnership arrangement with Diakonia?

What worked well? What didn’t work well? Why?

What are the clear recommendations on partnership modalities going forwards?

10:15-

10:35

Coffee Break

10:45-

12:30

Partner organisation focus group discussions:

Did we make a difference – focus on key outcomes, success stories

Were these the right things to do – focus on relevance, contextual changes e.t.c.

Challenges

Recommendations

12:30-

13:30

Lunch and prayer break

46


13:30-

14:30

Report back - Presentations by selected group member

14:45-

15:15

Arrange with partners on site visits

15:30 Coffee break and end of session

End of day

Pre Workshop preparation.

Each partner to brainstorm on responses for the table below (This will entail the afternoon

session of the workshop so prior preparation could make that session easier for the

participants – it will also enable those who won’t be at the workshop to contribute towards

this).

Organisation Result area Activity to be

carried out in this

phase (Phase IV)

as per the log-

frame/inception

plan.

Status

a)No – not

achieved)

b) Partially - only

if two-thirds or

more of a

quantitative

target is achieved

c) Yes—fully

achieved

Underlying factors

for

achievement/non-

achievement.

47


Result 1

Result 2

Result 3

Result 4

48


Annex 4 - Partner interview template

Local Partners

Interview guide – DHRP phase IV

Introduction:

Name? Organisation or Group Name? What has been your role within the DHRP programme?

1. Overall programme results

1.1 What, in your view, is the most significant change between the start of the programme and the situation now?

1.2 Which top 3 factors contribute to the success of the programme? 1.3 What were the top 3 key challenges in implementing the programme/ your role in the

programme? 1.4 Which project activities planned for the project duration were not implemented? 1.5 What were the reasons for non-implementation (per activity)? ( tell us the internal issues,

external factors, etc for non-implementation) 1.6 What were the contributing factors to successful implementation? (internal and external)

List top 5: 1.7 Will any of the activities or practices continue after the programme has ended? If so, What

are your top 3 recommendations for sustaining the results achieved?

2. Management of the programme & partnership

3.1 How were decisions made in the programme? 3.2 What was your relationship like with Diakonia? 3.3 How were progress and challenges reported? And how were challenges followed up and

dealt with? 3.4 How often do you meet with Diakonia staff and what support do they offer your

organization?

3. Capacity building, coordination, and advocacy

3.5 How were people in the community involved in the programme activities? Did they have a good understanding of the programme purpose and of their role?

49


3.6 How did the community work together to implement the activities? Were there any challenges to working together?

3.7 Has your organization undergone capacity assessments and if yes when did Daikonia last assess your capacity?

3.8 Has your organization received any training from Diakonia over the last 2 years? If so what were the components and how did you manage to transfer the same to the community?

3.9 What other services has Diakonia offered your organization? E.g.: Technical support (Supporting the NGO in producing annual reports). Moral support (linking your organizations to other stakeholders?

3.10 How did the target beneficiaries respond to the activities? 3.11 What are some of the cost effective ways in which the same money availed could

achieve even better results?

4. Lessons learned and recommendations

4.1 What are the biggest things that went well in the programme and what did you learn the most?

4.2 What can be improved? How? 4.3 What would you do differently next time? 4.4 What are the recommendations for future programming related to improving

democracy and human rights in Somalia for greater impact and sustainability of the project?

Rating system – To be used to cross check the log-frame

For outcomes, the rating system has three points: positive change, negative change and

unchanged (no change). The three ratings reflect progress on outcomes, without attributing

the progress to any partner. The three ratings are meant to reflect the degree to which

progress has been made towards or away from achieving the outcome. The methodology in

all three ratings is to compare, as measured by outcome indicators, the evidence of

movement from the baseline either towards or away from the end target.

Positive change—positive movement from baseline to target as measured by the

outcome indicator(s);

Negative change—reversal to a level below the baseline as measured by the outcome

indicator(s);

Unchanged—no perceptible change between baseline and target as measured by the

outcome indicator(s).

The three ratings reflect the degree to which an output’s targets have been met. This serves

as a proxy assessment of how successful an organizational unit has been in achieving its

outputs. The “partially achieved” category is meant to capture those en route or particularly

ambitious outputs that may take considerable inputs and time to come to fruition.

No—not achieved;

Partial—only if two-thirds or more of a quantitative target is achieved;

50


51


Annex 5 – Staff questionnaire template

DIAKONIA staff

Interview Topic List – DHRP Phase IV

Introduction:

- Subject of evaluation - Duration of interview - Structure of interview

What has been your role within the DHRP programme?

1. Overall programme results

Impact

2.1 What is the most significant change between the start of the programme and the situation now?

2.2 What change has the programme made in people’s lives? Examples? 2.3 Which factors contribute to the success of the programme? 2.4 Were there any unintended results in the programme? Were they positive or negative and

in which ways did they affect the different stakeholders? 2.5 Were there innovations implemented and documented and what was the impact? 2.6 Did the programme activities and results influence the work of other organisations and

programmes? How do we know this? Effectiveness

2.7 What has been Diakonia’s progress towards planned output, outcome and impact results? (refer to log frame template)

2.8 What is the achieved data value of the output, outcome and impact indicators as identified in the log frame? (refer to log frame template)

2.9 What complementary responsibilities do other players (beneficiaries, partners and government line ministries, or other agency) have in relation to the project?

2.10 What were the contributing factors to successful implementation? (internal and external) List top 5:

1. __________________________________________________________________

2. __________________________________________________________________

3. __________________________________________________________________

4. ---------------------------------------------------------------------------------------------------------

5. ------------------------------------------------------------------------------------------------------------

52


2.11 What were the most critical factors or constraints affecting performance? Key challenges? How did your organisation deal with these difficulties?

Relevance

2.12 Was the programme formulated and implemented in accordance with national and local strategies? What are these national and local strategies?

2.13 What considerations influenced your decisions on- (a) The project thematic focus?

(b) The project’s geographical focus?

(c)The project target beneficiaries?

2.14 To what extend have cross-cutting issues such as human rights and gender equality been integrated in the design and implementation of the programme? Examples?

2.15 To what extent has the programme been adjusted during implementation given changed circumstances? Examples and why?

2.16 What kind of feedback mechanisms have been put in place? How have they been used? 2.17 What is the opinion of key stakeholders on Diakonia's performance and engagement with

them? Sustainability

2.18 Did the programme design include an appropriate sustainability and exit strategy (including promoting national/local ownership, use of local capacity, etc.)? Which measures were put into place to ensure sustainability?

2.19 What is the likelihood of results being sustainable after the programme has ended? Why?

Efficiency

2.20 How economically were resources/inputs (funds, expertise, time, etc.) converted to Diakonia results at the output level?

2.21 To what extend did the budget and actual expenditure at the country/county level reflect identified programme priorities?

2.22 What were the available resources and actual expenditures compared to planned budget? If not where were the constraints?

A. Direct Cost

How much money was budgeted for direct intervention

How much money out of the budget was disbursed

Was the amount of money budgeted for equal to the money disbursed

If your answer is yes no in the question above what are the reasons

B. Overhead

How much money was budgeted for administration cost intervention

How much money out of the budget was used

53


Was the amount of money budgeted for equal to the money budgeted

If your answer is yes no in the question above what are the reasons

C. Indirect Cost

What % of the budget was budgeted for staffing

Were the staff handling the project adequate

Was money budgeted for staffing adequate and if not why

Value for Money

2.23 Which opportunities to increase value for money have been identified and successfully implemented during programme implementation?

2.24 What were the most critical factors or constraints affecting value for money during programme implementation and what were the lessons learnt regarding VfM?

3. Management of the programme & partnership Relevance

3.1 To what extent has the programme been demand led? 3.2 In your opinion, have beneficiaries been involved in all programme stages (baseline-if any,

identification, design, implementation and monitoring and evaluation?

Impact

3.3 Has enough attention been devoted to influencing the decisions of key ‘champions’ in Government/Civil society?

Efficiency

3.4 Are partners strategically selected and adequately resourced? 3.5 How effectively have risks been managed during programme implementation? 3.6 How did you manage to laise with other stakeholders implementing similar project in the

field. In another word, what mechanism is in place to avoid duplications? Sustainability

3.7 How effectively has local ownership of the programme been increased? 3.8 In what way has Diakonia shared best practises and lessons learned with other partners and

donors in the country?

4. Partner capacity Relevance

4.1 To what extent has the programme aligned with the plans and ambitions of the individual partners?

4.2 To what extent have the capacities of partners been assessed and strengthened? 4.3 How often do you assess the capacity of your partners using organizational capacity

assessment tools (OCAT)? Impact

54


4.4 Which interventions have been most successful in the country? 4.5 Has enough been done to develop synergies with other programmes so as to increase

impact? 4.6 Has the partner benchmarking and self-assessment process been done? If yes, how useful

was this for increasing impacts? Effectiveness

4.7 Have individual partner operational plans been developed? If yes, are they realistic and robust enough?

Value for Money

4.8 What has been the cost for capacity building vis a vis the impact made (link with 4.2)?

5. Quality of the PM&E system Relevance

5.1 What kind of PM&E system has been used in the programme? 5.2 How has performance data been collected (by who, frequency, methodology, data source)? 5.3 Have annual reviews of the M&E system and log frame been conducted? Have these

resulted in appropriate changes in the system? 5.4 To what extend did the collected data reflect the data needs throughout programme

implementation? Impact

5.5 How has the PM&E system contributed to improved programme performance? 5.6 How are stakeholders involved in the planning, implementation, monitoring and evaluation

of the programme? Efficiency

5.7 Which proportion of programme budget has been allocated for PM&E? 5.8 Has the budget for PM&E been sufficient?

Effectiveness

5.9 To what extend have recent reviews, evaluations and identified best practises been used to inform decision making and improvement of programme implementation? Examples?

Sustainability

5.10 How has the PM&E capacity of local partners been strengthened? 5.11 What is the likelihood of local partners continuing to effectively monitor and evaluate

their programmes after the end of cycle? 5.12 Which gaps in PM&E capacity among local partners have been identified and which areas

need future strengthening? Value for Money

5.13 Which opportunities to increase value for money have been identified and successfully implemented during programme PM&E?

Replicability

5.14 What best practises and lessons learned have been identified in programme PM&E?

55


Lessons learned and recommendations

6.1 What are the main best practices and lessons learned? 6.2 What can be improved? How? 6.3 What would you do differently next time? 6.4 What are the recommendations? 6.5 What are the next steps to be taken as follow up?

Replicability

6.6 Do you think this programme can be replicated elsewhere? 6.7 If yes, what advice would you give to other organizations/ people who will replicate this

model? Specific adjustments in the strategies, focus and planned results?

Closing

- Anything to add? - Any documentation to share? - Any questions? - Sharing of evaluation findings - Thank you very much!