mid-term review of the syria needs analysis project

8/13/2019 Mid-term review of the Syria Needs Analysis Project

1/55

Syria Needs

AnalysisProject (SNAP)External mid-

term review

Andy Featherstone,

Independent consultant,

October 2013


2/55

iExternal mid-term review of the Syria Needs Analysis Project

Syria Needs Analysis Project (SNAP): External mid-term review

Andy Featherstone, October 2013

EXECUTIVE SUMMARY

The Syria Needs Analysis Project

SNAP is a one-year project delivered through a partnership between ACAPS and MapAction running

from December 2012 December 2013. SNAP aims to link with humanitarian actors working from

operational hubs to facilitate a comprehensive independent analysis of the humanitarian situation in

Syria and neighbouring countries, including conducting rapid needs assessments. In so doing, SNAP

aims to equip the humanitarian community with information that would enable better humanitarian

assistance to the populations affected by the Syria crisis.

Purpose of the review and methodology

The purpose of the review is to obtain an independent evaluation of the performance of the SNAP

project against the targets outlined in the projects logical framework analysis, using the evaluation

criteria of relevance, effectiveness, efficiency, connectedness, and coherence. The methodology for

the review consisted of the following activities:

Review of SNAP project proposal, reports and other relevant background material. Interviews with Assessment Capacities Project (ACAPS) Project Management Team in Geneva,

MapAction management, key donors, United Nations (UN) agencies and Non-governmental

Organisations (NGOs) headquarters.

A 12-day fieldtrip which included interviews with SNAP staff, donors, UN agencies and NGOs inJordan, Lebanon and Turkey

An online survey of users of SNAP outputs (see annex 3 for the survey results).The review was commissioned by the ACAPS Board and was managed by a steering committee (SC)

composed of representatives from the ACAPS board, donor and partner representatives.

Overall findings

Over the period under review, SNAP has offered significant value to the humanitarian community in

strengthening the targeting of assistance and in making an important contribution to a shared

situation awareness. That is has achieved this in the context of a complex regional crisis and a

fragmented humanitarian architecture is testament to the professionalism and tenacity of the SNAPteam.

Relevance

The relevance of SNAP has stemmed from its ability to fill critical gaps in the information and

analysis of the humanitarian community. The SNAP-authored Regional Analysis for Syria (RAS) is

widely credited to have been the first coherent regional analysis, while the SNAP-supported Joint

Rapid Assessment of Northern Syria (J-RANS) remains the only coordinated needs assessments that

have been conducted in northern Syria. While the thematic reports and scenario documents tend to

have less profile they are credited with providing much-needed analysis and in offering a common

language on key humanitarian issues. SNAPs agility in identifying and responding to the needs of the

humanitarian community has gone a significant way to bringing a level of coherence to a very

complex and fragmented humanitarian context.


3/55

iiExternal mid-term review of the Syria Needs Analysis Project

Connectedness

There was a general level of satisfaction expressed by review participants about the

complementarity of SNAPs information and analysiswith other providers. While in the months after

its deployment comparable products were few in number, with time the number and variety has

increased and while duplication is still relatively small there is a need to look proactively at ways inwhich SNAP could best complement the work of others. This is exemplified by the steps taken to

develop a follow-up needs assessment in northern Syria. Coordinated by the Office for the

Coordination of Humanitarian Affairs (OCHA) and IMPACT, SNAP is participating in this initiative and

in so doing has taken an important step in strengthening collaboration.

Coherence

Despite SNAPs approach to expanding by seizing opportunities and responding to needs its

information and analysis products are coherent and all make a contribution to its core purpose. That

said a vision and set of principles that articulate SNAPsdistinctive competencies and which can help

future-proof it against loss of institutional memory would be a wise investment.

Effectiveness

SNAP has made impressive progress against the outputs in its log frame and is on track to meet

expectations by the current close of the project. Several of its early publications were considered

extremely timely and have laid a foundation for SNAP to build on. Interviews with NGOs, UN

agencies, donors at project, country, regional and headquarters level provided a mosaic of uses for

SNAPs information and analysis which was considered by most participants who participated in the

review to be of high quality which helped foster trust and credibility with SNAPs key clients. It was

more difficult to determine the perceptions of national and regional organisations or the Arabic-

speaking world more broadly and there would be value in SNAP reflecting on the whether these are

targets and if so how best to tailor its products to better include them. An exploration of perceptions

of the independence of SNAPs analysis revealed that a majority feel that it is free from significantpolitical bias which is considered of key importance to its users, particularly in an environment

where access constraints and operational security concerns have served to degrade perceptions of

the independence of information.

Efficiency

In establishing SNAP as an operational entity in Lebanon, Turkey and Jordan, ACAPS and MapAction

have both worked outside their organisational comfort zones but have made some wise decisions.

While a comparison of alternative models suggests that SNAP is comparatively resource-heavy,

analysis suggests that it would have been impossible to achieve the same added value at distance,

via a series of secondments or through a more modest deployment strategy. An analysis of SNAPs

distinctive competencies relative to other information and analysis providers highlights itscomprehensive regional analysis, its independence from a fragmented humanitarian architecture

and its agility in identifying and seizing opportunities as being unique. A presence across the region

has undoubtedly allowed SNAP to develop links, strengthen trust and build a comprehensive

regional analysis and in so doing provides a compelling justification for the decision to adopt the

model.

Value for money (VfM)

While it is methodologically complex to assign a quantitative value to an intangible outcome (a

shared situational analysis), an analysis of SNAPs VfM suggests that it has striven to identify cost

drivers and has identified means of strengthening the economyof the project albeit learning some

important lessons about resourcing for the future. While the high cost of the SNAP model in

comparison with the alternatives could lead one to question the efficiencyof the project, SNAPs

distinctive competencies in comparison to other information and analysis providers and its


4/55

iiiExternal mid-term review of the Syria Needs Analysis Project

effectivenessin delivering against its objectives and making progress towards its outcome provides a

compelling justification for an operational deployment. This has provided an important foundation

for its success which would likely have been compromised by having a more limited deployment in

the region, by adopting a model of agency secondments or by seeking to deliver the project

remotely from Switzerland (ACAPS) and the UK (MapAction). This analysis supports a conclusion that

the SNAP project does indeed offer value for money.


5/55

ivExternal mid-term review of the Syria Needs Analysis Project

Table of Contents

Executive summary i

Contents iv

Acronyms v

SNAP: Regional Analysis of Syria, 26 September 2013 vi

1. The purpose of the review and methodology 1

1.1 The purpose of the review 1

1.2 Methodology 1

1.3 Limitations 1

2. The context of the humanitarian situation in Syria and SNAP 1

2.1 Context of humanitarian situation in Syria 1

2.2 Background to the Syria Needs Analysis Project 3

3. Relevance 6

4. Connectedness 10

4.1 SNAP information and analysiscomplementary to or duplicating the work of others? 12

5. Coherence 14

6. Effectiveness 14

6.1 Progress of SNAP against its objectives 14

6.2 An analysis of risks and assumptions 16

6.3 SNAPs contribution to strengthening targeting and promoting a shared situational analysis 17

6.4 Timeliness 196.5 The value of independent analysis in Syria 20

7. Efficiency 22

7.1 Distinctive competency analysis 22

7.2 Alternative means by which SNAP could have achieved its outcomes 24

7.3 Towards a judgment on the efficiency of SNAP 25

8. Value for money 26

8.1 Towards a judgment on value for money 28

9. Recommendations for the future of SNAP 28

9.1 Challenges for the future: Maintaining SNAPs relevance in a changing context 30

9.2 Reflections on the added value of the SNAP model for crises elsewhere 31

9.3 Lessons learned 33

10. Conclusion and summary of recommendations 33

Annexes

Annex 1: Review participants 37

Annex 2: Fieldwork schedule 39

Annex 3: Online survey 40

Annex 4: ToR for the review 45


6/55

vExternal mid-term review of the Syria Needs Analysis Project

Acronyms

ACAPS Assessment Capacities Project

ACU Assistance Coordination Unit

AWG Assessment Working Group

DFID Department for International DevelopmentDNA Disaster needs Analysis

HC Humanitarian Coordinator

HCT Humanitarian Country Team

HIU Humanitarian Information Unit

ICG International Crisis Group

IDMC Internal Displacement Monitoring Centre

IDP Internally Displaced Person

JHA Joint Humanitarian Assessment

J-RANS Joint Rapid Analysis of Northern Syria

J-RASS Joint Rapid Analysis of Southern Syria

NGO Non-governmental NGONRC Norwegian Refugee Council

OCHA Office for the Coordination of Humanitarian Affairs

OFDA Office for U.S Foreign Disaster Assistance

RAS Regional Analysis for Syria

RRRP Regional Refugee Response Plan

SDR Secondary Data Review

SHARP Syria Humanitarian Assistance Response Plan

STIMA Syria/Turkey Information Management Task Force

UN United Nations

UNHCR united Nations High Commissioner for Refugees

VfM Value for MoneyWFP World Food Programme


7/55

viExternal mid-term review of Syria Needs Analysis Project

SNAP: Regional Analysis of Syria, 26 September 20131

1Source: SNAP (2013) Regional Analysis for Syria, Overview, 26 September 2013


8/55

1External mid-term review of the Syria Needs Analysis Project

Syria Needs Analysis Project (SNAP): External mid-term review

Andy Featherstone, October 2013

1. The purpose of the review and methodology

1.1 The purpose of the review

The goal of the review is to obtain an independent evaluation of the performance of the SNAP

project against the targets outlined in the projects logical framework analysis, using the evaluation

criteria of relevance, effectiveness, efficiency, connectedness, and coherence. At the outcome level,

the purpose of SNAP is that populations affected by the Syria crisis benefit from improved targeted

response provided by humanitarian stakeholders operating with a shared situation awarenessand

the focus of the review will be on assessing the progress made towards achieving this.

The review covers the period from the commencement of the SNAP project in December 2012 until

end-August 2013. Taking into account the innovative nature of the SNAP, it also seeks to identify

lessons learned with a view to identifying the role that a project such as SNAP can play in othercontexts.

The review was commissioned by the ACAPS Board and was managed by a steering committee (SC)

composed of representatives from the ACAPS board, donor and partner representatives.

1.2 Methodology

The methodology for the review consisted of the following activities:

Review of SNAP project proposal, reports and other relevant background material. Interviews with ACAPS Project Management Team in Geneva, MapAction management, key

donors, UN agencies and NGOs headquarters. A 12-day fieldtrip which included interviews with SNAP staff, donors, UN agencies and NGOs in

Jordan, Lebanon and Turkey

An online survey of users of SNAP outputs (see annex 3 for the survey results).1.3 Limitations

While the fieldwork, travel to Geneva and follow-up telephone interviews permitted discussions

with a representative sample of SNAPs users, the exception to this was in Syria itself which it was

not possible to travel to. Despite efforts to contact and interview informants from 9 agencies, no

interviews were conducted and only 2 of the 94 participants of the online survey worked in Syria

(both for UN agencies). It is unclear why participation from within Syria was so poor although it is

possibly linked to sensitivities concerned with sharing information over the telephone. While this is aweakness of the evaluation, there was good participation from agencies and staff working in Syria

cross-border (10 agencies that participated in the review were working in this way) and there were a

far greater number who were working with Syrian refugees in neighbouring countries. Discussions

were also held with headquarters staff who managed operations within Syria and the surrounding

countries.

2. The context of the humanitarian situation in Syria and SNAP

2.1 Context of the humanitarian situation in Syria

The Syria crisis is complex and has caused extensive human suffering. An estimated 6.8 million

people in Syria, almost one-third of the population, have been affected by the conflict, including


9/55


more than 4.25 million displaced inside Syria.2 On September 3, the UN High Commissioner for

Refugees (UNHCR) announced that the number of Syrians displaced as refugees exceeded 2 million,

with 97% fleeing to countries in the immediate surrounding region, including Turkey, Lebanon,

Jordan, Iraq, Egypt, and other parts of North Africa.3

At the end of May 2012 an agreement was reached between the Syrian Government andrepresentatives of the United Nations which allowed access by 8 UN agencies and a handful of NGOs

to key locations for the delivery of assistance.4By August 2013 the number of NGOs working inside

had risen to 125although significant gaps in the delivery of life-saving assistance continue to exist

prompting the Emergency Relief Coordinator and the UN High Commissioner for Human Rights to

jointly release a statement urging all parties to respect their obligations under international human

rights and humanitarian laws to protect civilians and to allow neutral, impartial humanitarian

organizations safe access to all people in need, wherever they are in Syria.6Outside of the country, a

growing number of organisations have responded to the challenges of delivering humanitarian

assistance from within the country by supporting its delivery from neighbouring countries.

In addition to struggling to meet the humanitarian crisis within Syria, huge needs exist inneighbouring countries where governments assisted by the humanitarian community are seeking to

provide assistance to vast numbers of refugees. As of 15-September 2013, UNHCR estimated that

there were 2.038 million refugees.7Of this number, 739,000 were being hosted by Lebanon, 519,676

were in Jordan and 463,885 were in Turkey.8

A humanitarian crisis of such significant proportions has required huge resources and while donors

have been generous, there has been a consistent funding shortfall. Two appeals were launched to

meet needs inside and outside of Syria; at the time the review was conducted the 2013 Syria

Humanitarian Assistance Response Plan (SHARP) with a requirement of $1.4 billion was 46.7%

funded;9the 2013 Syria Regional Refugee Response Plan (RRRP) with a requirement of $2.981 billion

was 42.8% funded.

10

The scenarios for the Syria conflict remain complex and agencies operating within Syria and the

surrounding countries face a continuing struggle to prepare response mechanisms and to allocate

2Source: OCHA Syria Humanitarian Bulletin, Issue 33, 27 August9 September 2013 (internet). Available at

http://reliefweb.int/sites/reliefweb.int/files/resources/Syria%20Humanitarian%20Bulletin%20Issue%20No%20

33%20%281%29.pdf3Source: UNHCR press release: 2 million Syrians are refugees, September 3 2013 (internet). Available at

http://data.unhcr.org/syrianrefugees/documents.php?page=2&view=grid&Country%5B%5D=1224Source: Humanitarian Policy Group (2013) Syria crisis: The humanitarian response, 15

thJune 2012,

Roundtable, London Overseas Development Institute (internet). Available at

http://www.odi.org.uk/sites/odi.org.uk/files/odi-assets/events-documents/4907.pdf5UN OCHA, Syrian Arab Republic: Humanitarian Presence: International NGOs (as of 7 August 2013)

6United Nations press release, UN humanitarian and human rights chiefs urge immediate safe passage for

civilians and aid workers in Homs and Aleppo, Syria, 12 July 2013 (internet). Available at

http://reliefweb.int/report/syrian-arab-republic/un-humanitarian-and-human-rights-chiefs-urge-immediate-

safe-passage-07Of this number 1,851, 572 are registered and a further 186,766 are awaiting registration (see

http://data.unhcr.org/syrianrefugees/regional.phpaccessed on 15/09/13)8Source: UNHCR, Syria Regional Refugee Response (internet). Available at

http://data.unhcr.org/syrianrefugees/regional.phpaccessed on 15/09/139Source: OCHA FTS (internet). Available at

http://fts.OCHA.org/reports/daily/ocha_R1_A1007___1309150100.pdf.Accessed on 15/09/1310

Source: OCHA FTS (internet). Available at

http://fts.OCHA.org/reports/daily/ocha_R1_A1010___1309150100.pdf.Accessed on 15/09/13
http://reliefweb.int/sites/reliefweb.int/files/resources/Syria%20Humanitarian%20Bulletin%20Issue%20No%2033%20%281%29.pdfhttp://reliefweb.int/sites/reliefweb.int/files/resources/Syria%20Humanitarian%20Bulletin%20Issue%20No%2033%20%281%29.pdfhttp://reliefweb.int/sites/reliefweb.int/files/resources/Syria%20Humanitarian%20Bulletin%20Issue%20No%2033%20%281%29.pdfhttp://data.unhcr.org/syrianrefugees/documents.php?page=2&view=grid&Country%5B%5D=122http://data.unhcr.org/syrianrefugees/documents.php?page=2&view=grid&Country%5B%5D=122http://www.odi.org.uk/sites/odi.org.uk/files/odi-assets/events-documents/4907.pdfhttp://www.odi.org.uk/sites/odi.org.uk/files/odi-assets/events-documents/4907.pdfhttp://reliefweb.int/report/syrian-arab-republic/un-humanitarian-and-human-rights-chiefs-urge-immediate-safe-passage-0http://reliefweb.int/report/syrian-arab-republic/un-humanitarian-and-human-rights-chiefs-urge-immediate-safe-passage-0http://reliefweb.int/report/syrian-arab-republic/un-humanitarian-and-human-rights-chiefs-urge-immediate-safe-passage-0http://data.unhcr.org/syrianrefugees/regional.phphttp://data.unhcr.org/syrianrefugees/regional.phphttp://data.unhcr.org/syrianrefugees/regional.phphttp://data.unhcr.org/syrianrefugees/regional.phphttp://fts.unocha.org/reports/daily/ocha_R1_A1007___1309150100.pdfhttp://fts.unocha.org/reports/daily/ocha_R1_A1007___1309150100.pdfhttp://fts.unocha.org/reports/daily/ocha_R1_A1010___1309150100.pdfhttp://fts.unocha.org/reports/daily/ocha_R1_A1010___1309150100.pdfhttp://fts.unocha.org/reports/daily/ocha_R1_A1010___1309150100.pdfhttp://fts.unocha.org/reports/daily/ocha_R1_A1007___1309150100.pdfhttp://data.unhcr.org/syrianrefugees/regional.phphttp://data.unhcr.org/syrianrefugees/regional.phphttp://reliefweb.int/report/syrian-arab-republic/un-humanitarian-and-human-rights-chiefs-urge-immediate-safe-passage-0http://reliefweb.int/report/syrian-arab-republic/un-humanitarian-and-human-rights-chiefs-urge-immediate-safe-passage-0http://www.odi.org.uk/sites/odi.org.uk/files/odi-assets/events-documents/4907.pdfhttp://data.unhcr.org/syrianrefugees/documents.php?page=2&view=grid&Country%5B%5D=122http://reliefweb.int/sites/reliefweb.int/files/resources/Syria%20Humanitarian%20Bulletin%20Issue%20No%2033%20%281%29.pdfhttp://reliefweb.int/sites/reliefweb.int/files/resources/Syria%20Humanitarian%20Bulletin%20Issue%20No%2033%20%281%29.pdf


10/55


humanitarian funding assistance. This insecurity increases the importance of accurate and timely

situation analysis of the needs and vulnerabilities of populations affected by the conflict.

2.2 Background to the Syria Needs Analysis Project

SNAP is a one-year project delivered through a partnership between ACAPS and MapAction running

from December 2012December 2013. It seeks to deliver independent analysis of the humanitariansituation and in so doing, facilitate a better targeted response and shared situational analysis

through the collation and analysis of existing assessment data and the production of:

Regional Analysis for Syria (RAS) - A monthly general brief produced by ACAPS and MapActioncollated from a review of existing assessment and response data. The brief focuses on gap

analysis, lessons learned, priority needs, displacement profile, sectoral analysis;

Thematic Reports - specialised briefings on relevant sectors; topics and indepth country/Governorate briefings;

Scenario reports.SNAP has also provided support to the humanitarian community on technical assistance oninformation management and joint assessments specifically on the Joint Rapid Assessment of

northern Syria.

Figure 1: The Assessment Capacities Project and MapAction

In accordance with fundamental humanitarian principles, and as endorsed by the Good Humanitarian

Donorship Initiative, humanitarian aid should be based on a clear understanding of the needs of the affected

population. However, providing a sufficiently coherent picture of humanitarian requirements, especially in the

initial phases of an emergency can prove challenging. The Assessment Capacities Project (ACAPS) was

established to help overcome these issues. ACAPS is an initiative of a consortium of three Non-Governmental

Organisations (NGO), HelpAge International, Merlin and Norwegian Refugee Council. The objective of the

project is to assist in strengthening global, regional and in-country needs assessment in coordination with itspartners which includes NGOs, United Nations and academics.

MapAction is a UK-based NGO that supports humanitarian agencies through the provision of information

management and geographic information system services with a view to creating a shared operational picture

to assist in the targeting of humanitarian assistance.

Why was SNAP launched?

The SNAP project was borne out of a particular set of circumstances that were present in Syria

towards the end of 2012 which included a crisis of growing proportions for which there was a lack of

coherent analysis and scant information on the location and scale of the needs. One of the

implications of the regional nature of the emergency which went far beyond Syrias own borders

was that there was a lack of a shared situation awareness which was manifest in inconsistencies in

how data was presented and the way in which needs were extrapolated. Specific impediments to

creating a shared situational awareness include the following;

The regional nature of the crisis which includes both internally displaced persons (IDPs) in Syriaand refugees in neighbouring countries has resulted in information management being divided

between two UN agencies, the United Nations High Commissioner for Refugees (UNHCR), which

has responsibility for the refugees, and the United Nations Office for the Coordination of

Humanitarian Affairs (OCHA) which focuses on negotiating access to affected [displaced] people,

information management and coordinatinghumanitarian response and advocacy.11

11Source: OCHA in 2012 and 2013, Syrian Arab Republic (internet). Available at

http://www.OCHA.org/ocha2012-13/syria.Accessed on 30/09/13
http://www.unocha.org/ocha2012-13/syriahttp://www.unocha.org/ocha2012-13/syriahttp://www.unocha.org/ocha2012-13/syria


11/55


Complex challenges facing (i) agencies which are registered in Syria and working within thecountry and (ii) agencies that are working cross-border from neighbouring countries which has

led to fragmented operations and reporting such that information-sharing between the two

groups has been limited. The extent of the needs and the risks associated with working in Syria

cross-border have also tended to mean that many agencies work in isolation of each other which

has resulted in relatively few multi-agency or multi-sector assessments. For the agencies basedin Syria, while an OCHA-led Joint Humanitarian Assessment (JHA) has been prepared, the

methodology has been field-tested and enumerators have been trained, implementation of

primary data collection is still pending approval from the Government and while some sectors

have made headway in seeking to generate an overall picture of needs (most notably in the food

sector), there has been very limited multi-sectoral or cross-lines assessment data.

While the Syria Regional Refugee Response Inter-agency Information Sharing Portal12has madesignificant progress in providing a coherent and comprehensive overview of numbers, needs and

news and hosts a rapidly expanding library of relevant documents, the complexity of the refugee

situation does not lend itself easily to analysis because of the differing status of those fleeing

Syria13

and the manner in which they are accommodated which has implications for their

visibility and the services they receive.

The lack of coherent analysis has numerous implications for the international response to the crisis,

particularly in the following ways:

There has been limited analysis of humanitarian needs which has adversely affectedhumanitarian appeals, particularly the 2012 Syria Humanitarian Response Plan (SHARP). In the

absence of access in large parts of Syria and given the paucity of assessments, there was an

important need to provide donors more accurate information about priority needs to allow

funding to be mobilized.

Operational agencies have been concerned about the impartiality of assistance; in the absenceof comprehensive information about needs, it has been extremely difficult to make informed

decisions about priority needs in government and opposition-controlled areas, cross-lines andcross-border. Given the scale of the needs throughout Syria and other countries affected by the

crisis, there is no question about the importance of aid that is being provided but there have

been concerns about a lack of knowledge about where the gaps are.

The complexities to providing humanitarian assistance in Syria has fragmented humanitariancoordination and exacerbated existing weaknesses in how data is shared between organisations.

It is these gaps in analysis that SNAP has sought to fill by collecting and synthesizing information,

undertaking analysis and supporting coordinated needs assessment. In doing this SNAP has relied on

its own horizon scanning and the support of an Amman-based Advisory Group comprised of

representatives from donors, NGOs and UN agencies to guide its work.

Brief history of SNAP

The initial concept of the SNAP project14

was borne out of a joint MapAction/Assessment Capacities

Project (ACAPS) scoping mission to Beirut in August 2012 which noted concern from many parts of

the humanitarian community of information gaps which existed and was followed by discussions

with the Department for international Development (DFID) which facilitated discussion with OCHA

concerning the added value of an operational unit to liaise with humanitarian stakeholders in order

12Seehttp://data.unhcr.org/syrianrefugees/regional.php

13There are 3 different groups; (i) those residing in camps, (ii) those who have the appropriate papers and are

therefore regularly residing in a country, and; (iii) those who are irregular, meaning residing in a host country

without the required documents (Source: SNAP (2013) Legal status of individuals fleeing Syria, June 2013)14

At the time the concept note was drafted in September 2012, the project was known as the Humanitarian

Assessment Project (HASP) but this was later changed to SNAP.
http://data.unhcr.org/syrianrefugees/regional.phphttp://data.unhcr.org/syrianrefugees/regional.phphttp://data.unhcr.org/syrianrefugees/regional.phphttp://data.unhcr.org/syrianrefugees/regional.php


12/55


to collect information and generate regular reports on the situation of the affected populations.15

To

achieve this, a small ACAPS/MapAction team hosted by the Norwegian Refugee Council (NRC) was

deployed to the region with an initial deployment in Beirut. The first product from the team was the

Syria Disaster Needs Analysis published on 22nd

December 2012. This was followed by the Regional

Analysis of Syria (RAS) published in January 2013 which over the subsequent 9-months has become

the backbone of SNAPs work.

With a presence in the region, the SNAP team took an entrepreneurial approach to identifying

priorities which prompted them to provide technical support to a group of NGOs and the ACU for

coordinated needs assessment undertaken from the relative security of Turkey. An initial

deployment in support of the emerging NGO Forum in January 2013, prior to the deployment of an

OCHA team offered an opportunity to support the collation and synthesis of assessment data. While

this deployment was initially outside SNAP (a direct secondment from ACAPS to an NGO), it rapidly

became clear that SNAP should expand its role to include such direct support to assessment

initiatives. A revised proposal was agreed by the donor whereby the technical support required to

facilitate primary data collection in Syria, specifically the Joint Rapid Assessment of Needs (J-RANS) -

with reports completed on 17 February 2013 (J-RANS I) and 22 May 2013 (J-RANS II), was providedby SNAP.

In the process of developing the RAS, gaps in humanitarian information and analysis came to light

which the SNAP team has sought to fill through the publication of thematic reports with a profile of

the Aleppo Governorate (building on the J-RANS) in April 2013 which was followed by reports on the

legal status of individuals fleeing Syria in June 2013. As SNAPs knowledge and anal ysis of the

situation has developed, so too have its opportunities to add value, with a document outlining

Scenarios for the region released in February and updated in September 2013. These were produced

with a view to supporting shared strategic planning, providing early warning and promoting

preparedness activities. SNAP has also sought to harvest learning from its own work (particularly the

J-RANS) and the work of others through the launch of a report on needs assessment lessons learnedreport in September 2013 (see figure 2).

Figure 2: Timeline of SNAP information and analysis products

SNAP Output Title Date

Regional Analysis of Syria RAS SeptemberSyria (part I) and Host countries (part II) 26 September 2013

RAS AugustRegional analysis of the Syria conflict 28 August 2013

RAS July - Syria (part I) and Host countries (part II) 31 July 2013

RAS June - Syria (part I) and Host countries (part II) 26 June 2013

RAS May - Syria (part I) and Host countries (part II) 30 May 2013

RAS AprilThe crisis overview 30 April 2013

RAS March - Syria (part I) and Host countries (part II) 28 March 2013

RAS February - Syria (part I) and Host countries (part II) 27 February 2013

RAS JanuarySyria (part I) and Host countries (part II) 28 January 2013

Thematic reports Needs assessment lessons learned September 2013

Syrian border crossings September 2013

Impact of the conflict on Syrian economy & livelihoods July 2013

Legal status of individuals fleeing Syria June 2013

Aleppo Governorate profile April 2013

Scenario reports Second report September 2013

First report February 2013

Coordinated needs

Assessment

Joint rapid assessment of northern Syria (J-RANS II) 22 May 2013

Joint rapid assessment of northern Syria (J-RANS I) 17 February 2013

15Source: Private email communication between ACAPS and OCHA, Geneva, 02 August 2012


13/55


3. Relevance

Relevance is concerned with assessing whether the project is in line with local needs and priorities

(as well as donor policy).16

In the context of the SNAP review the relevance criterion speaks to the

extent to which SNAP has provided information and analysis that is tailored to the needs of

humanitarian agencies seeking to provide assistance to those requiring it.

At the time of SNAPs launch in December 2012, very limited information existed about the

humanitarian situation in Syria due to the limited access that many agencies had to assessing needs

in the country. The international organisations operating in Syria were reliant on obtaining

government permission for the provision of assistance and while there were some clandestine cross-

border humanitarian operations being carried by a handful of NGOs, the majority of agencies were

not in a position to provide assistance. At the same time, estimated numbers of IDPs were increasing

rapidly (see figure 3).

Figure 3: Estimated IDP numbers within Syria (August 2012September 2013)17

Outside of Syria, the situation was different; there was far greater access to the refugees but

capacity was limited and while information was available, there was a lack of analysis. SNAPs arrival

also coincided with a massive increase in refugee numbers in each of Lebanon, Jordan, Turkey, Egypt

and Iraq such that agency capacity was stretched leaving very limited space for information

collection and synthesis (see figure 4 below).

Regional Analysis of Syria

As a consequence of the rapid increase in numbers of people requiring humanitarian assistance and

the limited capacity of humanitarian organisations and the limited access that they had to key parts

of Syria, information on the crisis was severely fragmented. This was exacerbated by the lack of a

single agency with a mandate to collate, synthesize and analyse information on the humanitarian

situation of those displaced in Syria in addition to those that had sought refuge in neighbouring

countries. For this reason alone the production of SNAPs first RAS was considered a significant and

16Source: ALNAP (2006) Evaluating humanitarian action using the OECD-DAC criteria: An ALNAP guide for

humanitarian agencies, Overseas Development Institute, London17

Figures are not always available for the exact end of the month due to varying OCHA reporting times. Hence

a months data may be taken from the earliest report of the following month. Source: OCHA Humanitarian

Bulletin, Issues 133 inFerris E, Kirisci K and Shaikh S (2013) Syrian Crisis: Massive Displacement, Dire Needs

and a Shortage of Solutions, Brookings Institution, pp.iv


14/55


timely achievement. That the exercise coincided with the beginning of a spike in population

movements and associated humanitarian need made the analysis all the more relevant to the needs

of a diverse set of stakeholders within the humanitarian community who were unable to access this

information from elsewhere.

The RAS has helped us makesense of the broken links between different projects within theregion NGOProgramme Director, Turkey

Figure 4: Estimated number of Syrian refugees (March 2012September 2013)18

For operational NGOs and UN agencies, the lack of coherent information and analysis had militatedagainst them developing a holistic picture of needs and while few agencies reported using the RAS to

make programming decisions, the majority considered it the only resource that has consistently

linked IDP needs in Syria with those of refugees in neighbouring countries making it invaluable for

broader planning purposes. Staff newly deployed to the region spoke of it as being their sole

reference and sector specialists spoke of it being a valuable resource for linking up their more

detailed geographical or sectoral knowledge with sectors and areas beyond their expertise.

Donor agency staff (OFDA, DFID, CIDA), analysts (Internal Displacement Monitoring Centre and NGO

policy staff) and headquarters staff strongly endorsed the enduring relevance of the RAS pointing to

its value as a reference resource to complement their own agency analysis and spoke of the value of

the hyperlinks provided to source material used in the document. With the 9

th

version released atthe end of September, many informants spoke of its value in documenting the changes in the

humanitarian situation over time.

We consider SNAP to be highly relevant and extremely useful. The monthly updates provide

us with a regional analysis and assist us in triangulating information on the ground

Headquarters-based donor representative

A small number of informants spoke of the declining relevance of the RAS with time; in particular

OCHA agency staff spoke of the information they received from sources both inside and outside of

18Figures are not always available for the exact end of the month due to varying OCHA reporting times. Hence

a months data may be taken from the earliest report of the following month. Source: OCHA Humanitarian

Bulletin, Issues 133 inFerris E, Kirisci K and Shaikh S (2013) Syrian Crisis: Massive Displacement, Dire Needs

and a Shortage of Solutions, Brookings Institution, pp.iv


15/55


Syria which they considered provided more detailed information about needs and as a consequence

they referred less to the document now than they had in the early months when there was far less

information and analysis available.

Joint Rapid Assessment of northern Syria

SNAPs role in the J-RANS was to provide technical support both at the early stages of methodologydevelopment and assessment training, but also at the end of the process during the processing and

analysis of the data, an activity which SNAP took a lead on with a view to doing this in partnership

with the humanitarian community.

The J-RANS was viewed as an exceptionally important contribution particularly as at the time the

assessments were undertaken, there was an almost complete lack of analysis about needs in

northern Syria. The problems this lack of data posed for the humanitarian community were

considerable; while a small number of individual agency assessments had been conducted in

northern Syria, information only begun to be shared with the establishment of the Assessment

Working Group (AWG) hosted by the NGO forum which in OCHAs absence had taken on the role of

information management and coordination. Operational agencies spoke of a dearth of informationto target their programming decisions and donor representatives spoke of the challenges of seeking

to allocate funds without needs analysis information to support their recommendations.

While there was a general view that the J-RANS methodology was imperfect, given the constraints,

particularly security, it was considered to have yielded a level of data that few could have envisaged

being available and in terms of the humanitarian response, it is credited by many as the key to

unlocking significant donor funds at a time when the response was under-funded. For this reason it

is difficult to over-state the pivotal role that the J-RANS played. Beyond the relevance of the

assessments in leveraging funding, they also highlighted the needs across large parts of northern

Syria and in so doing provided the evidence that operational agencies needed to target their

interventions. In the same way as the RAS is considered a trusted source of analysis on thehumanitarian situation across the region, the J-RANS was considered by many to have shed much-

needed light on humanitarian needs in northern Syria and as such played an important role in

targeting assistance.

At the time it was released, the J-RANS was cutting edge and very relevant. Nothing like that

existed at the time and we used the analysis to inform our programme NGO sectoral

coordinator

The J-RANS provided the basis for dialogue on needs. This was an invaluable contribution.

Headquarters-based donor representative

For many agencies, particularly those working cross-border, the J-RANS has been the most relevantsupport provided by SNAP.

J-RANS I and II were invaluable to many NGOs and provided the cornerstone for many of our

projects. However since then it has gone quiet; nothing has filled the gapNGO Information

Management Officer, Turkey

While the assistance currently being provided by the Global Clusters to strengthen sectoral

leadership and coordination in Turkey may start to address this gap, there was concern about the

time that it could take to develop and refine sectoral analysis and the implications for an

understanding of needs.


16/55


Thematic Reports

The thematic reports tended to receive a more muted response during the fieldwork although the

results of the online survey suggest they have each filled important information gaps. Some of the

early thematic reports were considered to be very long which made them harder for users to digest

although they were welcomed by sector specialists. The more recent Syrian Border Crossings report

was extremely well-received and while one participant expressed concerns about the wisdom offocusing undue attention on unofficial border crossings, a far greater number of respondents

considered it offered the humanitarian community a common frame of reference and as such was

invaluable.

Scenario documents

There was a generally positive response to questions about the relevance of the scenario documents

from the majority of those interviewed with several NGOS saying that they had incorporated either

part or the whole of the document into their agencys scenario planning and as a consequence

considered the documents to be extremely relevant. While a few participants pointed to

discrepancies with an OCHA document released at a similar time, others considered this to be

positive as they were able to compare the documents and triangulate them against their own views.

The scenario documents have been useful as its helpful to get an outside perspective. Its

easy for an organisation to become blinkered.NGO Director, Turkey

The scenariodocument came out at the same time as OCHA released their document but we

thought this was helpful as it allowed us to triangulate information and analysis. Regional

NGO emergency coordinator, Jordan

With the humanitarian situation in the region being so dynamic, many agencies considered the

update of the scenario document a useful and timely exercise.

Evidence on use of SNAP products from the online surveyThe different ways in which SNAPs information and analysis products were used wereendorsed by

the results of the online survey (see figure 5 below). An analysis of the responses received offers

some interesting results and trends in how the products were used;

The J-RANS was the document used the greatest number of times by survey participants andoffered the broadest selection of uses which cut across operational and strategic purposes. J-

RANS I & II were the documents most frequently used to target programme assessments, to

support advocacy and for proposal writing.

The RAS was the second most frequently used SNAP product and was the document most oftenused by survey respondents for purposes of background reading. It was also the document most

often used for strategy development. It was used far less frequently to target needs assessment(16% of the respondents use the RAS for this purpose).

The scenario document was widely used to inform strategy development (by 33% of those whoparticipated in the survey) with a larger proportion using it as background reading (59%).

The thematic papers understandably appealed to the smallest audience with 62% ofrespondents indicating that they had used the Aleppo Governorate profile. Of the three papers

listed, the report on the Impact of the conflict on the Syrian Economy and Livelihoods was the

most frequently read with 64% of survey participants having used it as background reading and

20% using it to inform strategy.

The survey supports a view that in addition to providing important contextual knowledge for

humanitarian organisations, staff have used SNAPs information andanalysis for different purposes,often strategic, but also to inform operational decision-making.


17/55


Figure 5: Analysis of how SNAPs information and analysis has been used19

The results of the online survey also demonstrate the enduring relevance of SNAP to date with 63%

of the 94 respondents agreeing with the statement that SNAP has adapted its products over time

and in so doing has ensured its continuing relevance to the needs of humanitarian stakeholders.This

is important feedback for SNAP as the humanitarian community is now better organized than when

the project commenced and some of the more obvious gaps in information and analysis have now

been addressed. That is not to say that SNAPs existing products are insufficient to justify its

continued relevance, but it does suggest an important need to continue horizon-scanning for ways

that it can most effectively add value in the current situation (this is discussed in more detail in

section 9.1).

4. Connectedness

In the context of the SNAP review, the connectedness criterion seeks to assess the extent to which

SNAP has made links to and complements the work of other providers of information and analysis.

While the situation in and around Syria has presented a significant challenge to information and

analysis, there is evidence to suggest that capacity for analysis is improving as a result of a scale-up

in the capacity of traditional information management (IM) agencies in addition to an increase in

non-traditional providers. This has provided a complex web of agencies that are seeking to fill gaps

in different aspects of information collection, processing and analysis and at different levels. Arudimentary analysis of these agencies provides a mosaic of efforts (see figure 6).

1993 people responded to the question which provided 5 different uses of the data (including not used) and

allowed participants to select as many from the list as were relevant for each of the products listed


18/55


Figure 6: Analysis of primary data collection and analysis initiatives in support of humanitarian response

The figure seeks to identify stakeholders that are regularly providing information and analysis on the impact of

the Syrian conflict and makes a judgment on the nature of their activities (data collection through to data

analysis) and how these are used (from operational to strategic).20

In absolute terms, the provision of information and analysis has improved over time with both

traditional and non-traditional actors offering a far wider variety of products now than existed 9-

months ago. In addition to what is presented in the figure, there are also a number of confidential

briefings which are shared with only a select number of humanitarian agencies. While this may be

justified for reasons of operational security, from an information management perspective it makes

for a fractured analysis.

For certain parts of the response such as for the refugee response in Jordan, Lebanon and Turkey,

information about needs and assistance was considered to have improved significantly and

information-sharing between humanitarian partners is generally perceived as good. However, in

other areas, particularly inside Syria in both the north and the south interviews revealedsignificant gaps in knowledge about the humanitarian situation and while there have been some

notable successes to articulate country-wide needs (by the World Food Programme for example)

efforts to fill these gaps have not yet come to fruition.21

20The size of each of the stakeholders relative to each other is for presentational purposes only and does not

indicate importance. The position of each of the stakeholders should be viewed in the context of the two axes

rather than in comparison with others (i.e. SNAP, the International Crisis Group and Brookings each provide

analysis on the impact of the Syria conflict albeit with some primary data collection and their products are

primarily strategic in nature).21

For northern Syria, the ACU are in the process of establishing a Dynamic Monitoring System (termed

Dynamo) while in the South, OCHA has developed a joint humanitarian assessment (JHA) which aims to

provide a Syria-wide picture of humanitarian needs although this has been stalled pending final approval from

the government


19/55


4.1 SNAP information and analysis complementary to or duplicating the work of others?

The review found evidence that SNAP has been successful in positioning itself as a credible and

trustworthy user of data and as a consequence it benefits from information from a wide range of

agencies in support of its regular secondary data review. SNAP also plays a role in facilitating

information exchange, sharing raw data with a trusted group of NGOs, donors and OCHA for

purposes of input into its analysis. While figure 6 shows a relatively crowded space, a number ofthose agencies depicted have specific audiences (Integrity collect information on behalf of DFID,

Brookings and ICG have an objective of making policy change) or their information and analysis is yet

to be implemented (the ACUs DYNAMO is still under development and the JHA is yet to be

implemented) which leaves a relatively small number whose products are directed towards the

broader humanitarian community. Of particular importance is UNHCRs information management

role for refugee responses, OCHAs role for other affected populations, IMPACT which offers bespoke

information and analysis products to UN agencies and humanitarian organisations and SNAP. While

at a field level, information-sharing between these organisations has generally been effective, the

relationship between OCHA and SNAP has at times been strained, as outlined in an InterAction/ICVA

mission report from April 2013. Describing the creation of the NGO Forum in Antakya, Turkey, the

report explains that;

Supported by the Syria Needs Assessment Project (SNAP), the Forum has been able to

complete needs assessments and map data, a function that has enabled the NGO Forum to

play a strong leadership role in information management While relations are cordial

between OCHA and the NGO community, a lack of organisational trust was evident.22

While there was a concern from some members of OCHA that there was some duplication between

its own products and those produced by SNAP, the review failed to find significant evidence of this.

The online survey revealed a general level of satisfaction with SNAPs information and analysis which

was considered to be complementary to that of others (see figure 7).23

Figure 7: Results of the online survey opinion ranking exercise on complementarity/duplication

Of the 89 respondents to the online survey question, over 84% indicated that SNAP complemented

the work of others either extremely or moderately well, with only 2.25% considering that there was

22Interaction and ICVA (2013) Mission Report: An NGO Perspective on the Response to the Syria Crisis23

Staff from only a single agency felt that SNAP was duplicating the work of others during the field work that

accompanied the review

46.07

38.2

13.48

2.25

0 10 20 30 40 50

SNAP complements the work of otherinformation and analysis providers

SNAP complements the work of other

information and analysis providers

There is some duplication between SNAP

and other information and analysis

There is significant duplication between

SNAP and other information and analysis

% agreement


20/55


significant duplication. A selection of the comments provided by those participating in the survey

provides a more nuanced picture of the views of humanitarian stakeholders.

It provides a very useful complement to the official reports of others brings together in one

place news and analysis from various sources. Otherwise wed have to spend a lot more time

tracking down other resources.

SNAP adds a robust analysis that no one else seems to provide holistically.

Particularly for the opposition-controlled areas, SNAP is the only source of reliable and

comprehensive information.

The information/data of SNAP and other providers is relatively limited given access issues

and so it is not surprising that there is some overlap. I appreciate the triangulation that is

done for SNAP production and would suggest that it would be helpful to have a bit more

emphasis on government/regime perspectives and identification of priorities.

Efforts taken by SNAP to coordinate with others

SNAP has sought to coordinate with OCHAs Amman-based information and analysis unit by

including them as part of an Advisory Committee which provides strategic guidance to SNAP. In this

capacity they also receive SNAP products in advance of being published and hence can have input or

make comments on the documents prior to wider circulation. SNAP participate in the coordination

forums that are chaired by OCHA and there is regular contact between staff for purposes of

information exchange. To date there has been less formal coordination with IMPACT although

information is regularly shared on activities. This was an area that both agencies could seek to

strengthen in the future particularly as common areas of interest emerge.

Strengthening collaboration is an important issue for the future and the results of the survey provide

some important feedback on this; while the fragmented humanitarian architecture is not a result of

SNAPs presence, as an independent entitythat can work with humanitarian partners but which is

outside of the traditional humanitarian architecture, SNAP has provided important support to the

humanitarian community and has made up for some of its shortcomings through the information

and analysis it has provided. Despite this, the status quo of a fragmented humanitarian architecture

is not desirable in the long-term and a case was made by many of the review participants for the

humanitarian community working more closely together in the future.

The problem is the humanitarian architecture which is far bigger than SNAP; this needs to

work more effectivelyRegional UN agency representative, Jordan

There is little doubt that the more joined-up the humanitarian community is, the greater the

likelihood there is of achieving the best possible humanitarian outcomes for people affected by thecrisis in Syria. In progressing this important agenda, SNAPs efforts to collect, synthesize and analyse

data provides offers an important contribution. In this respect it is encouraging that OCHA, IMPACT

and SNAP are currently working together on developing a methodology for a proposed coordinated

needs assessment in northern Syria.

Recommendation: The review highlighted the importance with which many humanitarian agencies and

donors attach to a follow-up coordinated needs assessment for northern Syria. While a J-RANS III was a

strong recommendation of humanitarian agency staff at the time the review took place, progress has since

been made in developing a methodology for a comprehensive baseline assessment using a set of indicators

consistent with the JHA. SNAPs participation in this initiative will provide important support.


21/55


5. Coherence

In the context of the review, the coherence criterion speaks to the extent to which the constituent

parts of the SNAP project were clear and served to strengthen each other.

SNAPs outcome statement speaks of improved targeting of response and the creation of a sharedsituational awarenessand it is evident that all aspects of SNAPs work are consistent with these

ambitions. While this represents a success of the project and suggests that SNAP has performed well

in seizing opportunities that fit with its mandate, it also presents a challenge for SNAP in the future

as it has relied on strong leadership and relationships within the team in order to identify and exploit

relevant opportunities as they arise. This has been a strength of the project but it also could have

been a weakness as there was a risk that a continual search for relevance that could have taken it in

divergent directions that could have been to the detriment of the project. However, the RAS has

provided an important anchor for SNAP and has allowed it a degree of freedom to innovate. The

most significant innovation was in participating in the J-RANS and in doing so, taking on a support

role for primary data collection which was a departure from ACAPS distinctive competencies in

secondary data review and analysis. While there was the potential for SNAP to get drawn intooperations, it did well to maintain a capacity building and analysis role and in facilitating the

collection of data through coordinated needs assessment went considerable distance to furthering

SNAPs core purpose of creating a shared situational analysis.

As SNAP approaches the end of its first year, given the relatively high turnover of staff at key points

in time and the impact this has had on progress, there would be significant benefit from seeking to

more clearly articulate potential directions for SNAP, the possibilities that exist to build on its

existing portfolio of products, how it will seek to work with others and the identification of a set of

principles or parameters to guide its work as a means of future-proofing itself against potential

staffing changes that may compromise institutional memory. A written strategy would also serve to

militate against the potential for SNAP to lose focus and expand into areas that are outside itsinterests.

Recommendation: A written strategy which articulates SNAPs vision and identifies a set of principles or

parameters to ensure the coherence of its work would future-proof it against a loss of institutional memory

as a result of staff changes and militate against the potential for SNAP to expand into areas that are outside

its interests

6. Effectiveness

Effect iveness measures the extent to which an act iv i ty achieves i ts pu rpose, or whether th is

can be expected to happen on th e basis of the outpu ts. Impl ic i t w ith in the cr i ter ion ofef fect iveness is t imel iness.

246.1 Progress of SNAP against its objectives

Figure 8 (below) provides an indication of SNAPs performanceagainst its objectives and its progress

against the milestones outlined in the logical framework.

24Source: ALNAP (2006) Evaluating humanitarian action using the OECD-DAC criteria: An ALNAP guide for

humanitarian agencies, Overseas Development Institute, London


22/55


Figure 8: Summary table of progress against third quarter (Q3) milestones

Expected outputs Indicator Q3

milestone25

Progress against milestones

Output 1: Existing assessment

data is analysed and fed into

relevant coordinated

mechanisms with gaps and

priorities mapped and updated

narratively and visually at

country and regional level.

Common operational

understanding of

affected groups

conditions through

objective information

gap and needs analysis

9 general

analysis

briefs

Regional analyses (RAS) have

been published monthly since

project commencement. 9

RAS published by Q3

Output 2: Tracking of key issues

including priority needs, lessons

learned, information gaps

through continuous secondary

data analysis and technical

support to assessments in Syria

and neighbouring countries.

Improved targeting and

resource allocations for

specific needs of

different affected

groups supported

thematic analysis.

7 sectoral /

thematic

reports

5 thematic reports published

by Q3 including priority needs

(Aleppo Gov. profile),

assessment lessons learned

and papers on economy and

livelihoods, legal status of

refugees and border crossings

Output 3: Develop scenarios

from information gathered and

lessons learned to feed into

contingency plans as well as

other planning processes.

Contingency planning

processes and

preparedness measures

are better informed by

raised situational

awareness.

2 scenario

exercises

2 scenario exercises published

by Q3 and evidence of use by

agencies for contingency

planning purposes

Output 4: Technical assessment

advice and leadership given to

the Assessment Working Group

in Turkey and other stakeholders

in the region to enable the

collection and analysis of

primary data from Syria

Appropriate assessment

reports to enable

response planning in

Syria and improved

common operational

understanding of

priority needs and gaps

Inter-agency

rapid

assessment

of 7

northern

governorate

s, 1 urban

assessment

for Aleppo

town

J-RANS I, I.5, II undertaken

and results published and

used widely by the

humanitarian community for

targeting assessments and

operations, fundraising and

advocacy

Technical capacity is

brought to and available

to stakeholder forums.

(number of meetings)

Technical

capacity

made

available at

9 meetings

10 NGO Assessment Working

Group meetings (in Antakya,

Turkey) chaired and advised

12 NGO Coordination

meetings (in Turkey) attended

and assisted as required

8 Information Management

and Assessment Working

Group meetings attended and

assisted as required

The outputs that have been achieved are in line with expectations and have been highly valued by

many within the humanitarian community with feedback during the review showing a high degree of

satisfaction with their quality. Interviews conducted during the review, responses from the online

survey and download data from Reliefweb and the ACAPS website suggest that the documents are

widely read and broadly trusted which suggests that progress is being made against SNAPs

ambitions to create a shared understanding of needs.

25Q3 milestones and progress taken from the SNAP Q3 report to DFID, September 2013


23/55


6.2 An analysis of risks and assumptions

The risks and assumptions outlined in the log frame are worthy of brief discussion as they have at

times played an role in enabling or inhibiting progress to be made.

Assumption 1: Reputation of ACAPS and MapAction as reliable and non-operational partners

Both ACAPS and MapAction have significant experience supporting humanitarian partners instrengthening information management and analysis in a range of humanitarian environments. Their

reputation has played an important role in providing reassurance of the professionalism and

integrity of SNAP which has played a role in developing relationships of trust with humanitarian

partners.

Of note here is that both ACAPS and MapActions model of deployments is usually short-term and at

times it has been difficult to recruit to the longer-term profile required by the SNAP project. High

staff turnover has eroded the trust that has been essential to the success of the project which has

had implications for information-sharing and for retaining the projects agility to make the best

possible use of opportunities. In saying this its also important to note that the continuity in SNAPs

leadership in the region and the credibility this has bought the project has also played an importantpart in its success.

Linked to the issue of continuity of staffing is the issue of how the project has been referred to

during its lifespan particularly in Turkey where ACAPS and MapAction deployments were subsumed

into SNAP several months into the project and several agencies commented on the confusion that

this caused. While the SNAP brand is now far better known, the use of the ACAPS and MapAction

logos on the products and the use of the ACAPS mailing address for staff and for circulating the

products has caused confusion in the past.

Assumption 2: Cooperation and willingness of agencies to share information and analysis

Given the operational sensitivities of working in Syria and cross-border from neighbouring countriesand the concomitant complexities of information-sharing, there was the potential for SNAP to fail to

have access to key assessments and analysis which would have compromised the quality of its work.

While SNAPs establishmentwas supported by a small group of NGOs which it initially relied on to

provide it with information and to facilitate contact with other humanitarian agencies. Since this

time the team has been extremely successful in making its own contacts and in demonstrating

SNAPs added value.

It is of interest that some of the less supportive review participants considered that SNAPs analysis

was based on information available in the public domain. This is a fallacy and the trust that many

agencies, particularly NGOs, have in SNAP and the responsible way in which sensitive information

has been handled has resulted in SNAP having access to data that is not shared with others. This hasits challenges, particularly in how to address issues of attribution and several review participants

raised concerns about SNAPs use ofclosed source information.26

It is the view of the reviewer that

this is necessary given the operational security implications of sharing sensitive information.

That is not to say that SNAP has optimal access to information and there are a number of agencies

that choose not to share assessment information with the team which includes a number of

Damascus-based organisations (UN, a number of the INGOs and the Syrian Arab Red Crescent) which

reflects the disconnect between insiders and outsiders that is discussed in section 1.3

26Throughout SNAPs products, information sourced from personal interviews with persons unknown to the

SNAP project are referred to by the descriptor PI. Information sourced as a Trusted Source refers to

information received from an actor known and trusted by the project.


24/55


Assumption 3/4: Access to humanitarian actors/assessment forums continue to exist for the purpose

of harmonised needs assessment

With the exception of a handful of agencies, SNAP has benefitted from good access to humanitarian

actors and donor agencies through the coordination forums which now exist in the countries outside

Syria and through direct contact between SNAP analysts and agency representatives inside Syria.

Where coordination has been slow to be established, SNAPs analysts have played a particularlyimportant role in collecting information and analysis about the situation. As has been noted

elsewhere in this report, face-to-face contact between SNAP and the humanitarian community plays

a particularly important role in building trust and facilitating the collection and use of assessment

information.

In providing feedback on the breadth of SNAPs information gathering and secondary data review,

several respondents spoke of a perception that the RAS was compiled from international or western

data sources as opposed to Arabic or Syrian sources. While SNAP does draw its information and

analysis from a diversity of sources and has a number of Arabic-speaking analysts on the team, there

would be value exploring the reasons for this perception and also reflecting on whether SNAPs

analysis is primarily targeted towards an international/western audience or whether theres anambition for it to have greater influence and a larger audience outside of the international/English-

speaking humanitarian community.

Recommendation: SNAP should clarify who its target audience is and tailor and disseminate its products

appropriately. If it does want to have the potential for greater influence with national and regional

organisations then there will be a need to review its translation and dissemination strategy to ensure the

SNAP products are more easily accessible to non-English-speakers and adequately incorporate regional

perspectives

6.3 SNAPs contributionto strengthening targeting andpromoting a shared situational analysis

At the outcome level, the purpose of SNAP is that populations affected by the Syria crisis benefit

from improved targeted response provided by humanitarian stakeholders operating with a shared

situation awareness.

Reflections from review participants

Any analysis of the extent to which SNAP has strengthened targeting and contributed to a shared

situational awarenessis beset with problems of attribution, but the review highlighted examples of

how SNAPs work has contributed to meeting its purpose. While the majority of respondents

considered that the RAS was of too high a level of extraction to be of use in targeting assistance, the

J-RANS was cited as having contributed to the decision-making of humanitarian agencies on

geographic and sectoral targeting.

While the RAS is too big-picture for operations, the J-RANS I and II directly informed

humanitarian programmes. UN agency representative, Turkey

It is important to note that as a coordinated multi-sector needs assessment the J-RANS did not

always provide the detail required to initiate operations and some agencies spoke of a subsequent

need to reassess according to more detailed sectoral methodologies. There was also feedback

provided about the time it took to process, analyse and present the assessment data which some

felt was too slow given the important need to mobilise the humanitarian community and respond

quickly to the needs that had been identified.

The food security information [from J-RANS II] was a good indicator of the needs and helpedus target more detailed assessments. Food security sector representative, Turkey


25/55


The time lag between the assessments and the findings was too long; a month is a long time

to wait and in such a dynamic situation had implications for the operational use of the data

NGO Team leader, Turkey

The RAS, thematic reports and scenario documents have also played an important role in

contributing to a shared situational analysis; there was almost universal appreciation of the role the

RAS played in offering a comprehensive regional overview of the humanitarian situation. The

majority of NGOs, UN agencies and donor representatives interviewed in Turkey, Jordan and

Lebanon considered that it offered a level of coherence in its analysis that could not be found

elsewhere. While there were some conflicting recommendations about the length of the document

(having a shorter document versus not wanting to lose any of the analysis) and also about the

frequency with which it is published (having a quarterly RAS with shorter updates in the months

between versus wanting to maintain a monthly RAS), there were no such concern about its overall

impact as a monthly regional humanitarian digest.

As a Desk Officer, I am tasked with piecing together the information on the situation and

triangulating what we know. The RAS pieces it all together and is perfectly credible.

Headquarters-based NGO desk officer

For modest-sized donors like ourselves, there is no other way we would have access to this

sort of information if it wasnt for SNAP Headquarters-based donor representative

The RAS is the only analysis available that covers all sectors for all countries affected by the

crisis. Theres no other strategic document of this type NGO director, Jordan

The high level of credibility of the RAS with humanitarian agencies and donors means that it has

played a valuable role in influencing opinion and advocacy, informing funding decisions and

supporting the development of humanitarian strategy. That is not to say that the RAS is read from

cover-to-cover by all its users and the review highlighted a multitude of ways in which users engagedwith the document which tended to change with proximity to the crisis:

Project staff with a detailed knowledge of a specific sector or geographic area used the RAS lookoutside of their area of technical/geographic specialism

Heads of agencies working at a country- or regional-level tended to skim the RAS, looking fornew information. The longer the person had been deployed in the region, the less they tended

to refer to the document.

Headquarters staff tended to work through the document in more detail and considered it anessential monthly digest of the humanitarian situation.

Country, regional and headquarters-based advocacy staff considered the RAS provided animportant baseline, they appreciated the web-links to the sources where these were madeavailable and considered it a trusted and credible source of data.

The RAS has also been an important vehicle for building trust and respect for SNAP and has created a

platform to launch other products. A credible product has played a key role in getting stakeholder

engagement.

The use of SNAP information and analysis by advocacy and media organisations

An indicator for the effectiveness of SNAP is the use of its information and analysis by third parties in

support of policy and advocacy work which provides an indication of its credibility. Given that such

documents are generally targeted at policy-makers and are shared widely, this also contributes to

creating a shared situational awareness. While a comprehensive review of citations was not

undertaken, an analysis of several recently-published policy documents and use of SNAPs products


26/55


(and products supported by SNAP such as the J-RANS) in the media provides an indication of how

the information, analysis and mapping data have been used (see figure 9).

Figure 9: SNAP citations in recently published policy and advocacy documents27

Author Title SNAP document(s) cited in the report Date

BrookingsInstitution

Syrian Crisis: Massivedisplacement, dire

needs and a shortage

of solutions

J-RANS II, June 2013RAS, May 2013

RAS, June 2013

RAS, July 2013

Thematic report, Syrian Border Crossings, Sep 2013

18 Sep 13

Refugees

International

Aid inside Syria: Too

little but not too late

J-RANS I, January 2013

Aleppo Governorate profile, March 2013

25 Apr 13

United Nations

General

Assembly

Situation of internally

displaced persons in

the Syrian Arab

Republic

RAS, May 2013

J-RANS II, May 2013

15 Jul 13

Not disclosed Open Letter: Let us

treat patients in Syria

Aleppo Governorate profile, March 2013 16 Sep 13

Economist The regime digs in Conflict map reproduced

http://www.economist.com/news/middle-east-and-

africa/21579494-president-bashar-assad-and-his-

forces-have-won-new-lease-life-regime-digs/print

15 Jun 13

BBC Mapping the conflict Conflict map reproduced

http://www.bbc.co.uk/news/world-middle-east-

22798391

5 Sep 13

6.4 Timeliness

The timeliness criterion was of greatest importance in the early stages of SNAPs deployment at the

end of 2012 at which time the humanitarian community had very limited access to large parts of

Syria and as a consequence information was very poor. This coincided with significant increases in

the level of humanitarian needs as a consequence of an escalation in the conflict within Syria which

caused additional displacement and which initiated the flight of conflict-affected people into

neighbouring countries (see figures 3 and 4).

SNAPs arrivaland publication of the first RAS in January 2013 was undoubtedly well timed and by

providing the first comprehensive analysis of the humanitarian situation early editions of the report

were considered by many participants of the review to have made an extraordinary contribution.

Donors spoke of the importance of having access to a document that was comprehensive, that

provided big-picture analysis and that they could trust. The lack of such analysis had been a key

driver in DFIDs decision to fund SNAP and in large part it delivered what they needed to justify their

funding; representatives from other donor agencies, senior humanitarian and policy staff trying to

understand and communicate the scale of the crisis gave a similar account of the importance of the

analysis in the early months of the project.

With the passing of time, feedback is now more mixed about the timeliness of successive RAS

publications. While many still consider the monthly report an important barometer of change and

value the consistency of publication which permits the identification of trends, some others who

were either closer to the situation or had access to their own sources of information tended to feel

that a less frequent quarterly publication would be more appropriate.

27Each of the documents studies is fully referenced in the bibliography
http://www.economist.com/news/middle-east-and-africa/21579494-president-bashar-assad-and-his-forces-have-won-new-lease-life-regime-digs/printhttp://www.economist.com/news/middle-east-and-africa/21579494-president-bashar-assad-and-his-forces-have-won-new-lease-life-regime-digs/printhttp://www.economist.com/news/middle-east-and-africa/21579494-president-bashar-assad-and-his-forces-have-won-new-lease-life-regime-digs/printhttp://www.economist.com/news/middle-east-and-africa/21579494-president-bashar-assad-and-his-forces-have-won-new-lease-life-regime-digs/printhttp://www.bbc.co.uk/news/world-middle-east-22798391http://www.bbc.co.uk/news/world-middle-east-22798391http://www.bbc.co.uk/news/world-middle-east-22798391http://www.bbc.co.uk/news/world-middle-east-22798391http://www.bbc.co.uk/news/world-middle-east-22798391http://www.economist.com/news/middle-east-and-africa/21579494-president-bashar-assad-and-his-forces-have-won-new-lease-life-regime-digs/printhttp://www.economist.com/news/middle-east-and-africa/21579494-president-bashar-assad-and-his-forces-have-won-new-lease-life-regime-digs/printhttp://www.economist.com/news/middle-east-and-africa/21579494-president-bashar-assad-and-his-forces-have-won-new-lease-life-regime-digs/print


27/55


It is difficult to overstate the timeliness of the information that came from the two J-RANS that were

undertaken. Review participants spoke of a multitude of uses of the information; donors used it to

justify programming their funds; policy analysts used the figures to strengthen their messages about

the severity of the crisis and its impact on the Syrian people; NGOs used the data to justify their

programmes to donors and agencies working in northern Syria or who wanted to work in northern

Syria used the data as a basis to target their operations or to advocate on the importance ofinitiating operations. While some review participants quibbled over some aspects of the

methodology, there was consensus over the timeliness of the exercise as a means of promoting a

shared analysis of the situation and in galvanizing much-needed action.

Feedback wasnt so unequivocal for other SNAP products although the scenario document released

in February 2013 was considered to have come at a time when a number of agencies were seeking

to strengthen their contingency planning and so were well-received. The second scenario document

was similarly well-timed, being released at a time of heightened concern over security but its

publication soon after a similar OCHA document did result in mixed feedback both about the utility

of the document (some felt it provided important balance while others thought it served to

duplicate what already existed) and as a consequence, the timeliness also.

Of the thematic reports, the greatest number of comments was received about the Border Crossings

document. Agencies were very positive about the timeliness of the publication which documented

official and unofficial border crossings some of which had no commonly agreed name and in so

doing SNAP provided a common language for the humanitarian community.

As an overall comment on the timeliness of key products, SNAP has performed well in providing

information and analysis that are considered to have filled important gaps in a timely manner. The

challenge for SNAP in a humanitarian community that is now more mature and better organized

than it was earlier in the year is whether it can continue to identify opportunities and exploit gaps in

knowledge in such a timely way as it did for its earlier products such as the J-RANS.

6.5 The value of independent analysis in Syria

Perceptions of SNAPs independence were explored inthe online survey which asked the question,

to what extent do you consider that the J-RANS and RAS are independent or free from significant

political bias.Respondents were requested to assign a score of 1 to 5 with 1 being independent and

5 being not independent at all. The results are presented in figure 10. Over 60% of survey

respondents considered the J-RANS and RAS as broadly independent28

with approximately 10%

assigning a score which suggested concerns about a lack of independence.29

The figures from the survey are best interpreted in the context of a discussion about the

comparative independence of different sources of information and analysis. As outlined in the firstsection of this report, there are numerous challenges to gathering data on humanitarian needs in

Syria which in large part provided the raison detre for SNAP as a vehicle fo

mid-term review of the syria needs analysis project

Documents