Towards assessing CFS effectiveness Technical Consultative Workshop 14-15 April 2014, FAO Headquarters, Rome

Workshop Report

1. Background

A two-day technical consultative workshop was held at FAO Headquarters in Rome on

14-15 April 2015, as a follow up to one of the recommendations of the CFS at its 40th

session in October 2013. The CFS recommendation was to conduct periodic

assessments of CFS effectiveness in improving policy frameworks and in promoting

participation and coherence among stakeholders on food security and nutrition.

Specifically, it was recommended that a survey should be carried out to assess the

current situation, in order to identify gaps, constraints, success factors and possible areas

to be monitored as the base for assessing future progress.

2. Objective of the workshop

The objective of the workshop was to obtain technical expert advice and inputs for

designing a baseline survey that will set the stage for assessing CFS effectiveness. In

particular expert inputs were requested on:

(i) the key elements to consider in the baseline survey;

(ii) the options for collecting data, considering different parameters;

(iii) the sample of the survey.

A Technical Note was sent to the participants prior to the workshop, providing

background information on the workshop and the proposed approach to develop the

baseline survey. The Note is provided in Annex 1.

3. Participants

Participants included 15 technical experts on monitoring and evaluation of food security

and nutrition issues from CSO/NGOs, private sector, UN and development agencies, as

well as the CFS Secretariat and the Technical Support Team. The list of participants is

provided in Annex 2.

4. Methodology

Most time was devoted to discussions in working groups, followed by reporting and

discussion in plenary. Participants were divided in three working groups and remained

in the same working groups for group work on Day 1 and Day 2. Each working group

2

focused on one of the three outcomes of the CFS1. The group work on Day 1 focused on

the key elements to consider in the baseline survey while the group work on Day 2

focused on the data collection methods and the survey sample. Each working group was

asked to fill a matrix with all the inputs required to achieve the objective of the

workshop. Some criteria, key questions and indicators were suggested in the matrix but

the participants were free to modify, cancel or consolidate them.

The agenda of the workshop is provided in Annex 3. The instructions provided for

group work 1 and 2 are provided in Annex 4.

4. Results

The participants achieved most of the expected results, in particular regarding the scope

and content of the baseline survey. Specifically, participants provided inputs on the

criteria for assessing effectiveness, the high level questions to be answered by the

survey, indicators, information required to inform the high-level questions and the

indicators and the information sources.

Participants also provided a number of inputs on the data collection methods,

considering different parameters (cost, reliability, accuracy, etc.) and the sample survey

(selection criteria for selecting respondents, taking into account national, regional and

global levels, sample size, etc.).

The matrices completed by the three working groups are provided in Annex 5.

Some suggestions made by participants in the wrap-up session were the following:

- In the absence of a complete results chain for CFS, the M&E framework will have to

rely on a number of assumptions

- All three outcomes should be covered by the assessment of CFS effectiveness

- The M&E framework should integrate rights-based indicators,

- Some of the information identified in the matrices is already being collected by other

monitoring systems; therefore, the focus should be on assessing CFS’ effectiveness

in fulfilling its specific roles.

- The stakeholder surveys should be complemented by in-depth county level

assessments.

- The stakeholder survey should include CFS members and stakeholders.

- The FNS Forum should be considered as a source of information for the in-depth

county level assessments

- A panel of countries should be surveyed/assessed, on a voluntary basis

1 CFS overall objective is to contribute to reducing hunger and malnutrition and enhancing food

security and nutrition for all human beings. The three outcomes are the following: CFS Outcome A: Enhanced global coordination on food security and nutrition questions CFS Outcome B: Improved policy convergence on key food security and nutrition issues CFS Outcome C: Strengthened national and regional food security and nutrition actions

3

- The M&E exercise should be considered as an exercise to increase learning and CFS

awareness

- The proposed assessments should be piloted before they are implemented

- The baseline assessments should be carried out within the context of a broader M&E

framework

- Guidelines for implementing the assessments at country level should be developed

- In-depth county level assessments should be carried out by multidisciplinary teams

- CFS effectiveness assessments should focus note only on CFS platform capacities

but also on CFS members and stakeholders capacities

5. Evaluation of the workshop

Participants were asked to fill a form at the end of the workshop to evaluate different

aspects of the workshop (organization, content, facilitation, participants, workshop

results). The results are presented in Annex 6.

Overall, the participants expressed their satisfaction with the workshop. The level of

participation was found to be very high. Participants were also satisfied with the

achievements of the objectives of the workshop and the results obtained.

4

Annex 1: Technical Note

Technical Note

Introduction

1. The Committee on World Food Security (CFS) went through an extensive reform process in 2009 to enable it to more fully play its role in the area of food security and nutrition2. The CFS reform took place in the wake of rising food prices, financial and economic crises, increasing climate variability and extreme weather events that impact livelihoods, coupled with weak governance structures for food security and nutrition, which combined, highlighted the persistent and unacceptable levels of structural poverty and hunger in the world.

2. One of the roles identified in the CFS Reform document was to promote accountability and share best practices at all levels, where CFS should “... help countries and regions, as appropriate, address the questions of whether objectives are being achieved and how food insecurity and malnutrition can be reduced more quickly and effectively. This will entail developing an innovative mechanism, including the definition of common indicators, to monitor progress towards these agreed upon objectives and actions taking into account lessons learned from previous CFS and other monitoring attempts. Comments by all CFS stakeholders will have to be taken into account and new mechanisms will build on existing structures” (CFS: 2009/2 Rev.2, para6ii).

3. In January 2012, the CFS Bureau established an Open-Ended Working Group on Monitoring (OEWG-Monitoring) to help advise the Committee on how best to respond to this challenge3 (see Annex 1). Based on the work of the OEWG-Monitoring and subsequent discussions in CFS, the monitoring function in the context of CFS comes down to essentially two main areas. It is firstly about how to monitor CFS decisions and recommendations, to determine how well the Committee is meeting its overall objective of contributing to the improvement of food security and nutrition at various levels. Secondly, it is about recommending approaches to monitoring by Member Countries, sub-regional and global bodies in order to promote more accountability and improvement in addressing food security and nutrition programme delivery.

4. At the 40th session of CFS in October 2013, in the context of CFS monitoring, the Committee underlined the important role of CFS as a platform for stakeholders to regularly share experiences and practices on monitoring work in strategic areas at all levels (global, regional and national) and

2 CFS:2009/2 Rev.2 - Reform of the Committee on World Food Security final version -

http://www.fao.org/fileadmin/templates/cfs/Docs0910/ReformDoc/CFS_2009_2_Rev_2_E_K7197.pdf 3 CFS:2012/39/9 - Information Note on Monitoring CFS Decisions and Recommendations -

http://www.fao.org/docrep/meeting/026/me557e.pdf

Towards assessing CFS effectiveness Technical Consultative Workshop 14-15 April 2014, FAO Headquarters, Rome

5

the need to use monitoring and evaluation (M&E) to improve the work of the CFS, including the formulation of future CFS recommendations. One of the recommendations endorsed at CFS 40 was to conduct periodic assessments of CFS effectiveness in improving policy frameworks, especially at country level, and in promoting participation of and coherence among stakeholders on food security and nutrition. Specifically, CFS 40 recommended carrying out a baseline survey to assess the current situation as the base of assessing progress4.

5. In response to the above, the CFS Secretariat is holding a technical consultative workshop at FAO Headquarters in Rome from 14-15 April 2014 to share technical expert advice in designing a baseline survey towards assessing CFS effectiveness. It is recognized that this baseline survey exercise represents only a first step towards the identification and development of a broader CFS M&E framework. The intention is to develop/propose a baseline survey methodology for consideration by the OEWG-Monitoring in May 2014 and subsequently, to submit the proposal to CFS 41 in October 2014 for endorsement.

6. This technical note serves to provide background and facilitate discussions at the consultative workshop with respect to designing a baseline survey to set the stage for assessing CFS effectiveness. In this regard, further advice and clarity are needed, especially with respect to:

a) What are the key elements or dimensions to consider in the baseline survey?

b) Who is the target for such a survey and what sample size to consider?

c) How should this survey be implemented – what methodologies to consider for collecting and analyzing the information?

7. Section 1 of this document provides a brief context on the assessment exercise to be designed along with the steps to keep in mind while designing a survey. Section 2 briefly describes the vision and objectives of CFS, its composition, its work modalities, and the types of outputs it delivers/produces to highlight the complexities and challenges inherent in this exercise in the context of CFS. A working definition of “effectiveness” in the context of CFS is proposed in Section 3, as well as a list of possible elements or dimensions to be considered in developing the baseline survey methodology. Section 4 raises some of the critical issues to be taken into account in selecting the appropriate assessment approach and the modalities for its implementation. A brief summary of the evolution of CFS M&E discussions since 2012 leading to the CFS 40 decisions is provided in Annex 1. Definitions of key M&E terms are provided in Annex 2.

1. Towards the Development of an M&E Framework for CFS

8. The design and implementation of a baseline survey to assess the effectiveness of CFS has two main purposes. On the one hand, the survey should provide a reference base (baseline) for assessing CFS effectiveness over time. On the other hand, the results of the survey should help to inform the relevant areas and approaches on which an M&E framework for CFS should focus, according to mechanisms and arrangements to be agreed by all CFS stakeholders.

4 Ref. page 10 of CFS 2013/40 REPORT - http://www.fao.org/docrep/meeting/029/mi744e.pdf

6

9. It is important to emphasize that the implementation of such a baseline survey will be a first attempt towards assessing the effectiveness of CFS. Therefore, it is expected that the learnings generated from the implementation of the baseline survey will also support CFS and its stakeholders to better understand the concept of “CFS effectiveness” and how the Committee can collectively learn from the current situation to improve how it does its work.

10. It is also useful to keep in mind the basic steps that are usually involved in designing a baseline survey, which include:

1. understanding the context of the assessment and the key assessment questions (and/or indicators);

2. defining what information to collect and sources (primary/secondary); 3. identifying the assessment sample; 4. selecting the techniques and designing the tools for gathering data; 5. defining the methodologies for data processing and analysis; 6. establishing the reporting and dissemination modalities.

2. Overview of CFS vision, objectives, subsidiary bodies and composition

11. The vision of the CFS, as stated in the CFS Reform document is that “CFS constitutes the foremost inclusive international and intergovernmental platform for a broad range of stakeholders to work together in a coordinated manner towards the elimination of hunger and ensuring food security and nutrition for all human beings. The CFS will strive for a world free from hunger where countries implement the voluntary guidelines for the progressive realization of the right to adequate food in the context of national food security”.

11. To assist the Committee in guiding its work towards achieving this vision, at its 37th session, in October 2011, CFS approved a draft result-based framework (RBF), which included an overall objective and three outcomes, in accordance with previous recommendations of the Committee and based on the roles identified for CFS in the CFS Reform document. In addition, the Committee requested the CFS Bureau to work with the Secretariat to further integrate the 2012-2013 Programme of Work and Budget (PWB) with the results-based framework with a view to preparing a more detailed and prioritized Multi-Year Programme of Work (MYPoW)... The 2014-2015 MYPoW re-emphasized the overall objective and three outcomes and further refined major CFS workstreams and activities to support these objectives and outcomes5.

12. The CFS Overall Objective and expected Outcomes are provided, along with a brief paragraph describing each outcome, in the MYPoW as it follows:

CFS Overall Objective: Contribute to reducing hunger and malnutrition and enhancing food security and nutrition for all human beings

Progress towards of the CFS Overall Objective is expected to be made through the contribution of the following three expected Outcomes.

CFS Outcome A: Enhanced global coordination on food security and nutrition questions

CFS Outcome B: Improved policy convergence on key food security and nutrition issues

CFS Outcome C: Strengthened national and regional food security and nutrition actions

5 CFS Multi-Year Programme Of Work (MYPoW) 2014-2015

http://www.fao.org/fileadmin/templates/cfs/Docs1213/PWP-Programme_of_Work_and_Priorities_/MI036_CFS2013_40_9_Rev1_formatted_en_COMPILED.pdf

7

With regard to Outcome A, the CFS role of global coordination is to provide an inclusive and evidence-based platform for discussion and coordination to strengthen collaborative action among governments, international and regional organizations, CSOs, the private sector and other relevant stakeholders, in a manner that is in alignment with country needs. This role is conducted mainly through discussions at the CFS Plenary session, including examination of food security and nutrition initiatives and frameworks, and inter-sessional activities that support the work of the Committee. The outcome considers not only coordination within CFS, but also the way CFS works with other important global and regional fora and initiatives. Coordination can also serve to encourage a more efficient use of resources and the identification of resource gaps.

Under Outcome B, the CFS role in policy convergence is achieved through the formulation of policy recommendations, the development of international strategies and guidelines and other policy frameworks, based on best practices, lessons learnt, inputs from the national and regional levels and expert advice and opinions from different stakeholders. Policy convergence will include greater integration and coherence horizontally (among countries, organizations, stakeholders, etc.) as well as vertically (from local to global levels and vice versa). This outcome is further supported by the development of a CFS communication strategy that will aim to sensitize the decision-makers to the CFS recommendations and by the CFS Chair’s attendance to key fora.

With respect to Outcome C, the role of the CFS in facilitating support to national and regional food security and nutrition plans (i.e. policies, programmes, other actions, etc.) includes support/advice on development, implementation, monitoring and evaluation of plans to eliminate hunger and achieve food security and nutrition, based on the principles of participation, transparency and accountability. Progress on this outcome will also be a function of the responses provided by CFS to countries and regions and of the adoption of advice, tools, methods and frameworks that support coordinated responses resulting from CFS actions.

13. The implementation of CFS workstreams leads, among others, to the endorsement of a wide and diverse range of decisions and recommendations belonging to the following three categories: 1) CFS Products; 2) CFS Policy Recommendations; 3) Process-related recommendations. Details on that categorization are provided in Annex 1. 14. It is worth mentioning that CFS decisions and recommendations are not legally binding and their implementation at country level is on a voluntary basis. Nevertheless, the participation of representatives of national governments along with the multiple CFS stakeholders in the CFS debates should enable positive change and improvements at regional and country levels. 15. The diagram below graphically illustrates how the CFS Outcomes are expected to contribute to positive change at global, regional and national levels.

8

16. Regarding the composition of CFS and its work modalities, the Reform Document indicates that: “CFS is an intergovernmental Committee… composed of members, participants and observers and will seek to achieve a balance between inclusiveness and effectiveness. Its composition will ensure that the voices of all relevant stakeholders – particularly those most affected by food insecurity - are heard”. Members, Participants and Observers contribute to CFS workstreams according to different roles and responsibilities. CFS Members6 take part fully in the work of the Committee, approve meeting documents and agendas, submit and present documents and formal proposals, and interact with the Bureau during the inter-sessional period. Voting and decision-taking is the exclusive prerogative of Members, including drafting the Final Report of CFS Plenary Sessions. CFS Participants7 take part in the work of the Committee with the right to intervene in plenary and breakout discussions, to contribute to the preparation of meeting documents and agendas, submit and present documents and formal proposals. They commit to contribute regularly to inter-sessional activities of the Committee at all levels and interact with the Bureau during the inter-sessional period through the Advisory Group established by the Bureau. CFS may invite other interested organizations relevant to its work as Observers of entire sessions or specific agenda items. 17. The bodies of CFS are:

Bureau and Advisory Group. The Bureau is the executive arm of CFS. It is made up of a Chairperson (elected from Member countries) and twelve Member countries. The Advisory Group helps the Bureau advance the Committee’s objectives. It is made up of representatives from the following 5 categories of CFS Participants: 1) UN agencies and other UN bodies; 2) Civil society (CSOs) and non-governmental organizations (NGOs); 3) International agricultural research institutions; 4) International and regional financial institutions; 5) Private sector associations and philanthropic foundations.

6 CFS Members include all Member States of FAO, IFAD or WFP and non-Member States of FAO that are

Member States of the United Nations. 7 CFS Participants can include representatives from UN agencies and bodies, civil society and non-

governmental organizations and their networks, international agricultural research systems, international and regional financial institutions and representatives from private sector associations and private philanthropic foundations.

9

Plenary. The Plenary Session is held annually and is the central body for decision-taking, debate, coordination, lesson-learning and convergence by all stakeholders at a global level on food security issues.

High Level Panel of Experts (HLPE). The HLPE comprises the Steering Committee, made up of internationally recognized experts in a variety of food security and nutrition-related fields and ad hoc project teams. The HLPE provides scientific knowledge-based analysis and advice. HLPE reports are produced independently from CFS and its findings and recommendations serve as a basis for CFS policy debates.

CFS Joint Secretariat. It is a Joint Secretariat made up of staff from the UN Rome-based agencies (FAO, IFAD and WFP). Its task is to support the Plenary, the Bureau and Advisory Group and the HLPE in their work. The Secretariat is hosted by FAO at its headquarters in Rome.

18. The above information provided a brief overview of CFS to help understand the complexity and specificities of CFS, which is crucial to keep in mind when designing the baseline survey. The following section proposes a working definition of the concept of “CFS effectiveness” and introduces possible key elements/dimensions to be considered.

3. Effectiveness in the context of CFS 19. The concept of “effectiveness” is defined in the M&E literature by different actors including academia, international organizations, multilateral and bilateral cooperation agencies, NGOs, etc. Most of those actors consider the “effectiveness” as a key evaluation criterion focused on the review of the achievement of stated expected results in the context of development projects, programmes, strategies or policies8. 20. With reference to these internationally accepted definitions, a proposed working definition of effectiveness in the context of CFS is the extent to which the CFS Overall Objective and Outcomes are achieved, or are expected to be achieved. 21. The proposed definition of “CFS effectiveness” puts the emphasis on the realization of the CFS Overall Objective (Contribute to reducing hunger and malnutrition and enhancing food security and nutrition for all human beings), towards which the Committee contributes by promoting the achievement of its three Expected Outcomes. 22. The issues addressed by the three CFS Expected Outcomes are very complex given the multi-dimensionality of food security and nutrition. A comprehensive assessment of CFS effectiveness in contributing to the improvement of world FSN is extremely challenging and would require long-term and in depth analysis. It is also important to emphasize that CFS itself is multi-faceted and the contribution of the three CFS Outcomes towards the improvement of world FSN would have to

8 The OECD “Glossary of Key Terms in Evaluation and Results Based Management” defines effectiveness as the

extent to which the development interventions’ objectives were achieved, or are expected to be achieved, taking into account their relative importance. The OECD considers the “effectiveness” as one of the key evaluation criteria of development aid. Typically, the effectiveness of development aid is assessed in combination with the following complementary key evaluation criteria: Relevance; Efficiency; Sustainability; Impact. A significant number of institutions involved in international cooperation for development, including the UN, the EU and bilateral cooperation agencies, refer to the above OECD evaluation criteria to define their own approaches to monitoring and evaluation. http://www.oecd.org/development/peer-reviews/2754804.pdf

10

consider complementary and coherent actions promoted by all actors, such as national governments, international development agencies, financial institutions, CSOs, private sector, etc. 23. For these reasons, the implementation of the recommended “periodic assessments of CFS effectiveness” should be seen as complementary to other types of M&E activities. For the specific purpose of designing a baseline survey, examples of possible assessment dimensions are proposed in Table 1.

Table 1: Examples of possible dimensions to consider for CFS baseline survey

Possible Dimensions (provisional)

Definition of the Dimensions (what they measure)

1 Relevance of CFS Extent to which CFS addresses FSN priorities at global, regional and national levels

2 Inclusiveness Extent to which CFS includes all relevant stakeholders

3 Participation Extent to which CFS has active participation from relevant stakeholders

4 Convergence Extent to which CFS promotes convergence among CFS members and stakeholders (both horizontal and vertical integration and coherence)

5 Collaboration Extent to which CFS is engaged with other relevant fora and initiatives

6 Communication

Extent to which CFS work and results are adequately communicated

4. Assessment methodology to consider for a CFS baseline survey 24. Section 3 indicates possible dimensions to consider for a baseline survey, which can be assessed by adopting different approaches and methods. The selection of the appropriate assessment approach depends, to a large extent, on: a) the nature and “evaluability” of the elements to be assessed; b) the depth of the analysis to be conducted; c) the financial and human resources available. 25. Qualitative assessment methods generally involve gathering information about values, opinions and behaviours, as expressed by groups or individuals from their own perspectives. Qualitative assessments are usually carried out through a combination of observations, individual and group discussions and/or interviews. Different techniques can be used to collect data in a qualitative assessment. The dimensions proposed in Table 1 are particularly suitable for being assessed through a qualitative method, such as an expert opinion survey, in a useful and cost-effective manner. 26. As indicated in Section 2, the results from the implementation of the “CFS effectiveness baseline survey” are expected to provide a reference base for appraising progress and improvements over time. Accordingly, the implementation of the survey should be periodically

11

replicated (for example every 4 to 5 years) using the same methodology and focusing on the baseline assessment exercise. That aspect is critical and should be taken into account during the design phase. 27. In order to ensure that future surveys are consistent and implementable at an affordable cost, the assessment dimensions and key indicators on which to focus must be strategically selected. They should take into account the modalities of implementation and the challenge of identifying an adequate sample of responders to survey. As mentioned, the adoption of the “key Informant Interview” technique seems to be a suitable option for the purposes of the CFS effectiveness survey. Accordingly, the instrument to conduct the survey has to be identified. Some of the possible options are the following ones:

1. Structured interviews; 2. Semi structured interviews; 3. Focus group discussions.

28. In all cases, the proposed dimensions in Table 1 of Section 3 need to be transformed into “questions” (to be included in a structured or semi-structured questionnaire) or discussion guides. They should be developed in such a way that the answers or opinions expressed can be applied to the respective indicators for each dimension. A method to aggregate and report the results needs to be identified. 29. Once the method and technique for conducting the survey have been selected, the survey sample should be clarified. The identification of the survey sample might require revisiting the steps above. Some of the key questions to be answered for identifying the survey sample are:

What type of informants should be surveyed? Which selection criteria should be applied? Who should be interviewed? How to ensure a balanced representation of different types of informants, representing the

different CFS constituencies? Should the sample already know about CFS and its products? Should CFS Members and other stakeholders be considered differently?

Annex 1. Evolution of thinking on CFS M&E and summary of work carried out

1. An Open-Ended Working Group on Monitoring (OEWG-Monitoring) was established by the Bureau in 2012. It met twice during the year and reported to CFS 39 in October 2012. The Committee asked the OEWG-Monitoring to continue its work, aiming to: “review existing global, regional and country initiatives in monitoring food security and nutrition; identify innovative approaches, gaps and possible collaboration amongst various actors and approaches. In 2013, a Technical Support Team (TST) was set up to advise the OEWG-Monitoring. Two OEWG-Monitoring meetings and a workshop were held in 2013. The combined outcome of those events informed reporting to CFS 40 in October 2013. 2. The work carried out by the OEWG-Monitoring initially focused on exploring various themes or areas for monitoring, grouped as:

a) The situation and trends in food insecurity in the world. b) Actions and initiatives addressing FSN. c) The implementation of CFS decisions and recommendations d) The value added of CFS itself; for instance in the effectiveness of its reform, its inclusiveness,

participation and satisfaction of participants in its institutional arrangements and processes.

3. It has been recognized how items (a) are already monitored by development and research agencies, notably FAO in its annual State of Food Insecurity in the World (SOFI) reports. There are also initiatives to monitor item (b). It was agreed that the CFS Monitoring exercise should focus mainly on items (c) and (d). Group (c) are items for which CFS Members, having agreed to specific courses of action, are accountable in their own right, while (d) may be where CFS - as a body - has accountability9. 4. The OEWG-Monitoring recognized that monitoring in the context of CFS is firstly about determining how well the Committee is meeting its overall objective of contributing to the improvement of food security and nutrition at various levels. Secondly, it is about recommending approaches to Member Countries, sub regional and global bodies in order to ensure that M&E mechanisms lead to enhanced accountability and improvement in policy and programme delivery10. In particular, it recommended that CFS monitoring should focus on the following two streams of work:

Stream 1 - monitoring of CFS decisions and recommendations. Stream 2 - facilitating monitoring of outcomes related to food security and nutrition at

country and regional level.

5. During 2013, OEWG-Monitoring discussions focused on: linkages between monitoring and the communication strategy; the process of preparations for monitoring the implementation of the Voluntary Guidelines on the Responsible Governance of Tenure of Land, Fisheries and Forests in the Context of National Food Security (VG-GT) at CFS 40; the categorization of CFS decisions and recommendations; the need to set criteria for the preparation of CFS policy recommendations; the review of the initial mapping of existing initiatives. The OEWG-Monitoring workshop of July 2013 helped to consolidate thinking on monitoring, drawing on the lessons learned in past efforts, reviewing examples of existing M&E mechanisms and helping to identify key elements of innovative

9 Adapted from the Chair’s summary of the 1st OEWG meeting on 22 March 2012

10 Ref. paragraph 6 of The CFS Reform Document –CFS 2009/2 Rev 2:

http://www.fao.org/fileadmin/templates/cfs/Docs0910/ReformDoc/CFS_2009_2_Rev_2_E_K7197.pdf

13

approaches, gaps/constraints and the possible collaboration between various actors and approaches. Based on the outcomes of its discussions, the OEWG developed a set of recommendations11 for consideration by the Committee at CFS 40. Key points were:

a) The importance of a good communication strategy; before monitoring outcome and impact on the ground, interested stakeholders should be made aware of CFS products and outputs. A plan for the dissemination of CFS decisions is needed and should draw upon the networks of all CFS stakeholders.

b) The wide range of CFS decisions and recommendations imply different kinds of actions by various stakeholders as well as different forms/levels of monitoring their application/implementation. The following categorization was proposed to help rationalize the types of CFS decisions and recommendations:

I. CFS Products : This group includes the final results of CFS multi-stakeholder processes;

II. CFS Policy Recommendations: This group includes primarily the results of the Policy Convergence Round Tables sessions held during Plenary;

III. Process-related recommendations: These are more specific recommendations addressed to CFS Secretariat, Bureau, Advisory Group, and/or the HLPE. They include specific tasks or reporting requests.

c) Of the above categorization, group (ii) are generally of a broad nature and may not be

articulated in a way to facilitate implementation. Group (iii) are tasks normally reported on by the Secretariat or specific CFS stakeholders12. CFS should not attempt to monitor all decisions and recommendations but rather focus on category (i) above; the major, strategic and catalytic products, such as the VG-GT, the Global Strategic Framework for Food Security and Nutrition (GSF), the forthcoming Principles for Responsible Agricultural Investments (CFS-RAI) and the Agenda for Action for Addressing Food Insecurity in Protracted Crisis (CFS-A4A).

d) Monitoring mechanisms should build on existing mechanisms at global, regional and national level. Key desired characteristics include: local ownership, rights-based, inclusiveness and multi-stakeholder participation, grounded in multi-sectoral policy frameworks, ensure adequate country capacities and resources and include both qualitative and quantitative aspects. They should be in line with the five principles set out in the GSF13.

e) Based on lessons learned from previous monitoring efforts14 and advice from the Rome-based agency monitoring and evaluation experts, a rigid framework for monitoring CFS decisions and recommendations should be avoided. The development of a light survey instrument followed by periodic assessments should be considered.

11

A framework for monitoring CFS decisions and recommendations” CFS 2013/40/8 http://www.fao.org/docrep/meeting/029/mi320e.pdf 12

A document is usually prepared for the annual CFS session that gives a summary of the implementation status of most process-related recommendations. See for instance “Following progress on decisions and recommendations of the Committee on World Food Security” CFS 2013/40/Inf.13 www.fao.org/fileadmin/user_upload/bodies/Conference_2011/IMG/pdf.gif 13

Paragraph 93 of CFS 2012/39/5 Add.1 - Global Strategic Framework for Food Security and Nutrition www.fao.org/cfs/cfs-home/global-strategic-framework/en/ 14

CFS 2008/3: Follow-up to the World Food Summit: Report on Progress in the Implementation of the Plan of Action ftp://ftp.fao.org/docrep/fao/meeting/014/K3058E1.pdf

14

They should be flexible and adaptable to specific circumstance, not an attempt to create a “one-size-fits-all” system.

6. CFS at its 40th session in October 2013 endorsed the recommendations15 highlighting the importance of CFS monitoring for improving CFS effectiveness, the need to focus CFS monitoring on the Committee’s major strategic and catalytic products and “… endorsed the conduct of periodic assessments of CFS effectiveness in improving policy frameworks especially at country level and in promoting participation of and coherence among stakeholders on food security and nutrition. Specifically, it recommended carrying out a baseline survey to assess the current situation as the base of assessing progress”.

15

Paragraph 66 of “CFS40 Final Report” www.fao.org/docrep/meeting/029/mi744e.pdf

15

Annex 2. Note on M&E terminology

In order to ensure that the M&E function will be effectively incorporated in CFS, it is important that CFS stakeholders have a common understanding of key M&E terms and concepts. This note provides definitions and clarifications on Monitoring and Evaluation and related terms, aiming to promote their appropriate use in CFS documents and discussions.

Monitoring and Evaluation

Both Monitoring and Evaluation can apply to many things, including an activity, project, programme, strategy, policy, topic, theme, sector or organization. Usually, the terms “monitoring” and “evaluation” are used in combination to generically refer to assessments, reviews or performance measurements of development projects, programmes, strategies or policies. In the technical jargon, the “M&E” acronym is typically associated to plans, systems, arrangements, reporting mechanisms, methodologies etc. Nevertheless, it is important to point out how the terms “monitoring” and “evaluation” refer to different processes, although they are very much related.

Monitoring can be defined as the on-going process by which stakeholders (of development projects, programmes, strategies or policies) obtain regular feedback on the progress being made towards achieving the expected goals and objectives.

The above definition of monitoring puts emphasis on the on-going process of collecting, analysing and disseminating information on progress against achieving results. Accordingly, the mere review of progress made in implementing planned actions or activities should be associated to the analysis of the extent to which the targeted development results are being achieved. In other words, monitoring should be concerned not only with asking “Are we taking the actions we said we would take?” but also “Are we making progress on achieving the results that we said we wanted to achieve”? A critical aspect is that monitoring has to be considered as an internal management function. That means that, by definition, monitoring is carried out on a continuous basis under the responsibility of managers of projects, programmes, strategies or policies. This aspect represents a main difference with the evaluation process, which is defined as it follows.

Evaluation can be defined as a rigorous and independent assessment of either completed or ongoing development interventions to determine the extent to which they are achieving stated objectives and contributing to decision making.

Typically, the aim of evaluations is to determine the relevance and fulfilment of objectives, development efficiency, effectiveness, impact and sustainability. An evaluation should provide information that is credible and useful, enabling the incorporation of lessons learned into the decision–making process. A key word in the above definition is “independent”. Actually, the key distinction between the two monitoring and evaluation processes is that evaluations are carried out independently, to provide an objective assessment to managers and stakeholders of whether projects, programmes, strategies or policies they manage are on track or not. In practice, that means that an assessment can be

16

considered as an evaluation only when it is performed by experts that are not or have not been involved, directly or indirectly, in the design, financing and implementation of projects, programmes, strategies or policies. Evaluations are also more rigorous than monitoring in their procedures, design and methodology, and generally involve more extensive analysis.

Although the perspectives of observation are different (monitoring is an internal function while evaluations are independent), the aims of both monitoring and evaluation are very similar: to provide information and generate learning that can help inform decisions, improve performance and progressing effectively towards the achievement of planned results.

Monitoring and evaluation are complementary processes that serve several purposes. In the absence of effective monitoring and evaluation, it would be difficult to know whether the intended results are being achieved as planned, what corrective action may be needed to ensure delivery of the intended results, and whether initiatives are making positive contributions towards development impacts. Normally, monitoring and evaluation relate to pre-identified results. They are driven by the need to account for the achievement of intended results and provide a fact base to inform corrective decision making. They are essential management tools to support the accountability for results, resources entrusted to it, and learning.

Monitoring, as well as evaluation, provides opportunities at regular predetermined points to validate the logic of projects, programmes, strategies and policies, as well as to review the implementation of planned activities and to make adjustments as needed. Good planning and designs alone do not ensure results. Progress towards achieving results needs to be monitored. Information from monitoring needs to be used to encourage improvements or reinforce plans. Information from systematic monitoring also provides critical input to evaluation.

Evaluation complements monitoring by providing an independent and in-depth assessment of what worked and what did not work, and why this was the case. After implementing and monitoring an initiative for some time, it is an important management discipline to take stock of the situation through an external evaluation. The benefits of using evaluations are multiple. A quality evaluation provides feedback that can be used to improve programming, policy and strategy. Evaluation also identifies unintended results and consequences of development initiatives, which may not be obvious in regular monitoring.

Results Based Management

The United Nations Evaluation Group (UNEG) considers Planning, Monitoring and Evaluation as the three interconnected processes of the Results Based Management (RBM) approach.

RBM is defined as a broad management strategy aimed at achieving improved performance and demonstrable results16

16

UNEG: www.uneval.org

17

Planning can be defined as the process through which the stakeholders of development projects, programmes, strategies or policies identify a shared vision of the future, goals and objectives to be achieved.

In RBM, planning is the critical process through which policies, strategies, programmes and projects are designed, identifying the results that need to be pursued. Monitoring and evaluation support learning from successes and challenges and inform decision making, so that current and future development interventions are constantly improved. Linking Planning, Monitoring and Evaluation generates a continuous process of doing, learning and improving, which is commonly referred to as the RBM life cycle.

The diagram to the right shows the RBM life cycle: through the Planning process, key stakeholders identify a shared vision of the future, the goals and objectives to be achieved, the actions that need to be implemented and the required inputs; through Monitoring, stakeholders obtain regular feedback on the progress being made in delivering the planned outputs contributing to the set goals and objectives; the Evaluation process allows a rigorous and independent assessment of the results achieved, helping to determine the relevance and the extent to which development initiatives achieve the stated objectives. Findings and learning from Monitoring and Evaluation processes feed into new planning, aiming at enhancing current and future interventions and initiatives.

M&E Framework (or M&E Plan)

Effective and timely decision-making requires information from regular and planned monitoring and evaluation activities. While monitoring provides real-time information on the implementation of policies, strategies, programmes or projects, evaluation provides more in-depth assessments. The monitoring process can generate questions to be answered by evaluation. Also, evaluation draws heavily on data generated through monitoring, including baseline data, information on the implementation process, and measurements of progress towards the planned results through indicators. Monitoring and Evaluation actions need to be planned. The M&E Framework serves that purpose.

18

M&E Framework (or M&E Plan): is the instrument to plan for monitoring and evaluation. Typically, the M&E Framework includes a narrative and a tabular format matrix describing how monitoring and evaluation will be undertaken and the accountabilities assigned to different individuals, agencies and institutions.

A clear M&E Framework, agreed among the key stakeholders, is essential in order to carry out monitoring and evaluation systematically. The M&E framework should clarify:

What is to be monitored and evaluated The activities needed to monitor and evaluate Who is responsible for monitoring and evaluation activities When monitoring and evaluation activities are planned (timing) How monitoring and evaluation are carried out (methods) What resources are required and where they are committed

19

Annex 2: List of participants

Name Nationality Organization

1. Biraj Patnaik India Indian Right to Food Campaign

2. David Sarfo Ameyaw USA Alliance for a Green Revolution in Africa (AGRA)

3. Esther Wiegers Netherlands International Consultant

4. Jan Arend Germany German Institute for Human Rights

5. Katharine Downie Canada International Livestock Research Institute (ILRI)

6. Malohat Shabanova Tajikistan World Food Programme (WFP)

7. Mariano Poledo Argentina Ministry of Agriculture - Unit for Rural Change (UCAR)

8. Mona Younis USA International Consultant

9. Rehema Bavuma Uganda World Forum of Fish Harvesters & Fish Workers (WFF)

10. Rene Verduijn Netherlands International Consultant

11. Richard Caldwell USA Bill & Melinda Gates Foundation (BMGF)

12. Romeo S. Recide Philippines Association of Southeast Asian Nations (ASEAN)

13. Simon Kisira South Africa The New Partnership for Africa's Development (NEPAD)

14. Valeria Burity Brazil Conselho Nacional de Segurança Alimentar e Nutricional (CONSEA)

15. Wellington Mulinge Kenya Kenya Agricultural Research Institute (KARI)

16. Thomas Spangler (ESA) FAO (TST Member)

17. Jose VallsBedeau (ESS) FAO (TST Member)

18. Carlo Cafiero (ESS) FAO (TST Member)

19. Ahmed, Shukri (EST) FAO (TST Member)

20. David Palmer (NRC) FAO (TST Member)

21. Cristian Morales (ESA) FAO (TST Member)

22. Mark Smulders (ESA) FAO (TST Member)

23. Jim Lattimer WFP (TST Member)

24. Marzia Perilli IFAD (TST Member)

25. Molly Anderson CSM (TST Member)

26. Alberta Guerra CSM (TST Member)

27. Martin Wolpold-Bosien CSM (TST Member)

28. Alessandro Patriarchi CFS Secretariat

29. Alberto Suppa CFS Secretariat

30. Amira Muammar CFS Secretariat

31. Mark McGuire CFS Secretariat

32. Francoise Trine CFS Secretariat

33. Robert Sabiiti OEWG Chair

20

Annex 3: Agenda of the workshop

DAY 1 – Monday 14 April Location 8:30 – 8:45 Registration and collection of FAO building passes (FAO Pavilion – main

entrance)

SESSION 1 – INTRODUCTION 9:00 – 9:20 Welcome remarks by the Chair of the CFS OEWG-Monitoring, Robert Sabiiti, and

CFS Secretary, Kostas Stamoulis Philippines room (C277)

9:20 – 9:45 Introduction of participants

9:45 – 10:15 Presentation of the objectives and expected results of the workshop – Mark McGuire Q&A

10:15 – 10:30 Presentation of the agenda – Francoise Trine (Facilitator)

10:30 – 11:00 Coffee break

SESSION 2: KEY ELEMENTS TOWARDS ASSESSING CFS EFFECTIVENESS 11:00 – 11:30 Scope of the CFS baseline survey – Alberto Suppa

Q&A Philippines room (C277)

11:30 – 12:30 Group work – Key elements to consider for the baseline survey to be announced

12:30 – 14:00 Lunch break

14:00 – 15:30 Group work continues to be announced

15:30 – 15:45 Coffee break Philippines room (C277)

15:45 – 17:30 Reporting to plenary and discussion

18:00 – 19:00 Reception to be confirmed

DAY 2 – Tuesday 15 April

9:00 – 9:10 Recap and agenda or the day Philippines room (C277)

SESSION 3: MODALITIES OF THE BASELINE SURVEY 9:10 – 10:30 Group work – Defining the modalities of the baseline survey to be announced

10:30 – 10.45 Coffee break Philippines room (C277)

10:45 – 12:30 Group work continues to be announced

12:30 – 14:00 Lunch break

14:00 – 15:30 Reporting to plenary and discussion Philippines room (C277)

SESSION 4: WRAP-UP SESSION 15:30 – 15:45 Coffee break Philippines room (C277)

15:45 – 17:30 Wrap up Philippines room (C277)

21

Annex 4: Instructions for group work 1 and 2

Instructions for Group Work 1 – 14 April 2014

(2 hours ½)

1. Objectives: Identify: (i) key criteria for assessing effectiveness; (ii) high-level survey

questions17

related to each criterion; (iii) possible indicators related to each criterion; (iv)

information needs to inform each indicator; and (v) sources of information.

2. Organization: - The group will be moderated by a member of the CFS Secretariat - The nominated rapporteur will prepare and deliver the presentation of the results of the

group’s discussions in plenary - The results of the group’s discussions will be entered into an electronic form which will

be used for reporting in plenary. The form is available in the computer provided to each group

- The presentation in plenary should take no longer than 10 minutes

3. Activities

The group will: - Designate a rapporteur - Review the proposed criteria for assessing CFS effectiveness - Confirm, modify or consolidate the proposed criteria and identify new criteria if needed - Identify high-level survey questions for each criterion - Identify possible indicators for each criterion - Identify the information required to inform each indicator and the sources of information -

The rapporteur will: - Enter the results of the discussion in electronic format in the form - Present the results of the group in plenary (max 10 minutes)

4. Time allocation - Maximum one hour will be allocated to Objective (i), corresponding to the 11.30 to 12.30

time slot - The remaining 1 hour ½ in the afternoon will be allocated to the other objectives

5. Results - The electronic form filled with the results achieved by the group by 15.45, ready to be

presented in plenary

17

High-level survey questions are the questions that are expected to be answered by the survey, not the questions

to be asked to the respondents (e.g. through a questionnaire).

22

Instructions for Group Work 2 – 15 April 2014

(3 hours)

1. Objectives: Identify: (i) options for data collection methods18

, considering

implications; and (ii) survey sample

2. Organization: - The groups remain the same than in Group Work 1 - The nominated rapporteur will prepare and deliver the presentation of the results of the

group’s discussions in plenary - The presentation in plenary should take no longer than 10 minutes

3. Activities

The group will:

- Identify different options for data collection methods, considering the implications in terms of costs, accuracy/reliability, requested flexibility for analyzing the situation, readiness of the respondents to participate, potential and limitations of the method, etc.

- Identify the sample (respondents’ groups and sample size, considering national, regional and global levels)

The rapporteur will: - Enter the results of the discussion in electronic format in the form (two columns will be

added to the existing matrix: (Data collection methods and Survey sample) - Present the results of the group in plenary (max 10 minutes)

5. Results - The results achieved by the group by 15.45, ready to be presented in plenary.

18

Data collection methods: individual or group interview (structured or semi-structured), focus group discussion,

key informant interviews, expert panels, on-site observation, document review, internet search, etc.. Specific

tools to be used within each selected method could be discussed as well (especially in case of group interviews).

23

Annex 5: Results of the evaluation

Design Matrix of the CFS Effectiveness Baseline Survey – Outcome A(working group 1)

Proposed criteria for assessing effectiveness

Examples of High-level survey questions

(assessment questions)

Examples of indicators Information needs / tools

Information / data Sources

Responsible entity Survey Sample

1

Relevance of CFS (Extent to which CFS addresses FSN priorities at global, regional and national levels)

1. Does the CFS have the right agenda? 2. Are the decisions/recommendations relevant to the needs and contexts of the member countries/stakeholders? 3. Are the strategic and other decisions of the CFS taken up at the country level?

Extent to which the agenda accommodates the priorities of CFS members and participants? Evidence based papers/ reports considered by CFS while drawing up the agenda?

Opinions and perceptions by CFS Stakeholders Opinions and perceptions of multi-stakeholders at regional and national levels Problems at the national, regional and global level. Assessment

CFS Key informants CFS multi-stakeholders at the regional and national levels Assessments from secondary data including HLPE reports, country reports, independent assessments, FAO indices etc. HLPE/ CFS

CFS Secretariat/ bureau and advisory group (Budget for this exercise should be part of the M&E plan of the CFS and necessary resources should be made available to the appropriate entity, including the Secretariat and the advisory group, for doing this). All evidence based papers and reports are the primary responsibility of the HLPE Countries may also look at national level reports or other reports which reflect their priorities better.

Pilot countries (Champions approach/ Volunteer approach but with geographical balance and meaningful, representative sampling?) Participants (FAO, WFP, IFAD) or INGOS could take on the responsibility of the survey in some countries. Secondary data for all available countries on accepted parameters.

2

Inclusiveness (Extent to which CFS includes all relevant stakeholders)

1. Is CFS open to all the relevant stakeholders that can make contributions to its work 2. Is CFS open to all stakeholders who wish to participate?

Ratio/ Proportion of those who are engaged to those who should be engaged at the global, regional and national level. Degree of inclusiveness of CFS members, Participants and observers Degree of inclusion of constituencies most affected by food insecurity and malnutrition

Participants survey Number of members present in different sessions / meetings Number of meetings attended by the members in different committees Degree of representation of constituencies most

Documentation of the “work streams” of the CFS (members/ participants) Minutes of the meeting

CFS Secretariat/ bureau and advisory group (Budget for this exercise should be part of the M&E plan of the CFS and necessary resources should be made available to the appropriate entity, including the Secretariat and the advisory group, for doing this).

24

Composition of the participants Relationship with national multi-sectoral stakeholders platforms Growth in number of the constituencies over time Growth in the number of constituencies most affected by hunger and malnutrition Regional/ gender / stakeholder balance in all forums and meetings of the CFS. Different linguistic groups represented

affected by food insecurity and malnutrition present in different sessions/ meetings Opinions and perceptions of key informants Degree of interaction between the CFS and the national level multi-stakeholder platforms

Minutes of the meeting

3

Participation (Extent to which CFS has active participation from relevant stakeholders)

Do the key stakeholder (members, governments, UN, international regional bodies, IFIs, CSOs, private sector and other relevant actors), actively participate and contribute to the process in the CFS

Degree of existence of mechanisms that allow social participation Degree of participation of governments, UN, international and regional organisations, CSOs, private sector, IFIs and other relevant stakeholders at National Level, Regional level and Global level Respective roles of

Expert opinion survey Number of participants of constituencies most affected by food insecurity and malnutrition participating in different sessions/ meetings

Participants survey Interviews,

25

Extent to which the demands of all constituencies are met by the documentation done by the CFS

governments UN, international and regional organisations, CSOs, private sector, IFIs and other relevant stakeholders Degree of participation inclusion of communities most affected by food insecurity and malnutrition Respective roles of inclusion of communities most affected by food insecurity and malnutrition Availability of resources to ensure effective participation of those who need to participate the most Ability to mobilise or identify resources to ensure full participation of governments UN, international and regional organisations, CSOs, private sector, IFIs and other relevant stakeholders

Quantitative approach: Number of interventions Qualitative approach: useful interventions (if people believe that their participation is useful)

questionnaires. List of attendees (categories)

Sample of representative of different constituencies

4

Convergence (Extent to which CFS promotes convergence among CFS members and stakeholders (both horizontal and vertical

Do the CFS decisions reflect convergence amongst the stakeholders? Does the CFS actively promote convergence between different kinds of stakeholders (private sector / CSOs/ governments/ IFIs/ regional international bodies)?

Extent to which CFS agendas and issues have been adopted by members, governments, (private sector / CSOs / governments /IFIs/ regional international bodies

Opinions and perception of participants on adoption of CFS agenda and issues.

Independent evaluations Participants opinion survey

CFS Secretariat/ bureau and advisory group

Pilot countries (Champions approach/ Volunteer approach but with geographical balance and meaningful, representative sampling?) Participants (FAO, WFP, IFAD) or INGOS could take on the responsibility of the survey in some countries.

26

integration and coherence)

8

Results orientation (Extent to which the enhanced co-ordination of food security and nutrition at global, regional and national level by the CFS has produced results)

To what extent has the enhanced co-ordination led to significant results? To what extent do CFS decisions result in adoption of policies at the global, regional and national levels?

Extent to which policy documents at the global, national and regional level refer to the CFS decisions as inspiration? (consider CFS decisions as reference) Extent to which members and participants of CFS incorporate the CFS decisions in their own mandates and programs Number of countries and participants that have decided to include relevant parts of the CFS strategic and catalytic outputs into their plans and policies.

Citations and references Citations in annual reports and strategic documents

Policies and policy documents Regional and intergovernmental bodies documents (laws, policies, plans and programmes) Annual reports and strategic documents of CFS Members and participants

CFS Secretariat/ bureau and advisory group

Pilot countries (Champions approach/ Volunteer approach but with geographical balance and meaningful, representative sampling?) Participants (FAO, WFP, IFAD) or INGOS could take on the responsibility of the survey in some countries.

9

Impact (Extent to which CFS has contributed to the overall objective) (stand-alone not part of the outcome criteria)

To what extent has the CFS contributed to the reduction of hunger and malnutrition and the progressive realisation of the right to adequate food

Percentage reduction of malnutrition and hunger Number of countries that have moved towards the progressive realisation of the right to adequate food Countries that have adopted policies and legal frameworks that are compliant to right to food

Countries that are reducing food insecurity Policies that are based on the right to food

Food security status reports Policy documents

CFS Secretariat/ bureau and advisory group

10

Sustainability (Extent to which the CFS contribute to sustainable food security and nutrition)

To what extent the CFS agenda is addressing sustainability

27

???? (stand-alone not part of the outcome criteria)

11 Coherence

Degree of horizontal integration and coherence (among countries, organizations, stakeholders, etc.) Degree of vertical integration and coherence (from local to global levels and vice versa)

5

Collaboration (Extent to which CFS is engaged with other relevant fora and initiatives )

Extent to which platforms for sharing, learning and knowledge management have been facilitated by the CFS

7

Evidence-based decision-making (Extent to which CFS decisions are based on evidence)

Are CFS decisions and recommendations based on evidence? What evidence is available for the CFS and what mechanisms in place to use it?

Degree to which discussions and decisions taken are based on evidence

6

Communication ???? (Extent to which CFS work and results are adequately communicated)

Is CFS communicating adequately? Is CFS known at country level? Are its messages reaching the relevant decision makers/stakeholders?

Awareness at global level Awareness at regional level Awareness at country level

28

Design Matrix of the CFS Effectiveness Baseline Survey - OUTCOME B: Policy Convergence (Working Group 2)

Proposed criteria for assessing

effectiveness

Examples of High-level survey questions

Examples of indicators

Information needs

Information Sources

Data Collection method

Survey Sample

Capacity to produce agreed upon comprehensive frameworks, principles and decisions aligned with global, regional and national priorities

1 Is the CFS policy agenda aligned with global, regional, and national FSN priorities? 2 Are people who are most affected by food insecurity & malnutrition represented in CFS? 3 Is the CFS forward-looking? Do CFS decisions, recommendations & policies reflect emerging issues? 4 Does the CFS promote policy coherence across sectors? 5 Does the CFS successfully bring together actors to agree on policies? Are dissenting voices heard and captured in the final products? 6 Is CFS engaging strategically and constructively in regional and global fora to ensure policy coherence of relevance to FSN? (e.g. Post-2015, SCN, ICN2, G8/G20) 7 Do CFS policies and products incorporate

1. Extent to which the CFS agenda is aligned with global, regional and national FSN priorities 2. Extent to which concerns of marginalized and malnourished people are fully incorporated in the CFS agenda 3. Extent to which the CFS agenda includes emerging issues at all levels 4. Extent to which policy convergence exists across sectors 5. Inclusiveness of participation, extent of consensus, whether dissent is documented and “held” rather than ignored; percentage of topics raised that get to the floor of CFS, whether choice of issues to prioritize is evidence-based 6. How often CFS decisions influence other forums related to FSN, respect for CFS by Boards and participants in other forums that deal with FSN, how often the CFS structure is held up as a model, whether decisions and

1. What are national, regional and global FSN priorities? What is the degree of alignment between priorities and CFS agenda? 2. concerns of marginalized people; how concerns in national planning documents are identified 3. What are the emerging needs and issues related to FSN? 4. Which sectors are affected by each policy area? Are all included? 5. Total topics that are raised by CFS stakeholders vs. total topics that get to the floor 6. Other forums that are dealing with FSN and how CFS is linked

1a. National/regional policy documents (e.g., MAFAP); 1b. UN and donor agencies, members, participants 1c. CSOs 2. CSOs 3a. Global debates and scientific literature, conferences and drivers of food insecurity. 3b.Local trends and hot-spots related to FSN. 3c. Conflict & early-warning literature and debates. 3d. FAO’s SO1 assessment List of all “essential” stakeholders 1. List of those voices not being heard

1 Document review 2 Expert opinion survey

1a. Policy documents of selected countries in each region 1b. In-country CSOs, official gov’t documents 6 Key representatives and Board members of similar global forums (e.g., HLTF, Global Donor Platform, ECOSOC, SCN, OEWG on SDGs)

29

the principles of rights-based approaches? 8 Are CFS policy decisions and recommendations based on evidence supporting analysis and decisions at all levels? 9 What evidence is available for the CFS, is it transparent, and what mechanisms in place to use it?

issues are being referred to the CFS 7. Extent to countries’ policy and legal frameworks reflect CFS’s guiding principles 8. Degree of incorporation of RtF principles in CFS decisions, products and recommendations 9. Extent to which people’s movements and those directly representing interests of the marginalised (food insecure/-malnourished) are engaged, heard and represented? 10. Ratio of those who are engaged in national and regional forums to those who should be engaged 11. Degree to which source of evidence reflect unbiased diversity 12. Degree to which linkages between research and devt institutions (at all levels) are effective at supporting rational decision-making processes

Capacity to communicate awareness of frameworks, principles and decisions (is CFS

1. Are CFS decisions and products reaching appropriate stakeholders? 2. Have regional meetings been convened to raised awareness of CFS

30

communicating effectively to regional and national gov’t and NGO stakeholders)

products and policies? 3. Do appropriate people attend these meetings? 4. Is feedback from people attending regional forums being captured? 5. Is CFS known at country level? Are its messages sensitizing and reaching the relevant decision makers/-stakeholders? Are messages taken up? 6. Are issues and recommendations from the grass roots level taken on board through natl & regl fora to the CFS? 7. Is CFS adequately communicating the centrality of a human rights-based approach?

Capacity to take up frameworks, principles and decisions at the regional and national levels

1. Are institutional/ functional capacities and/or policy/legal/-regulatory frameworks in place at regional and national levels to facilitate ‘vertical policy coherence’ 2. Are CFS principles, decisions and policies reflected in national and regional institutions (law, policies, etc.)? 3. (Are countries adopting policy guidance provided by CFS (global forum)? (e.g. GFS, VGGT))

1. Degree to which the CFS recommendations and decisions influence institutional and legal reforms & policy implementation at all levels (FSN) 2. Extent of horizontal policy convergence (gender, RtF, etc.)

1. List of CFS decisions and key products dating back to 2010 2. Policy review (independent analysis of natl, regl global policies, strategies, programmes, etc) 3. .Opinions and perceptions by CFS members and stakeholders

1. CFS decision boxes 2. natl, regl global policies, strategies, programmes, etc). 3. CFS key informants at ntl, regl global levels

31

Design Matrix of the CFS Effectiveness Baseline Survey (Working Group 3) CFS roles in strengthening national and regional FSN actions / outcome C

Proposed criteria for assessing

effectiveness

Examples of High-level survey questions

(assessment questions)

Examples of indicators

Information needs

Information Sources

Data Collecti

on Method

s

Survey Sample

1

Relevance of CFS A

(Extent to which CFS support/advice/ /products address FSN priorities at global, regional and national levels)

Does the CFS have the right

agenda?

Are the decisions/

recommendations relevant

to the needs and contexts of

the member

countries/stakeholders?

Is the support-advice of the

cfs responding to countries

needs and capacity?

Opinions and

perceptions by CFS

Stakeholders

Put the questions

in a way that

enable

stakeholders,

participants and

countires, to tell

their story

CFS Key

informants

Not just official

but information

coming from

other

stakeholders

Human rights-based

approach A B

Is the support advice

provided by the CFS

reflecting the PANTHER

principles?

At what extent the right to

food is incorporated

32

2

Inclusiveness A B

(Extent to which CFS support/advice/products include all relevant stakeholders)

Is CFS open to all the

relevant stakeholders?

Is CFS open to all

stakeholders who wish to

participate?

Are the most affected by

food insecurity and

malnutrition included and

supported in their active

participation?

Degree of

inclusiveness of CFS

Participants

Breadth of

inclusiveness (i.e.

How wide is

inclusiveness)

Who is included?

Who is missing?

Interviews with

key informants,

experts,

networks, etc.

3

Participation A B

(Extent to which CFS has active participation from relevant stakeholders)

Degree of

participation by

CFS Members and

Participants

Contribution

provided CFS

4

1. Degree of

horizontal

integration and

coherence (among

countries,

organizations,

stakeholders, etc.)

2. Degree of vertical

integration and

coherence (from

local to global

levels and vice

versa)

5

Collaboration A

(Extent to which CFS is engaged with other relevant fora and initiatives )

3. To what extent does CFS

attract and work with other

fora and initiatives?

4. How does CFS work with

other fora and initiatives?

5. With which fora or initiatives

does CFS work?

6. Are there joint statements,

proclamations, authorships,

8. Fora and initiatives

with which CFS

works

9. Forms and

objectives of

collaboration

10. Joints

statements,

proclamations,

12. CFS records 13. CFS tracking 14. Complete

enumeration

33

etc.?

7. How many sources of

funding?

authorships, etc.?

11. Number and

types of funding

sources?

6

Communication A B

(Extent to which CFS work and results are adequately communicated and shared at all levels)

15. Is CFS communicating

adequately?

16. Are CFS and its

products known at country

level?

17. Are its messages

reaching the relevant

decision

makers/stakeholders?

18. Are CFS products easily

understood and accessible by

stakeholders?

Awareness at

global level

Awareness at

regional level

Awareness raising

at country level

fundamental to

achieve outcome

19. Number,

locus and form of

mention of CFS?

20. Visits to

website

21. Links to CFS

website

22. Print

articles (shared

by stakeholders)

23. Website

statistics

24. Online

survey

25. Follow up

interviews (what

they know about

CFS)

26. Statistics on

mention in

media, website

contacts, links,

tweets, etc.

27. Online

survey

28. Interviews

with the

“influence” panel

29. Survey of

sample of first-

time annual

meeting

participants

30. Survey of

users of CFS

products

31.

7

Evidence-based decision-making A

(Extent to which CFS decisions are based on evidence)

32. Are CFS decisions and

recommendations based on

evidence?

33. What evidence is

available for the CFS and

what mechanisms in place to

use it?

Degree to which

discussions and

decisions taken are

based on evidence

Responsiveness A

34. Is CFS responding to

requests for support, advice,

etc. on Outcome C coming

from stakeholders?

35. Are the requests to CFS

from stakeholders

36. Rate of

response to

requests

37. Number,

nature and source

of requests

38. Documentatio

n of action taken

39. Internal CFS

tracking

40. Documenta

tion review

41. Complete

enumeration of

requests

34

commensurate to the CFS

mandate (indication of

understanding of CFS)

Influence B

42. Do governments’

national and regional bodies

FSN plans reflect CFS

recommendations and

commitment for the RtF

43. Do regional and

national FSN policies,

programmes and budgets

reflect CFS input?

44. Who or what was most

influential in shaping your

FSN policy, program, etc.?

45. Do other stakeholders’

FSN policies reflect CFS

input?

46. Are countries setting up

multistakeholder platforms?

47. FSN plans,

policies,

programmes,

budgets

48. Interviews

49. Information

on platforms via

interviews and

online research

50. Key

informants

(decision-makers

and stakeholders

that were

involved in

planning

including civil

society groups

etc.)

51. Key

informants and

online

information

52. Relevant

ministries

53. Regional

bodies

54. Use of

multistakeholder

platforms if

functioning)

55. Document

review for

mention of or

reference to CFS

56. Structured

interview of key

informants to

assess

perceptions of

CFS role

57. Panel study

over time of 3

(high/med/low

FSN) countries in

each region

(same countries

over time)

58. Purposive

sample of key

FSN stakeholders

Note:

Interview via

Skype, with

some pre-call

prep work

Interviewees

to be selected

by respective

stakeholders

Pre-annual meeting online surveys

Surveys (questionnaire) and focus groups during annual meeting

Interviews during annual meeting

Panels over time: of individual countries, stakeholder bodies, etc.

Surveys, interviews, focus groups, etc. during meetings of other UN agencies or bodies

35

Annex 6: Results of the evaluation of the workshop

Rating between 1-5, with 1 = poor and 5 = excellent

ORGANIZATION 1 2 3 4 5

Workshop organization 1 8 8

Selection of venue 5 12

CONTENT 1 2 3 4 5

Relevance of the technical content 4 10 3

Clarity of hands out distributed 1 6 8 2

Attainment of workshop results 6 7 4

In line with my expectations 9 6 2

FACILITATION 1 2 3 4 5

Clarity on the process (how to achieve results) 1 7 7 2

PARTICIPANTS 1 2 3 4 5

Level of participation 1 2 7 7

WORKSHOP RESULTS 1 2 3 4 5

The objectives were achieved 2 10 4 1

The results are useful 1 6 7 3

OVERALL

1

2

3

4

5

Overall rating of the workshop 5 10 2

GENERAL COMMENTS / REMARKS:

- Important to report all the results of the workshop

- Conduct pilots in the countries regarding the diversity of the situation of food security

- Would have been better to work on questions that allow greater reflections on how to conduct the survey instrument (instead of working on matrix)

- Very impressive participation from CFS team! It was a pleasure to participate and learn more on CFS

- It was generally good. we hope that the report will reflect the views and opinions expressed

- Prior information, especially on the draft logic model (even in its draft form) would have been helpful

- Group work was commendable

- The outcomes mark starting steps; there may be need (even online) top further engage this group in taking these efforts to a logical conclusion

- Very difficult but useful process and I hope it has helped the CFS move this process forward. More background and insight into how outcomes were formulated would have been useful

- Need to have critical look at the best countries to measure the effectiveness and come up with meaningful instrument that will contribute to measure status and trends of CFS outcomes

36

- Was a very engaging workshop due and the efforts put in by the CFS team and FAO went a long way in keeping the level of participation so high

- All the best for developing a monitoring framework that all of us will be proud off

- I would like to be in touch in order to know about next steps in the design and implementation of the baseline study