documents.wfp.orgdocuments.wfp.org/stellent/groups/public/documents/rep… · web viewassessment...

Decentralized evaluation for evidence-based decision makingWFP Office of Evaluation

Decentralized Evaluation Quality Assurance System (DEQAS)

Quality Checklist for Decentralized Evaluation ReportVersion November 2015

[title of the decentralized evaluation]Overall

General Comments/Status

Length: Report is concise, not exceeding 40 pages excluding Annexes and Executive SummaryAccessibility Written in a clear and accessible manner The structure of the findings allow readers to

understand evaluation findings Key messages are summarised and highlighted The report presents a good balance between descriptive

and analytical information Terminology is used correctly The report is free from grammar, spelling, or

punctuation errors The tone of the report is constructive The language used in the report is simple and clear and

the report is reader friendly

Credibility The report presents successes and failures, positive and

negative sides in a balanced way.

T O R Q C V e r s i o n O c t o b e r 2 0 1 5 P a g e 1 | 10

The report raises critical issues when necessary The report’s findings and conclusions are fully evidence-

based.Editing

WFP format and templates are applied Acronyms are spelt out the first time they are used Paragraphs and pages are numbered electronically Table of contents is included and lists tables, graphs,

figures and annexes Tables and diagrams are used as relevant and are

numbered List of acronyms is included When data or quotes are provided, the report indicates

the source (either in a footnote or directly below a table/graph).

Cover Page

Uses WFP DEQAS Report Template Title of the decentralized evaluation is identical to that

in the TOR Date and status of the report (draft/final) indicated on

the cover page

Executive SummaryOverall: The Executive Summary should provide a complete and balanced synthesis of the evaluation findings, conclusions and recommendations in not more than 4 pages.

Expected Content Assessment criteria Comments/status

Introduction: main features of the evaluation including evaluation purpose and objectives, main users/ intended audience, context, subject of the evaluation.

Executive Summary is coherent, self-explanatory and self-contained

The key findings form the largest part of the

D E R e p o r t Q C – V e r s i o n N o v e m b e r 2 0 1 5 P a g e 2 | 10

Methodology Key findings on all the

evaluation questions Overall conclusions Recommendations

section All recommendations

are presented in brief The Executive

Summary could be used as a standalone aid to decision-making

Length does not exceed 4 pages

1. IntroductionOverall: Succinct overview of the evaluation subject, context and features, providing clarity on why and how the evaluation was carried out, in not more than 5 pages.


Clear reasons for the evaluation

Specific objectives Scope of the evaluation Stakeholders in the evaluation Intended users and how they

will use the evaluation findings

Introduction clearly sets the scene for the evaluation, including its rationale, scope, stakeholders and users.

Information is succinct and focused

1.1 Overview of the Evaluation SubjectFactual information regarding the evaluation subject including:

Type of intervention (operation, activity, thematic area, transfer modality, pilot project) Geographic scope of the evaluation subject Relevant dates: Approval date; start date; end date

Information is relevant and important to understanding the subject of the evaluation:o What it iso When it was designed o What are the key

inputs ($ value) o What are the planned

outputs? (beneficiaries, MT,


Planned outputs at design o Beneficiary numbers

(planned and revised) disaggregated by gender/activity

o Amount of transfers (food, cash, vouchers)

o Any other outputs Planned outcomes at design Key activities Main partners (Government; NGOs; Bilateral; Multilateral) Resources (% funded of total requirements) and key donors. If subject funded from pooled funds or is a component/activity of an operation, show resource allocated to the subject of the evaluation Assessment of the Logical Framework or similar tool from evaluation perspective Other relevant preceding/ concurrent activities/ interventions/operations Any amendments to initial design Include reference to:o Past evaluations/reviews related to the subjecto Maps/graphs for illustration

Cash &Voucher $)o What is the

target/scope?o What are the planned

outcomes? o Who is involved in its

implementation? Highlights relevant issues

from past evaluations and reviews that are relevant to the evaluation

Gender dimensions explained

Differences between original design and implementation are explained if appropriate

1.2 ContextOverview of the surrounding context directly relevant to the

Information is relevant and necessary to understand the


evaluation including:

Poverty, food and nutrition security

Government policies and priorities

Humanitarian issues Gender dimensions of the

context Key external events Features of international

assistance in the area Other WFP work in the area Work of other key actors

implications for the evaluation subject

Relevant indicators and latest information have been identified and trend data used

Gender context of country is described

Information draws from and is consistent with inception report, updated where appropriateInformation is explicitly geared to the evaluation subject, rather than being generically presented.

1.3 Evaluation Methodology and Limitations

Methodology used for the evaluation is summarized

Evaluation criteria applied to the evaluation are listed and justified (and if possible interpreted for the subject)

The main evaluation questions are set out

Summary of the overall methodological approach is provided

Specific methods and sampling used are provided

A statement on how validity and reliability was addressed is included

Limitations of methods or data

Information allows reader to understand how the evaluation was conducted

Methodological approach is comprehensive and presents a systematic approach that generates trust in the credibility of the evaluation

Methodological approach is coherent, logical and in line with the TOR and inception report

Specific methods and D E R e p o r t Q C – V e r s i o n N o v e m b e r 2 0 1 5 P a g e 5 | 10

availability encountered, and mitigation strategies applied/implications for the findings, are explained

Gender considerations for the data collection methods are explained

Full details in a specific Annex

sampling applied are relevant to the methodological approach and are feasible/appropriate

Limitations and their effects on the evidence base are included

Statement on validity and reliability generates confidence in the evidence base

Methods demonstrate consideration of gender equality and women’s empowerment

Full methodology provided in an Annex (updated from Inception Report in relation to its actual implementation).

2. Evaluation FindingsOverall: This section should be the most substantial of the evaluation report. It should present information that supports the conclusion(s) of the evaluation team against the key evaluation questions. It should not exceed 30 pages.


Presentation of clear data analysis against all evaluation questions, including triangulated information

Findings are substantiated

The findings have been formulated clearly and are based upon evidence collected

Findings are triangulated Findings use credible


by credible evidence that has been checked for accuracy, consistency and reliability

Limitations or gaps in evidence are presented

Indication of where evidence is inconclusive

Information sources are provided and accurately quoted

Findings are adequate to substantiate the conclusions and recommendations made by the evaluation team (next section)

Findings use gender (and equity) sensitive language throughout and data is disaggregate.

Findings are presented in a gender and culturally sensitive manner.

external sources of information

Findings address the evaluation criteria chosen

Information sources are provided, and where evidence is inconclusive this is clearly stated.

Findings address any limitations or gaps in evidence

Visual aids are used in a clear an accessible way to illustrate the argument

Information sources are quoted

The findings balance detail and synthesis

Findings reflect gender analysis and address gender sensitivity including use of gender disaggregated data Findings are presented by evaluation question

3. Conclusions and RecommendationsOverall: Conclusions and Recommendations are core components of the evaluation report; Lessons learned and Good Practices should be included if appropriate. The section should not exceed 5 pages.


3.1 Overall Assessment/Conclusion

Conclusions are logically derived from

Conclusions follow logically from the


findings and substantiated by evidence

There is clear referencing back to findings

Report contains a brief and balanced assessment of the evaluation findings related to the selected evaluation criteria and questions. This may be organised by criteria, theme and/or by evaluation question

findings, with clear and explicit linkage to them

A conclusive picture is presented, formed from the previous sections of the report

A summary against evaluation criteria is included

Conclusions are succinct, synthesising common findings

Conclusions include recognition of different stakeholder groups and gender aspects

No new evidence is introduced that has not been presented in the findings

Conclusions are impartial and free of bias

3.2 Lessons Learned and Good Practices (optional)

Lessons are presented that WFP and partners can use in future contexts to improve WFP projects and programmes.

Lessons include any positive and negative lessons

Good practices are presented which represent successful

Lessons contribute to wider organizational learning

Lessons specifically indicate what should be repeated or avoided in future contexts to guide future action

Good practice is potentially replicable

Good practices describe why things have


strategies or interventions that have performed well;

Emerging good practices can be presented (i.e. an intervention that has potential to perform well)

worked

3.3 Recommendations

Short paragraphs describing up to 10 recommendations, which are all relevant to the subject of the evaluation

Recommendations logically derive from the findings and conclusions and are clearly linked to the evidence base

Recommendations are realistic/feasible

Recommendations are prioritised, sequenced and addressed to specific target groups

Gender issues are addressed where appropriate in recommendations

Recommendations follow logically from the findings and conclusions

They are based on the analysis and evidence collected

They are realistic (implementable), presenting plausible options for improvement

They are clear and specific and clustered /presented in a logical order

They are prioritised and have a clear timeframe for implementation

The target group for each recommendation is clear and relevant to the intended users of the evaluation

Gender is integrated into the recommendations and where feasible,


recommendations for improving WFP’s performance on gender are provided.

Annexes Comment Comments/status

Annexes should support and expand on text in the main report. Including: Terms of Reference Documents Reviewed Stakeholders

Interviewed Evaluation matrix Data Collection tools Bibliography List of Acronyms

Annexes listed and numbered

Annexes referenced, where appropriate in the main report

Data collection tools are included

Not all working documents are included
