project development evaluation

8/2/2019 Project Development Evaluation

1/51

EVALUATION

Pat Ray M. Dagapioso

Project Development


2/51

Project Development Cycle

Project Planning

ProjectImplementation

ProjectEvaluation


3/51

Project Evaluation

It is a systemic and objective assessment of anongoing or completed project (ILO TechnicalOperation Manual)

Systematic Investigation of the worth or meritof an object, i.e., project (NSF Handbook forProject Evaluation)


4/51

Why is Evaluation Important?


5/51

Evaluation is Important

The importance of Evaluation lies in its aims: A. Determine the level of achievement of the

project objectives B. Development effectiveness, Efficiency, Impact

and Sustainability C. Provides information to help improve the

project D. Building understanding, motivation and

capacity amongst involved in the project


6/51

Evaluation is Important

The importance of Evaluation lies in its aims: E. Helping empower project partners F. Provides information to help improve the

project, and G. Provides new insights or new information that

was not anticipated


7/51

Projects and its Impacts


8/51

Managing for Impact

Inputs

(Investments)

Outputs

(Products &Services

Provided)

Outcomes

(ImmediateObjective)

Impact

(Long-TermSustainable

Changes)


9/51

Managing for Impacts

Outcomes: Medium-term effects of project outputs. Observable changes that can be seen from project

interventions. Impacts

Positive and negative; and primary and secondaryeffects of a project intervention.

Often detectable after several years.


10/51

Levels of Evaluation


11/51


Program

Project

Activities

Project

Activities

Projects

Activities


12/51


Project Evaluation: Focuses on an individual project funded under the

umbrella of a program. Evaluation collects data to improve the project as

it develops and progresses. Information is also gathered for the purpose of

whether the project has proceeded as planned,and has it reached its programs goal or theprojects objectives


13/51

Types of Evaluation


14/51

Types of Evaluation (Who Does it?)

Types of Evaluation according to who does it: A. Self Evaluation B. Internal Evaluation C. Independent Evaluation D. External Evaluation


15/51

Evaluation Types

Evaluators Degree of Impartiality

Costs to theAgency

Self-Evaluation Agency/ProjectManagers Themselves Low Low

Internal Evaluation From the Agency butnot necessarily part of

the project

Medium Medium

IndependentEvaluation

External Evaluators +Independent Members

from the Agency

Medium to High High

External Evaluation External Evaluatorswith no previous links

with the project

High High


16/51

Frequency of Evaluation


17/51

Frequency of Evaluation Per ILO Technical Cooperation Manual, the frequency of

evaluation proceeds in: A. Technical cooperation projects of 18 months duration or

less require a final evaluation upon completion; B. Projects of 18 to 30 months duration require annual reviews

and a final evaluation upon completion; C. Projects of over 30 months duration require annual reviews,

an interim evaluation and a final evaluation upon completion; D. All projects with a budget of over US$ 500,000 require an

independent evaluation at least once during the project cycle;

and E. Independent evaluations are required before starting a new

phase.


18/51

Frequency of Evaluation Annual reviews

Self-Evaluations that are managed by the project managementitself.

Interim Evaluations

Takes Place halfway through the project implementation. Final Evaluations Independent Evaluation

Can either be final or interim evaluations. Additional Evaluation

Takes place when there is a need for readjustment of the projectobjectives

Additional Independent Evaluations Recommended for projects that lasts up to 4 years.


19/51

Types of Evaluation according to thePhase of the Project Types of Evaluation according to its Phase:

A. Formative B. Summative


20/51

Formative Evaluation

Formative Evaluation assesses initial and on-going project activities.

2 Components: A. Implementation Evaluation B. Progress Evaluation


21/51


Implementation Evaluation The purpose of Implementation evaluation is to

assess whether the project is being conducted as

planned.


22/51


Progress Evaluation The purpose of progress implementation is to

assess progress in meeting goals of the program

and the project. This evaluation measures the progress at the

various stages of intervention.


23/51

Summative Evaluation

Summative evaluation assesses the qualityand impact of a fully implemented project.

Summative Evaluation collects informationabout outcomes and related processes,strategies and activities.


24/51

Summative Evaluation

Basic questions: A. To what extent does the project meet the

stated goals? B. Are greater number of the target population

has shown significant changes? C. Which components are effective? Which are

not? D. Were the results worth the cost? E. Is the program sustainable?


25/51

Evaluation Phase Proper


26/51

What to Evaluate?

Evaluation Criteria: A. Relevance and Strategic fit of the project B. Validity of the Project Design C. Project Progress and Effectiveness D. Efficiency of resource use E. Effectiveness of the Management Arrangement F. Impact Orientation and Sustainability of the

Project


27/51

Relevance and Strategic Fit

Does the project address a relevant need anddecent work deficit?

Was a needs analysis carried out at thebeginning of the project reflecting the variousneeds ofdifferent stakeholders?

Are these needs still relevant?


28/51

Relevance and Strategic Fit Have new, more relevant needs emerged that

the project should address? How does the project align with and support

national development plans, thenationalpoverty reduction strategy (PRS), nationaldecent work plans, national plans of action onrelevant issues (e.g., on employment creation,child labour, antitrafficking, etc.), as well asprogrammes and priorities of the nationalsocial partners?


29/51

Validity of Design

What was the baseline condition at the beginningof project? How was it established? Was agender analysis carried out?

Are the planned project objectives and outcomesrelevant and realistic to the situation on theground? Do they need to be adapted to specific

(local, sectoral, etc.) needs or conditions? Is the intervention logic coherent and realistic?

What needs to be adjusted?


30/51

Project Progress and Effectiveness Is the project making sufficient progress towards

its planned objectives? Will the project be likelyto achieve its planned objectives uponcompletion?

Have the quantity and quality of the outputsproduced so far been satisfactory? Do thebenefits accrue equally to men and women?

Are the project partners using the outputs? Havethe outputs been transformed by projectpartners into outcomes?


31/51

Project Progress and Effectiveness How have stakeholders been involved in project

implementation? How effective has the project been in

establishing national ownership? Is project management and implementation

participatory and is this participationcontributing towards achievement of the projectobjectives?

Has the project been appropriately responsive tothe needs of the national constituents andchanging partner priorities?


32/51

Project Progress and Effectiveness

Has the project been appropriately responsiveto political, legal, economic, institutional, etc.,changes in the project environment?

Has the project approach produceddemonstrated successes?


33/51

Efficiency of Resource Use Have resources (funds, human resources, time,

expertise, etc.) been allocated strategically toachieve outcomes?

Have resources been used efficiently? Have activities supporting the strategy been cost-

effective? In general, do the results achieved justify the

costs? Could the same results be attained withfewer resources?

Have project funds and activities been delivered

in a timely manner?


34/51

Effectiveness of ManagementArrangements Are management capacities adequate? Does project governance facilitate good

results and efficient delivery? Is there a clear

understanding of the roles and responsibilitiesby all parties involved?

Does the project receive adequate political,technical and administrative support from itsnational partners? Do implementing partnersprovide for effective project implementation?


35/51

Impact Orientation and Sustainability

Can observed changes (in attitudes, capacities,institutions, etc.) be causally linked to theprojects interventions?

In how far is the project making a significantcontribution to broader and longer-termdevelopment impact? Or how likely is it that itwill eventually make one?

Is the project strategy and project managementsteering towards impact?

What are the realistic long-term effects of theproject on the poverty level and the decent workcondition of the people?


36/51

Impact Orientation and Sustainability

Is there a need to scale down the project (i.e., if theproject duration is shorter than planned)?

Can the project be scaled up during its duration? If so,how do project objectives and strategies have to beadjusted?

Has the project successfully built or strengthened anenabling environment (laws, policies, peoplesattitudes, etc.)?

Are the project results, achievements and benefitslikely to be durable? Are results anchored in nationalinstitutions and can the partners maintain themfinancially at end of project?


37/51

Methodologies Used to Acquire Data


38/51

Techniques Used to Acquire Data

1. Survey 2. Interview 3. Focus Groups 4. Observation

5. Tests 6. Document Studies 7. Key Informant 8. Case Studies

Advantage/Disadvantages: Here .
http://nsf02057.pdf/http://nsf02057.pdf/


39/51

Evaluation Report


40/51

Evaluation Report Before any evaluation report will be presented, a

terms of reference (TOR) of the evaluation will befirst presented (and prepared by the evaluationmanager) for comments to the followingstakeholders (ILO) and before the evaluationproper:

A. Project Manager B. Main National project partners

C. ILO Field Office Director (for ILO only) D. Field Technical Specialist E. Donor


41/51

Terms of Reference Contents of the Terms of Reference of the

Evaluation: A. Introduction and rationale for evaluation

B. Brief background on project and context C. Purpose, scope and clients of evaluation D. Suggested analytical framework E. Main outputs F. Methodology to be followed G. Management arrangements, work plan and

timeframe


42/51

What happens with the termscontaining in the TOR?


43/51

Once the TOR is approved by the ProjectManagement, the evaluation manager can then

undergo the proper evaluation phase as

stipulated in the TOR.

The evaluation manager is assured of efficient useof resources:

1. Gathering relevant information prior and duringthe evaluation

2. Ensuring the efficient use of time3. Arranging for the evaluator to meet the right

people4. Scheduling interviews and meetings with

partners, and5. Ensuring efficient logistical arrangements.


44/51

Evaluation Report The Evaluation Report Contains:

Abstract Brief background on the project and its logic Purpose, scope and clients of evaluation Methodology Review of implementation Presentation of findings regarding project performance Conclusions Recommendations Lessons learned Annexes


45/51

Evaluation Report

Example of an Evaluation Report (per WorldBank):

Here.
http://eval-ev2.pdf/http://eval-ev2.pdf/


46/51

People Who Evaluate


47/51

Evaluation Managers

The evaluation manager is responsible formanaging all independent and internalevaluations.

The evaluation manager should have no linksto the project decision-making and henceshould not be the technical or administrative

backstopper of the project.


48/51

Evaluation Managers Roles:

Determine the target audience for the evaluation andthe key evaluation questions theevaluation shouldanswer;

Prepare the draft TOR for the evaluation (finalapproval is given by the evaluation focal person) andsend a copy of the approved TOR for information;

Identify the evaluation consultant(s), and obtain finalapproval for their recruitment from the evaluationfocal person;

Ensure smooth organization of the evaluation processand proper support to the evaluation team;


49/51

Evaluation Managers Roles:

Ensure proper stakeholder involvement in the entireevaluation process;

Ensure that gender issues are considered throughout

the evaluation process; Manage the process of preparing the evaluation

report (including circulating the draft report andcollecting comments);

Submit the final evaluation report to the evaluationfocal person for final review

Ensure proper follow-up on the recommendations anddissemination of lessons learned


50/51

Evaluators The evaluator carries out the evaluation and

prepares the evaluation report according totheTOR.

Roles: Adhere to internationally-accepted good practices and

solid ethical principals; Be skilled in implementing diverse evaluation

methodologies;

Ensure the evaluation is an inclusive and participatorylearning exercise; and Be culturally- and gender-sensitive


51/51

The End.

Thank You!

project development evaluation

Documents