Download - Project Evaluation, Recycling and Closing
Project Evaluation and
Recycling
Prepared by: Michael John D. Sison MDM student
Presented to: Josefina B. Bitonio, DPA
Professor
DM 211 Project
Development and
Management
RANGESAMY, 2008
Project recycling
Topics at glance
Project tools and
techniques
Project evaluation
and recycling
d. Stages
of project life
Project Evaluation and Recycling
Monitoring is defined as the routine collection and use of data to assess progress in achieving programmed objectives. These data are generally derived from program records
What is monitoring?
What is a project?
Project
Manager
A project is a
package of:
Objectives
Outputs
Activities
inputs
Project
Staff Project
Beneficiaries
Mercado: 2005
What is the origin of evaluation?
Evaluation has its origin in the Latin word “Valupure” which means the value of a particular thing, idea or action (Rengasamy, 2008).
Evaluation is the process of analyzing program and project inputs, transformation processes , and the effect and impact of outputs and outcomes against explicitly stated goals and objectives (Mathur and Inayatullah as cited by Miclat, 2005).
What is evaluation?
Differences between monitoring and evaluation (United Nations Evaluations Group, accessed 2014)
Project evaluation assesses activities that are designed to perform a specified task in a specific period of time. For example, a three-day workshop on volunteerism, a one-year fundraising program, a three-year HIV prevention intervention, a five-year career development innovation, or an ongoing newcomer youth orientation and integration training program are all projects that you can evaluate (Zarinpoush, 2006).
What is Project
Evaluation?
• To assess project results
• To improve project management and process planning
• To promote learning
• To understand different stakeholders’ perspectives
• To ensure accountability
In general, the purpose of monitoring & evaluation based in Rengasamy (2008) can be:
1. Efficiency refers to the amount of time and resources put into the project relative to the outputs and outcomes. A project evaluation may be designed to find out if there was a less expensive, more appropriate, less time-consuming approach for reaching the same objectives. 2. Effectiveness describes whether or not the research process was useful in reaching project goals and objectives, or resulted in positive outcomes. 3. Relevance or appropriateness describes the usefulness, ethics, and flexibility of a project within the particular context.
Underlying reasons (Rengasamy, 2008) for monitoring and evaluating are frequently framed in terms of:
What will happen if we mix the three?
It will enable judgment about whether the outputs and outcomes of the project are worth the costs of the inputs. Effectiveness, efficiency and appropriateness can be considered for the different methods, tools and approaches rather than questioning the value of the research approach as a whole (Rengasamy, 2008).
What to Monitor and
Evaluate (Rengsamy,
2008)? Outputs
• describe the concrete and tangible products of the research as well as the occurrence of the research activities themselves.
Processes
• describe the methods and approaches used for the project.
Impact
• describes overall changes that occur in the community to which the project is one of many contributing factors.
Reach
• describes who is influenced by the project and who acts because of this influence.
Outcomes
• describe the changes that occur within the community or with the project managers that can be attributed, at least in part, to the project process and outputs.
Understanding the
condition of the
community/target group
before the project was
initiated is useful in order
to provide a point of
comparison for monitor
and evaluating changes
that occur during the
project. Baseline survey
conducted at the
beginning of the project
can provide a point of
reference for comparison
and for understanding
change in the
community/target group.
Program Project Logmap
Mercado:2005
User
(Community)
Effects Personal
Outcomes Organization
Impacts Social
Improved:
1. Capacity
2. Efficiency
3. Effectiveness
4. Performance
5. Productivity
6. Quality
7. Profitability
8. Empowerment
9. Access
10. Availability
Increased:
1. Per capita
income
2. Employment
raised
3. Life expectancy
4. Literacy rate
Decreased
1. Mortality rate
2. (In)fertility rate
Consumers
Objectives
Improved:
1. Awareness
2. Knowledge
3. Attitude
4. Skills
5. Practice
Purpose of Evaluation (Rengasamy, 2008)
From an accountability perspective:
It is to make the best possible use of funds by the program managers
who are accountable for the worth of their
programs.
Measuring accomplishment in
order to avoid weaknesses and future mistakes.
VERSUS From a knowledge perspective:
Understanding people’s participation & reasons for the same.
It is to establish new knowledge about social problems and the effectiveness of policies and programs designed to alleviate them.
Evaluation helps to make plans for future
work.
Examples of Evaluation Purpose Statements (Zarinpoush, 2006)
• To assess the degree to which project objectives were achieved.
• To document the lessons learned.
• To provide recommendations for project development and improvement.
• To examine the changes that resulted from doing the project.
• To provide input to guide decision making for the upcoming renewal and extension of project funding.
Principles of Evaluation (Rengasamy, 2008):
It has a sense of
continuity.
It has a sense of
continuity.
It is inexpensive
It has a minimum hindrance to day-to-day work.
It has a total
participation, thus,
shows program totality.
It can be shared.
Preparing the Evaluation Study Proposal (Miclat, 2005)
Theoretical Framework
• The launching pad of the study from the abstract into the empirical world.
Methodology
• It discusses the methods and procedures to be used in the study.
Research Designs
• The blueprint of procedures that enable a researcher to arrive at valid and reliable findings and conclusions.
Sampling Technique
• Involves the population or universe to guarantee the validity of the study.
Research Instrument
• The data gathering device a researcher uses to gather data relevant to the study.
Treatment of Data
• The procedures on how the data would be analyzed taking into consideration the objectives of the study.
Additional two items on Methodology (Miclat, 2005)
The preparation of a research budget to carry out the study successfully should be stipulated.
Funding Requirements
The preparation of a time schedule in the conduct of the various activities of the evaluation study to maximally use time, money and manpower should also be included.
Timetable
●Impact evaluation
●Ex- post evaluation or (Summative / Terminal / Final)
●Final evaluation
3. PROGRAM COMPLETION
STAGE
●Monitoring Evaluation or Ongoing / interim.
●Concurrent evaluation
2. PROGRAM MONITORING STAGE
1. PROGRAM PLANNING STAGE
STAGES IN PROJECT EVALUATION (RENGASAMY, 2008)
●Pre – investment
evaluation
●Formative evaluation
●Ex – ante evaluation or
Early / Formulation
●Pre-project evaluation
●Exploratory evaluation
●Need assessment
●Formative evaluation
●Ex – ante evaluation or
Early / Formulation
●Pre-project evaluation
●Exploratory evaluation
●Need assessment
CRITERIA FOR DEVELOPMENT ASSISTANCE (RANGESAMY, 2008)
STEPS IN EVALUATION (RANGESAMY, 2008)
Learning about the program
Creating on evaluation plan and evaluation indicators
Briefing the concerned people
about the evaluation plan and indicators
Revising and elaborating the evaluation plan
Initiating evaluation Utilizing/Sharing
information
Implementing the Evaluation Study (Miclat, 2005)
Data Gathering
and Processes
Writing the Research
Report
Presentation and Analysis
of Data
Finding and Conclusions
Types of Evaluation:
Evaluation can be categorized under different headings (Rangesamy, 2008): A) By timing (when to evaluate) ●Formative Evaluation ●Summative Evaluation B) By Agency. Who is evaluating? ●Internal Evaluation ●External Evaluation C) By Stages ●On going ●Terminal ●Ex – post
Internal or External Evaluation (based on Rangesamy, 2008)
Internal Evaluation
It is also known as Enterprise Self
Audit.
It is a continuous process which is done at various points and in respect of various aspects of the working of an agency by the agency staff itself i.e. staff board members and beneficiaries.
VERSUS Internal
Evaluation
This is done by outsiders /Certified Management Audit.
Some donors may send consultants in order to see how far the standards laid down are put into practice.
Inter agency evaluation. In this type two agencies mutually agree to evaluate their program by the other agency.
Inter agency tours.
Desired Situation Sustained benefits
and impact
Present Situation
Mid-Term review End-of project or
final evaluation Ex-post or impact
evaluation
Time
Source: Rangesamy, 2008
Tools/ Techniques based on Rangesamy, 2008:
● First Hand Information ● Formal/Informal Periodic Reports ● Evaluation is also carried out through formal/informal reports *Formal Reports consist of: -Project Status Report -Project Schedule Chart -Project Financial Status *Informal reports such as: ● Anonymous Letters, Press Reports, Complaints by Beneficiaries & Petitions, etc. ● Graphic Presentations ● Standing evaluation review committees ● Project Profiles
Methods of Evaluation
Areas of Evaluation (Rangesamy, 2008):
Purpose: The review the objectives of the agency / project and how far these are being fulfilled. Programs: Aspects like number of beneficiaries, nature of services rendered to them, their reaction to the services, effectiveness and adequacy of services etc. may be evaluated. Staff: The success of any welfare program / agency depends upon the type of the staff an agency employs. Their attitude, qualifications, recruitment policy, pay and other benefits and organizational environment. These are the areas which help to understand the effectiveness of the project / agency.
Financial Administration:
The flow of resources and its consumption is a crucial factor in any project / agency. Whether
the project money is rightly consumed any over spending in some headings, appropriation and
misappropriation. These are some of the indicators that reveal the reasons for the success or
failures of any project.
General:
Factors like public relations strategies employed by the project /agency, the constitution of the
agency board or project advisory committee and their contribution future plans of the agency
are important to understand the success or failures of any project.
PROJECT EVALUATION (RANGESAMY, 2008)
Analysis on how successful the project has been in Transforming the means (i.e. the resources and inputs allocated to the project) through project activities into concrete project results Provides the stakeholders with information on
inputs/costs per unit produced
Overall Objectives
Efficiency
Means + Preconditions
Activities+ Assumptions
Results + Assumptions
Project Purpose + Assumptions
Change
utilization
action
allocation
Analysis on how well the production of project
results Contributes to the achievement of the
project purpose,
i.e.: Are there clear Indications of changes and
improvements that benefit the beneficiaries of the
project?
Uses base-line information on the pre project
situation as a starting point
Effectiveness
Impact
Analysis of the overall effects of the project
Analysis of the contribution of the project purpose to
the overall objectives
Focus on long-term changes in the environment of the
project “Collection” and analysis of information at
the levels of communities and society at large
focusing on the final beneficiaries of the project
Also analysis of unintended impacts (negative
and positive)
Views About Evaluation (Rangesamy, 2008):
● Evaluation as an analysis – determining the merits or deficiencies of a program, methods and processes. ● Evaluation as an audit – systematic and continuous enquiry to measure the efficiency of means to reach their particular preconceived ends. In the Agency Context: ● Evaluation of administration means appraisal or judgment of the worth and effectiveness of all the processes (e.g. planning, organizing, staffing, etc.) designed to ensure the agency to accomplish its objectives.
What is Project
Recycling?
Definition of Project Recycling
This is a process of iterations and reiteration. This is a post project activity wherein a separate planning based on results will have to be prepared. Depending upon the results of the project reports/analyses a project may be replicated or modified to suit the time, place, beneficiaries of the intended project (CLSU). The results of outputs and outcomes in terms effects and impacts are eventually recycled and feed-backed as inputs in the planning process (Miclat, 2005).
Is there a need for publication and dissemination of the evaluation?
Publication and dissemination of the evaluation
Project Evaluation and Recycling Model (Sison, 2014)
Data Gathering
and Processes
Writing the Research Report
Presentation and Analysis
of Data
Finding and Conclusions
Publication and Dissemination of
the Evaluation
Plan Update
Project Recycling, Re-modification
or Removal?
Embark on ‘new’ Project
Evaluation
Evaluation Study Implementation by Miclat, 2005
Stages of Project Life (Bitonio, 2014)
Project Initiation
Project Planning
Project Execution
Project Closure
Communication
What are your answers?
• What went right?
• What went wrong?
• How do you make the next project process better?
• Best Practices?
At the end of the Project
Project Initiation
Project Planning
Project Execution
Project Closure
Post Implement
Review
Monitor and Control Plan the
Project
Define the Project
Communication
• Client Satisfaction input/report
• Formal acceptance
• Project Schedule (Actual vs.
Estimated)
• Lessons Learned
Project Life with Recycling Cycle (Sison, 2014)
Communication
Publication and Dissemination
of the Evaluation
Plan Update
Project Recycling, Re-
modification or Removal?
Embark on another Project
with ‘new’ criteria for
Project Evaluation
Project Initiation
Project Planning
Project Execution
Project Closure
Post Implement
Review
Monitor and Control
Plan the Project
Define the Project
Closing the Project
Closing the Project
• Closing projects involves gaining stakeholder and customer acceptance of the final products and services, and bringing the project to an orderly end.
• Closing includes verifying that all of the deliverables are complete, planning for transition of products and services into the organization, and preparing a final presentation and report.
• It is also important to reflect on what can be learned to improve future projects. As philosopher George Santayana said, “Those who cannot remember the past are condemned to repeat it.”
Source: sbuweb.tcu.edu/mackay/INSC%2040813/Lectures
Closing a project involves
creating:
– Administrative closure procedures
– Contract closure procedures
– Final products, services, or results
– Updates to organizational process
assets
Source: sbuweb.tcu.edu/mackay/INSC%2040813/Lectures
• In closing out a project, you must prepare:
– A customer acceptance/project completion
form
– A final report and presentation
– A transition plan (provided as part of the final
report)
– A lessons-learned report (after a “sticky note”
party)
• Organizing an event/celebration/luncheon, etc.
for the project team to celebrate a job well done.
Source: sbuweb.tcu.edu/mackay/INSC%2040813/Lectures
Sample Table of Contents for a Final Project Report
Sample Customer Acceptance
/Project Completion Form
Source: sbuweb.tcu.edu/mackay/INSC%2040813/Lectures
Source: sbuweb.tcu.edu/mackay/INSC%2040813/Lectures
Sample of Lessons Learned Report
Lessons Learned Report Cont.
Source: sbuweb.tcu.edu/mackay/INSC%2040813/Lectures
References:
BOOK Miclat, Jr. Eusebio F. Strategic Planning in Education: Making Change Happen. Rex Book Store, Inc. 2005. WEB Rengasamy, Srinivasan. There is No Management without Monitoring. slideshare http://www.slideshare.net/srengasamy/project-monitoring-evaluation-s-presentation?utm_source=slideshow&utm_medium=ssemail&utm_campaign=download_notification Posted Oct. 11, 2008 Accessed March 10, 2014 Bitonio, Josefina B. Project Development and Management 2014 ________________. Project Recycling. Chapter7.doc syllabus clsu Zarinpoush, Fataneh Project Evaluation Guide for Nonprofit Organizations. Imagine Canada. 2006 ________________________. Toolkit to Combat Trafficking in Persons. United Nations Evaluation Group. 08-58296_tool_10-1.pdf
buweb.tcu.edu/mackay/INSC%2040813/Lectures
#ProjEvalReport
to evaluate!