5 day m&e presentation compiled

8/3/2019 5 Day M&E Presentation Compiled

1/87

1

Address: Shahri Naw, between Hanzala Masjid and Blossom Hospital, Street #01, House # 50, Kabul, AFG

A Leader and Pace-setter in Capacity Development by Offering High QualityTrainings & Consultancy Services


2/87

2

Monitoring & Evaluationof Projects

Part 09

by: Samandar MahmodiM&E, SPSS and Survey Design Specialist


3/87

3

What is Monitoring & EvaluationThe collection of data and information inprograms and projects, in combination

with its analysis and use.Monitoring and Evaluation arecomplimentary, but distinct functions


4/87

4

What is Monitoring?

Monitoring is the regular and continues collectionof data and information on specific indicators to:

Demonstrate to management and key

stakeholders the extent of progress, and showprogress in relation to what was planned

Assist in timely decision-making processes

Ensure accountability and Transparency

Provide the basis for evaluation and learning

identify implementation problems oropportunities


5/87

5

What is Evaluation?

Assesses relevance, efficiency, effectiveness,impact and sustainability in relation to projectobjectives.

Supports strategic decision making processThe systematic and objective assessment of anon-going or completed project, program or policy,its design, implementation and results. The aim is

to determine the relevance and fulfillment ofobjectives, efficiency, effectiveness, impact andsustainability.


6/87

6

Evaluation ContdEvaluation relies on data generatedthrough monitoring activities as well asinformation obtained from other sources(e.g., studies, research, in-depthinterviews, focus group discussions,surveys etc.). Evaluations are often (but

not always) conducted with theassistance of external evaluators. And itcould be Formativeor Summative


7/87


8/87

8

Why is M&E Important?Monitoring and evaluation helps program implementers to:

Make informed decisions regarding program operationsand service delivery based on objective evidence

Ensure the most effective and efficient use of resources

Objectively assess the extent to which the program ishaving or has had the desired impact, in what areas it iseffective, and where corrections need to be considered

Meet organizational reporting and other requirements, and

convince donors that their investments have beenworthwhile or that alternative approaches should beconsidered


9/87

9

When do we Monitor and Evaluate?

We monitor to know whether, are we on the right path toachieve what we have planned or not

To find problems and weaknesses, solutions for problemsand improvements for weaknesses.

To find opportunities and strengths

M&E is an integral part of every project, from ()

M&E should be planned during the (.) of a project

M&E should be executed by all individuals and institutions

which have an interest in the projectMonitoring should normally be planned according tooperation/implementation plans


10/87

10

Project Implementation and M&E

Look! This is a project, a projectinvolves several phases from startto finish.

1. You see a situation

2. Plan for a project to improve thesituation

3. Plan for improvements, selecttargets

4. Carry out the project

5. Produce outputs, outcomes andfinally impact

6. All these stages should bemonitored and evaluated whetherthey are as intended or not.


11/87

11

Illustration of M&E in a Project

M&E will not disappear problem, or to curethem, and is not a solution, but they arevaluable tools. And can:

1. Help you identify problems and theircauses

2. Suggest possible solutions to problems

3. Push you to reflect on where you aregoing and how you are getting there

4. Provide you with information and insight

5. Encourage you to act on the information

and insight

6. Increase the likelihood that you will makea positive development difference


12/87

12

Discussions

What did you learn so far?

Questions and Answers.


13/87

13

Monitoring Vs. Evaluation

Monitoring Evaluation

What Inform.. Directly available info about implementation More detailed information; harder to get

Purpose Determines if activity proceeding asintended

To see whether objectives are being reached andLearning lessons for future

Timing Regularly during implementation Periodically and after completion

Sources Project Records Monitoring data, plus external information

Use of Results To improve quality of implementation, andadjust planning

To judge the impact, adjust objectives, decide aboutthe future of the project

Done by Project Staff Outsiders

Done for Management Funders and Planners


14/87

14

Group Discussion

Distinguish Monitoring and Evaluation.

Your Practical Monitoring and Evaluationexperience in differentiating M&E.

Distinguish the followings: A Survey to measure overall targets you have set?

A trip to Balkh province to see how the

implementation of survey is going? A trip to Nangarhar to see how the project is being

implemented?


15/87


16/87

16

Different ways of doing an Evaluation

Rapid Participator Appraisal: Originally used in rural areas,while applicable in most communities. This is a qualitativeway of doing evaluations in a period of short time. It is usedas a starting point for understanding a local situation and is aquick, cheap and useful way to gather information. It involves

the use of secondary data review, direct observations, semi-structured interviews. It is flexible and interactive.

External Evaluation: Done by a carefully chosen outsider.

Internal Evaluation: Done by internal staff of the organization

of different organizational units.

Interactive Evaluation: This involves a very active interactionbetween outside evaluator and the organization beingevaluated.


17/87

17

When do we need M&E results

during the Program Cycle?During situation analysis: identification ofoverall program focus, lessons learned frompast program implementation are studied and

taken into account in the program strategies;During program design: data on indicatorsproduced during the previous program cycleserve as baseline data for the new program

cycle. Indicator data also enable programdesigners to establish clear program targetswhich can be monitored and evaluated


18/87

18

When do we need M&E results during the

Program Cycle? Contd

During program implementation: monitoringand evaluation ensures continuous tracking of

program progress and adjustment of programstrategies to achieve better results;

At program completion: in-depth evaluationof program effectiveness, impact and

sustainability ensures that the program wasenough effective and that lessons on goodstrategies and practices are available fordesigning the next program cycle.


19/87

19

Types of Evaluation

Formative

Summative


20/87

20

Formative EvaluationTo improve the design and performanceof an on-going program / project

Mid-term Evaluation or Mid-term Review


21/87

21

Summative EvaluationTo make an overall judgment about theeffectiveness of a completed program,often to ensure accountability

Ex-post EvaluationImpact Assessment


22/87

22

Management Information System (MIS)

An operational system that is capable ofcommunicating or reporting physical andfinancial information to relevant project

management staff, with a clear definition ofwho collects what information, how and when.

Or a database system used to collect project

data and information of different section, andpatterns.


23/87

23

Management Information System (MIS)

Everything you monitor and evaluate should berecorded and kept in a structured system, both inhard and soft copies. And a rule of thumb in M&Esays that nothing should go in MIS system without

a proof and a hard copy in your files. Therefore,there is a very close link between MIS and M&E,that you should keep your M&E data in a wellfunctioning MIS system.

The hard copies or files of proof for what you have

reported, will be very useful in audit, or when justsomeone asks where did you get this information.


24/87

24

MIS and M&E Relationship

Data

InformationImpact

EvidenceAction

Knowledge

Organize

analyze

Integrate (MIS)

Package

(MIS)

Package &

Communicate

to planner and

stakeholders

(MIS)

Influence

the plan

(Planners

)

Implement

the Plan

(Decision

Makers)

ManagementInformationSystem /

MonitoringAnd

Evaluation


25/87

25

Areas where M&E is usedWe involve the combination of some the following

Informing

Planning and decision makingAssessing progress

Promoting Accountability

Learning

InfluencingPromoting partnership


26/87

26

Informing, Planning and

Decision makingInforming: Translating Findings into usablecommunications

Planning and Decision making: based on M&E Data,making decisions, and modifying / formulating newplans

Or

Providing information about alternative concepts oroptional courses of actions


27/87

27

Assessing progress and

promoting AccountabilityAssessing Progress: establishing whatprogress has been made, toward pre-

determined objectives, leading tomodifications where necessary

Promoting Accountability: explaining

how money will be / was spent,resources utilized, and the justification ofthat investment


28/87

28

Learning and InfluencingLearning: understanding what hashappened in the past, to inform and

improve future approachesInfluencing: Generating evidence andarguments that the program influenced

target groups / reached the goals


29/87

29

Promoting Partnership

Make the adjustment for the investmenton the project, and encourage the

partner (donor) for theExtension/expansion/Replication of theproject.


30/87

30

What did you learn so far?

Questions and Answers!


31/87


32/87

32

Day Three


33/87

33

Different Evaluation ApproachesApproach Major Purpose Typical Focus

QuestionsLikelyMethodology

Goal-based Assessing achievementsof goals and objectives

Were the goalsachieved? Efficiently?Were they the right

goals?

Comparing baselineand progress data

Decision-Making

Providing Information Is the projecteffective? Should itcontinue? How mightit be modified?

Assessing range ofactivities related toproject context,inputs, process andproducts.

Goal-free Assessing the full range ofproject effects, intendedand unintended.

What are all theoutcomes? Whatvalues do they have?

Judging projectworth, usingqualitative andquantitative methodsto come acrosspossible results.


34/87

34

Types of Monitoring and Evaluation Data

Primary Data: This is the data which is collectedprimarily by your project through different ways andapproaches.

Secondary Data: This data is the analysis or use of data

from sources other than your organization. Baseline Data: Is base data for your indicators, which

could be primary or secondary.

Mid-Term Data: The primary data progress upon periodof time before end of the project.

End-line or Impact Data: The final progress of ourbaseline data for indicators, which is called the over allresult on the community you served.


35/87

35

?What is Indicator?


36/87

36

What is an IndicatorA tool to assess if we have achievedwhat we have set out to achieve

A monitoring tools to measure progresstoward objective

Can be used to measure different types

of factors (risk, inputs, process, output,outcome, impact)

Maybe qualitative or quantitative


37/87

37

Why have indicators?Simplify details and complexity

Inform and guide decisions and actions

Can be effective operational andcommunication tools

Help define development as well as

measure it (M&E)


38/87

38

?Who should define them?


39/87

39

Who should define them?Policy makers, Implementers andbeneficiary groups

How?

Through Partnership and participation inidentifying, revising and using indicators


40/87

40

What indicators are required?

Indicators are influential in guidingdecision making and action, so -

Measure what we value most

So you need to define indicators tomeasure different levels

No one indicator measure all aspectsof your project or development


41/87

41

Levels of Indicators

Objectives Types of Indicators

Inputs Inputs: Different resources used to achieve precedingindicators

Activities Process: measures what happens during implementation.Completion or milestones from activity plan

Outputs Outputs: Immediate physical or financial outputs of theproject, number of teachers trained in new syllabus

Purpose Outcomes or effects: more specific to project activities(deal more with direct effect of project outputs onbeneficiaries)

Goal Impacts: Medium or long term developmental change, e.g.Proportion of girls completing secondary education,%increase


42/87

42

So what do we monitor?

All Most Some Few

InputsOutputs

OutcomesImpacts

Monitoring

Process EvaluationEvaluation

Effectiveness Evaluation

Long-term effectsChanges in child

mortalitySocial norms

higher educationincreased

Intermediate effectsBehaviorchangeAttitudechange

Number ofstafftrainedCoverageof qualityservices

ResourcesStaffFundsMaterials


43/87


44/87

44

Indicators are either qualitative or

quantitative contdQualitative indicators, imply qualityassessment

Compliance withQuality of

Extent of

Level of

Note: it is possible to express quantitatively viaa scoring system (e.g. scoring system couldbe developed to track improvements inquality)


45/87

45

Steps in developing indicators

1. Identify what is to be measured

2. Develop trail measures

3. Assess each trail indicator, usingagreed criteria

4. Select and organize the best

indicators for project and user group


46/87

46

Question?What makes a good indicator...?


47/87

47

What makes a good indicator?

Specific

Measurable

Available /Achievable

Relevant

Timely

SMART


48/87

48

SMART

Specific: is the desired outcome clearlyspecified?

Measurable: can the objective be measure?Achievable: is the objective likely to beachieved within the time and budget?

Relevant: is the outcome relevant to theproject goal or main objective?

Timely: is the time specified to achieve theobjective?


49/87

49

Answer the following questions:

1. What is a project?

2. What is M&E?

3. What are the different ways of doing evaluations?

4. What are the different evaluation Approaches?

5. What is an Indicator?

6. What is baseline data?

7. What is Impact?


50/87

50

Thanks

See you in Next Session


51/87

51

Day Four


52/87

52

Monitoring tools

Which tools are used to gathermonitoring and evaluation data?


53/87

53

Focus Group Discussion

A group of target groups are gathered, the topic isintroduced to them and they talk about it.

Focus Group Discussion is good because it is:

InexpensiveRapid Technique

Participants themselves are sources of data

Session typically last 1 to 2 hours

Share certain common characteristics (Sex, Age,Social background, etc)

Data is mostly accurate


54/87

54

Used when?

Analysis + Planning

As part of review

Monitoring regular project lifetime


55/87

55

The facilitator is expected toGuide each session

Allow the discussion to flow freely

Refocus the discussion as necessary

Intervene to bring out salience ifparticipant do not


56/87

56

InterviewsThere are mostly four types of interviews

1. Structured Interviews: use a

questionnaire, fixed number ofquestions, pre-selected range ofanswers

2. Simi-structured Interviews: Do notuse questionnaire, topics andquestions are pre-determined, butmay arise more while interview.


57/87

57

Interviews Contd

3. Unstructured interviews: Topics are pre-determined but questions are notdetermined, these interview are used to

collect general information, data from allaspect of programs or subjects.

4. Telephone Interviews: could bestructured, Simi-structured, or

unstructured, but by telephone, easy andcost-effective and are mostly veryaccurate.


58/87

58

What contribute to a Good or

Bad interview?How do you judge the validity orreliability of information given at an

interview?Facts: Commonly agreed specific truth

Opinions: Persons or Groups views

Rumors: Information from an unknownsource

Be careful of above three things


59/87

59

Some tips on conducting Interviews

Use same language as intervieweeObserve non verbal indicators, bodylanguage, facial expression

Use the 6 helpers (who, what, when, where,why, how)Formulate questions clearlyAvoid why questions too often

Avoid finishing their sentencesNote takingThank interviewees


60/87

60

Types of questions

Closed-ended (record single response)

Open-ended (record single response)

Multiple Choice (MCQs) (single / more)


61/87

61

Community Meetings

This involves a gathering of a large group ofbeneficiaries to whom questions, problems andsituations are put to inputs to help measuring

indicators. Community meetings are useful for getting a

broad response from many people on specificissues.

Community meetings are difficult to facilitate,and requires a very experienced facilitator.


62/87

62

Taking Action

Monitoring and Evaluation have little value if theorganization or project does not act on the informationthat comes out of the analysis of data collected.

Once you have the finding, conclusions andrecommendations from your monitoring and evaluation

process, you need to:

Report to management and field staff Report to your stakeholders Learn from the overall process

Make effective decisions and how to more forward; and, ifnecessary

Modify your objectives, methods and approaches.


63/87

63

Reporting

You need to report your findings, conclusionsand recommendations in different mechanism,to different stakeholders in different times in the

project cycle. Usually report to: Board Members

Management

Staff / Field staff

Beneficiaries Donors

Wider development community


64/87

64

Problem Tree

Problem Identification, Setting objective

Situation

Analysis

Problem

Analysis

Objectives

Analysis

Alternative

Analysis


65/87

65

Situation Analysis

The situation analysis should identify keyproblems/opportunities/Development

constraintsWhat the main problems are?

What are the opportunities to resolve?

What are the constraints to affectdevelopment?


66/87

66

Situation AnalysisContd

Capabilities and Constraints

Social

Technical

Economic

Environmental

Political


67/87

67

Problem Analysis

Explore causes and effects of problems

What and who are the main causes?

Why and when this happened?

How this happened?

Where this started?

What will happen next?

Who will ask us?


68/87

68

Now develop a problem tree

Identify a focal problem (s)

Place the direct effect above

Place the causes below k

Extend as far as possible

Find solution for every analyzed

situation

Convert it to objectives tree


69/87

69

Problem Tree

Focal Problem(Poor Reporting)

Will decreasefunding

Will fail theProject to closure

PoorManagement

No budget

Capacity buildingNeeded

Other staffRequired


70/87

70

Identifying Core problem

What does a core problem means?

Is there solution for core problem?

Is this true? A core problem is the one with nosolution

No, management says, there is no problemwithout solution, but that is the existence of

negative state.

Why is there no WW3? Because ofmanagement and communication.


71/87

71

Group work

Identify a core problem

Develop a problem tree

Convert the Problem tree toobjectives tree


72/87

72

Questions and Answers


73/87

73

Thank You

See you in next Session


74/87

74

Day Five


75/87

75

Introduction of a USAID ProjectPerformance Management/Monitoring

Plan (PMP)


76/87

76

DAC Criteria

Development Assistance CommitteeDAC is a criteria of Evaluation developed in 2002 inSwitzerland, when evaluating programs and projects,its useful to consider DAC criteria.

RelevanceEffectiveness

Efficiency

Impact

Sustainability


77/87

77

Relevance

Does the aid activity fit the policies of thedonor and the recipient (client)

Does it meet the priorities of the target groupAre activities consistent with the overall goaland objectives


78/87

78

Effectiveness

A measure of the extent to which an aidactivity attains its objectives.Use the following questions to evaluate

effectiveness:

To what extent were the objectives achieved /are likely to be achieved?

What were/are the major factors influencingthe achievement or non-achievement of theobjectives?


79/87

79

Efficiency

Efficiency measures the outputs -- qualitativeand quantitative -- in relation to the inputs.

It is an economic term which signifies that isthis uses the least costly resources possible inorder to achieve the desired results. Thisgenerally requires comparing alternative

approaches to achieving the same outputs, tosee whether the most efficient process hasbeen adopted.


80/87

80

EfficiencyContdWhen evaluating the efficiency of a program ora project, it is useful to consider the followingquestions:

Were activities cost-efficient?

Were objectives achieved on time?

Was the program or project implemented in

the most efficient way compared toalternatives?


81/87

81

Impact

Positive and negative effects of theintervention, direct or indirect, intended orunintended, use the following questions toevaluate Impact

What has happened as a result of theprogram or project?

What real difference has the activity made tothe beneficiaries?

How many people have been affected?


82/87

82

Sustainability

Sustainability is concerned with measuringwhether the benefits of an activity are likely tocontinue after donor funding has been

withdrawn. When evaluating sustainability, usethe following questionsTo what extent did the benefits of a programor project continue after donor fundingceased?

What were the major factors which influencedthe achievement or non-achievement of of theprogram or project?


83/87

83

What is a work plan and how to develop a

work plan?A work plan is a description of the necessary activities

set out in stages, with rough indication of the timingIn order to draw a good work plan, the implementers

should:

List all the tasks required to implement a project; Put the tasks in the order in which they will be

implemented; Show allocation of the responsibilities to the actors;

and Give the timing of each activity.Note: you can also allocate an estimated budget for

each of the activities in a work plan


84/87

84

What is a work plan and how to develop awork plan?

The work plan is a guide to project implementation anda basis for project monitoring. It therefore helps to:

Finish the project in time;

Do the right things in the right order;

Identify who will be responsible for what activity


85/87

85

A Common M&E system

A Monitoring and Evaluation system includes: Monitoring and Evaluation staffing and AoRs for every staff

Data collection

Filing system

Management Information System (Computerized)

Design of methodologies

Administering data collection instruments

Data Analysis

Reporting

Feedback to management and field staff Management of the performance monitoring plan (PMP)

Instruments and Methods for Data QualityAssurance/Assessment (DQA)


86/87

86

So, Now

Lets close and let each other know how isyour Monitoring and Evaluation System?

I will let you know, how is my M&E system


87/87

Thanks for the ParticipationSee you in next Training

5 day m&e presentation compiled

Documents