masterclass performance measurement framework
Post on 16-Apr-2017
304 Views
Preview:
TRANSCRIPT
Performance Measurement Framework
Shared framework of indicators
ImpactAreas GuidingQuestion Indicator
LivelihoodandWellBeingArethebasicneedsofthe
farmersbeingmet?
FoodSecurity:Accesstosufficientfood
Income
AssetsPerceivedWell-Being
GenderWhatarewomen’srolesand
benefitsinthiscrop?
Women’sparticipationincrop
EquitableAccesstoTraining
ParticipationinDecision-Making
EnvironmentalPerformance
Isthelandwellstewarded? Adoptionofconservationpractices
FarmProductivity
Arefarmersrealizingthepotentialoftheirfarm?
Adoptionofgoodagpractices
EstimatedProductivity
CropRevenueorNetIncome
AccesstoServices
Dofarmershaveaccesstoservices?
Accesstocredit,trainingandinputs
Arefarmersusingtheseservices?
Useofcredit,training,andinputs
TradingRelationshipsArefarmersexperiencinggoodtradingrelationships?
TBD
What is a shared approach?
• A common framework of indicators and metrics to help guide the practitioner.
• A suite of indicators to pick what’s appropriate. Not a single set of indicators.
• Shared approach implies committing to use the same indicators and metrics when asking the same questions.
Why a shared approach?
From shared questions about farmers to shared approach for:
• Greater efficiency and effectiveness
• Reduced burden on suppliers and farmers
• More effective community learning
Criteria leading to framework
• Fewest questions (indicators) that give “sufficient” insight into livelihoods and performance.
• Affordability and scalability vs scientific robustness
• Simplicity vs nuance
• Embedding approaches for regular monitoring and reporting
Characteristics of Performance Measurement
Indicators and metrics appropriate for:
• Surveys under 30 minutes
• Minimally trained enumerators or even self reporting
• Across a wide range of supply chain types
Yields actionable data
Cost efficient enough to scale
Grew from overlap in a theory of change many share
Out of the ToC grew common learning questions
the Learning Questions dictated the Shared Framework of indicators
ImpactAreas GuidingQuestion Indicator
LivelihoodandWellBeingArethebasicneedsofthe
farmersbeingmet?
FoodSecurity:Accesstosufficientfood
Income
AssetsPerceivedWell-Being
GenderWhatarewomen’srolesand
benefitsinthiscrop?
Women’sparticipationincrop
EquitableAccesstoTraining
ParticipationinDecision-Making
EnvironmentalPerformance
Isthelandwellstewarded? Adoptionofconservationpractices
FarmProductivity
Arefarmersrealizingthepotentialoftheirfarm?
Adoptionofgoodagpractices
EstimatedProductivity
CropRevenueorNetIncome
AccesstoServices
Dofarmershaveaccesstoservices?
Accesstocredit,trainingandinputs
Arefarmersusingtheseservices?
Useofcredit,training,andinputs
TradingRelationshipsArefarmersexperiencinggoodtradingrelationships?
TBD
Scope of the Shared Approach Framework
Core reporting metrics
(a subset for reporting &
communication)
Shared PM Framework (detailed enough to be
“actionable”) Full Measurement Study
(will usually address additional questions)
Made-to-fit
Each organization will have it’s own purpose and goals for
measurement, but when we share learning questions in common, we can align on
indicators and increase the value of our data by gathering it
in a common way.
M&E in the Programme Cycle
Financing and
contracting
Final Evaluation
Implementation
Scoping
Project Design M&E
Strategy/Framework
Operational Planning M&E
Plan/Matrix
Partici
pation
Monitoring and
Evaluation
Impact Assessment
Year 1 Year 2 Year 3 Year 4 Year 5
M&E systems
Deep dive
Deep dive
Year 0
Baseline Impact Assessment
Interventions
Goal: Evaluate impact of specific changes (interventions) so that outcomes can be attributed to the specific interventions. Valid control groups (counterfactuals) are needed.
Performance Monitoring
Year 1 Year 2 Year 3 Year 4 Year 5
Performance Monitoring
Chain wide indexes
Goals: Assess “status” and track change over time. Outcome change cannot be attributed to specific causes but trends can be tracked and correlations observed.
Mixing methods
Year 1 Year 2 Year 3 Year 4 Year 5
(2) Performance measurement to track broad annual change on few key indicators (short survey by technician, 10 min)
Baseline
(1) Baseline to key issues, get stakeholder buy in, and tailor survey to specific context. Research + focus groups
Interventions
(3) Focused impact Assessment on small sample to answer specific causation question. Trained interviewers plus control group
Example of thinking process
Learning questions: Are the farmers consistently food secure in our supply chain? Is food security improving?
Indicators: Food security
Specific Metric: % of farmers with 2 months or more of food insecurity
Approach: Youth in coop run short interviews during annual meeting using Ipod-based system.
Survey question: Are there times in the year where you have to reduce meal size, skip meals, or change diet? Which times in the year? (number of months)
What you want to know
What you ask a farmer
Appropriate methodology depends on purpose (and budget!)
Purpose: Why do you want to know? What will you do with the results? Who is the audience?
Learning questions & population: What questions are you trying to answer? About whom?
Indicators, Metrics, and Questions: What are the appropriate indicators to track? What specific survey questions are appropriate and effective?
Approach: How and to whom are you going to ask the questions?
Embedding: How could this be part of an ongoing system?
Purpose
Learning Questions
ToC
Approach Indicators/
survey
Embedding
Of course -- depends on purpose, specific questions, and resources
Key Methodology Questions
How do we collect the data?
From whom do we collect data?
Who collects the data?
Self reporting Group setting Indirect Household
Group gathering point
Rough sampling
Voluntary individual
Statistical sampling w/control
Coop TA Youth Self Certifier Researcher
Of course -- depends on purpose, specific questions, and resources
Key methodology Questions
How do we collect the data?
Who collects the data?
Self reporting Group setting Indirect Household
Group gathering point
Rough sampling
Voluntary individual
Statistical sampling w/control
Coop TA Youth Self Certifier Researcher
household survey for rough baseline
From whom do we collect data?
Of course- depends on purpose, specific questions, and resources
Key Methodology Questions
How do we collect the data?
Who collects the data?
Self reporting Group setting Indirect Household
Group gathering point
Rough sampling
Voluntary individual
Statistical sampling w/control
Coop TA Youth Self Certifier Researcher
From whom do we collect data?
Of course- depends on purpose, specific questions, and resources
Key Methodology Questions
How do we collect the data?
Who collects the data?
Self reporting Group setting Indirect Household
Group gathering point
Rough sampling
Voluntary individual
Statistical sampling w/control
Coop TA Youth Self Certifier Researcher
From whom do we collect data?
top related