evaluating global and regional partnership programs (2011 evaluation week)

Evaluating Global and Regional Partnership Programs

IEG Week 2011

© Universalia

GRPP Evaluated

• Cities Alliance• Association for the Development of Education in

Africa (ADEA)• Global Facility for Disaster Reduction and Recovery• Water and Sanitation Program• Global Facility for Disaster Reduction and Recovery

(GFDRR)• The International Land Coalition• Global Road Safety Facility (GRSF)• Global Donor Platform for Rural Development• infoDev

October 2011 2

ISSUES WITH TERMS OF REFERENCE

3October 2011

Terms of Reference

• Scope and focus comparable: relevance, efficacy, efficiency, sustainability, Management.

• Limited links between evaluation questions and program maturity

• Strong focus of relevance of objectives – less on relevance of design

4October 2011

Terms of Reference

• Efficacy– Clear questions and appropriate

resources allocated to evaluate inputs, progress of activities and outputs

– Questions on outcomes tend to downplay outcomes multiplier, downstream effects , unintended effects, boundaries of effects

– In general unsuficcient resources allocated to evaluating outcomes

5October 2011

Terms of Reference

• GRPP include efficiency questions – they are difficult to answer– Changing scale and scope of GRPP– Absence of benchmark

• Cost-effectiveness questions more difficult to answer often due to an absence of alternatives

6October 2011

ISSUES IN THE CONDUCT OF THE EVALUATION

GRPP Members

• Diversity and multiplicity– More consultation, more participation,

increased time requirements – Diverse expectations about the deliverables

• Power shifts– Participation of large Foundations creates a

new dynamics amongst the membership– Need to ensure that actual inbalance is not

reflected in the evaluation• Effects of turnover

8October 2011

GRPP Members

• Reporting evaluation results to members who are– Host– Member– Donor– Partner– Beneficiary

• Potential conflict of roles, or goal displacement?

9October 2011

GOVERNANC E ISSUES: FRAMEWORKS UNDERLYING GRPP EVALUATIONS

10October 2011

Structural Complexity of the GRPPs

• The GRPPs represent various forms of networks:– Association– Facility– Coalition– Platform– Program– Partnership

• Yet, evaluation of GRPPs draws significantly on organizational analysis literature

11October 2011

What Do We Know About Network Evaluation?

• The literature on evaluation of networks and inter-organizational networking indicates that research has centered on the driving forces behind networks rather than on effectiveness of networks.

• Effective GRPP governance should perhaps be examined through variables and indicators from the network literature or the team literature

12October 2011

• Assesses team effectiveness through several predictors of team performance – Cohesiveness– Size– Leadership– Motivation– Goal group– Composition

Team Literature

13October 2011

• Assesses the effects of networks using several variables:– Social influence– Power– Social exchange– Knowledge Management– Social Capital

Social Network Analysis

14October 2011

Conclusions

• Additional tools are required for practitioners

• Evaluations of GRPP requires sector expertise and experts familiar with organization, teams and network dynamics

23-04-07 © Universalia 15