Defining Organizational Effectiveness

George MitchellTransnational NGO Initiative, Moynihan

Institute of Global Affairs, Syracuse UniversityApril 6, 2010

Organizational Effectiveness: Overview

• Lack of simple, accessible, relevant and credible information about NGO effectiveness

• Stakeholders, such as donors, forced to rely on incomplete, poor quality information

• How can NGOs help provide more meaningful information?

• Outline– Part I: definition – Part II: disclosure

2

Why Definition Matters• NGOs want to be effective and to communicate

their effectiveness to stakeholders, such as donors.

• Donors want their contributions to support effective NGOs.

• But if stakeholders define organizational effectiveness differently they will employ different criteria and reach different conclusions

• The question of organizational evaluation (and therefore reputation, etc.) depends upon the question of definition

3

DEFINING NGO EFFECTIVENESSEvidence from the Transnational NGO Interview Project

4

The Transnational NGO Initiative

• Face-to-face, in-depth interviews with 152 leaders (mostly presidents and CEOs) of international NGOs rated by Charity Navigator

• Topics included governance, goals, strategies, obstacles, transnationalism, effectiveness, accountability, communications, partnerships and collaborations, leadership, etc.

• About 209 hours of digital recording transcribed and coded (quotations and statistics)

5

TNGO Data on Organizational Effectiveness

• Interviewer: “Let me ask you about the concept of effectiveness, which is something we all have trouble defining. How does your organization define effectiveness?”

• The (non-mutually exclusive) categories for leaders’ open-ended responses:– Resources– Flexibility– Innovation– Expertise– Contacts– Staff/peer competency– Stakeholder satisfaction – Goal achievement– Evaluation– Other (omitted)

6

Organizational Effectiveness: Overall Results

7

Organizational Effectiveness

8

Two Main Definitions of Effectiveness

9

Two Definitions of EffectivenessOutcome accountability• “Well I mean ah, for us to be effective is to

achieve the programmatic or strategic goals that we’ve identified.”

• “We define it as whether or not we are sort of getting the tasks achieved that we set for ourselves.”

• “We set important goals and, and we achieve them.”

• “if we’ve done the work that we’ve said we would do, that’s...that should be one level of effectiveness.”

• “It’s the commitment you make to doing what you said you were going to do.”

• “it’s when you are doing what you’re saying you’re doing, that you’re serving your mission...and that you’re able to show that you’re serving your mission…”

Overhead Minimization• “to be effective in [DELETED] is to deliver

services and assistance to the people of [DELETED] at low cost”

• “The uh, amount of money that’s actually getting to the, to the field dedicated to the programs…and secondarily the actual impact of projects that you can quantify.”

• “…we can’t determine the outcomes, so we measure products. We, we measure outputs. What is sent over you know, what its purpose is, where it goes. We don’t necessarily know how many people it will effect…you can measure success primarily by your outputs.”

• “our best ace card is our efficiency in terms of how we, we have a low overhead”

Goal attainment, promise-keeping, program evaluation

Functional expense ratios, accounting for outputs 10

Effectiveness as Overhead Minimization

• Why?– Reaction to external pressures (internalization,

marketing)? – Underlying cultural belief that low overhead

corresponds with organizational worthiness? Evidence? (Urban/Indiana study)

• Who?– These leaders are three times more likely to be from

larger NGOs (professionals, economies of scale).– These leaders are ten times more likely to be from

“amelioration” NGOs (accounting for outputs).

11

ASSESSING ORGANIZATIONAL EFFECTIVENESS

Two understandings, two possible systems

12

Communication Breakdown?

Your organization is ineffective! Your

overhead is too high!

But we accomplished everything we

promised…

13

The Disclosure Problem

• Stakeholders make judgments based on the credible information available to them – In lieu of meaningful “effectiveness data,” ratings

based on financial data and checklists have become increasingly popular.

– Many users implicitly assume such ratings have something to do with effectiveness.

• Many donors are forced to rely upon very little information… (anecdote)

14

Two Definitions, Two Possible Disclosure Systems

Outcome accountability• Definition: demonstrable

goal attainment or promise fulfillment

• Disclosure: states ex ante, goals or promises, states ex post

Overhead minimization• Definition: maintenance of

normal functional expense ratios, etc.

• Disclosure: financial information, much of it relying on discretionary classification

15

How can NGOs disclose more helpful information about organizational effectiveness to stakeholders?

The Wrong Format: Statement of Program Service Accomplishments

16

A Disclosure Format for Outcome Accountability

Indicator State Ex Ante

Goal or promise

State Ex Post

Percent Achieved

Percent of Budget

Learning

Water access (avg. time)

60 min. 30 min. 45 min. 50% 100% Local water contaminated

∑ab=50%

17

Evaluation Challenges

• Although most NGO leaders associate organizational effectiveness with goal achievement and evaluation, according to Charity Navigator less than 10 percent of NGOs implement meaningful evaluation systems…

• At the end of the day, are NGOs held accountable for outcome achievement?

• The basic principles of outcome accountability– Articulate specific goals and identify appropriate indicators – Evaluate progress as rigorously as feasible – Credibly and systematically disclose results

18

Conclusions

• The TNGO Initiative website: http://www.maxwell.syr.edu/moynihan_tngo.aspx

• Subscribe to the blog.• Email: gemitche@maxwell.syr.edu

19