EGOS Colloquium 2012Subtheme 20: Rationalization and Professionalization of the Nonprofit Sector

5 – 7 July - Helsinki, Finland

Co-exist, Colonize, or Combine?Accounting for patterns of discourse on nonprofit evaluation

Carrie Oelberger, Achim Oberg, Karina Kloos, Valeska Korff, Woody Powell

Is talk cheap?

Language is a constitutive feature of social life

• creates cohesion and enforces social boundaries

• reflects and reinforces common systems of norms

• serves to enroll and exclude

Creates communities

Crossing Boundaries

Boundary work as the establishment of local interlanguages – pidgins and creoles – that emerge in the interstices between social domains (Galison 1997).

Interlanguage• facilitates local communication across social and linguistic

boundaries.

• enables coordination of action across place, time and context.

Connects communities

Influences from the domains of • Civil society and associations• Science• Management

Confluence of disparate world views and languages

Nonprofit evaluation discourse is an illuminating case to analyze communication at the interface of social spheres

Nonprofit Evaluation

Research Questions

Who is contributing what to the discourse on nonprofit evaluation?

1. What kind of discourse patterns form when different languages come into contact?

Does a boundary-spanning interlanguage emerge?

2. What organizational features influence an entity’s proclivity to use such interlanguage?

Websites as self-representation

• Websites constitute purposeful presentations of organizations• Information provided on websites has the capacity to travel

across spatial and symbolic distances• Symbolic representation of an organization whose structure

and content reflect the features of a particular entity (Pollach, 2004)

• Comparable in its signaling character to national symbols: flags and anthems (Cerulo, 1993)

Language used on websites reflects an organizations intentional portrayal of itself to diverse audiences

Sample selection by use of Webcrawler

• Focus on websites:Websites as purposeful self-presentations

• Crawler network: Websites are added based on number of incoming references by identified members of the relevant sample. Snowball sample

• Inclusion/exclusion decision: Collective analysis of website content to appraise extent of contribution to non-profit evaluation discourse.

• Coding of entities:Websites are aggregated and core organizational features are coded.

Sample Characteristics

419 highly interconnected entities involved in nonprofit evaluationDemography• Age: 2 to over 200 years old

• Size - Scale: one person blogs to 250,000 employee global organizations;

Scope: local, regional, national and scale

Institutional Properties• Form: 56% nonprofits, 13% for-profits, 3% branches of state or national government, 14%

transnational organizations, and 14% non-organizational forms• Activity: evaluation, funding, consulting, networking, media, advocacy, research, social services

Resources and Constituencies• Revenue Streams: foundation grants, government grants, corporate funding, individual donors,

fee-for-services, membership fees, endowment, public equity market and taxes• Target Audiences: social service beneficiaries, donors, nonprofits, for-profits, (transnational)

government and the public

Collecting Keywords

Keywords are “significant, indicative words in certain forms of thought” that make up a distinctive, domain-specific vocabulary (Williams 1969: 14)

1) Iterative process of identifying keywords to develop a vocabulary of nonprofit evaluation:• Mined the discourse on the websites and consulted experts• Created word clusters (associational, scientific, managerial) Process resulted in 196 terms categorized in 3 clusters (e.g. participation, social

change, justice = associational; data, systematic, framework = scientific; performance, benchmarks, outcomes = managerial)

2) Counting of occurrences of each term on all websites to calculate relative percentage of language used by an entity.

3) Co-occurrence analysis to affirm validity of clusters. Assessment of entities’ individual language use, and the collective pattern of

language use among all entities in the sample

Models of Discourse Patterns

Option 1: Entities co-exist, separated into linguistic factions

Option 2: Entities combine and blend different languages.

Option 3: One language colonizes and dominates the discourse.

Observed Distribution of DiscourseRe

lativ

e la

ngua

ge u

se

thesroinetwork.org

swtgroup.net

cerise-m

icrofinance.org

mullagofoundation.org

efqm.org

corostrandberg.com

ladb.org

sphereproject.org

wdi.umich.edu

usaid.org

fbheron.org

rainforest-alliance.org

organizationalresearch.co m

eandco.org

compasspoint.org

arnova.org

iisd.org

robinhood.org

broadfoundation.org

joycefnd.org

gistfunders.org

gmfus.org

americanprogress.org

hfpg.org

alliance1.org

aecf.org

seechangeevaluation.co m

ncvo-vol.org.uk

nonprofitquarterly.org

unstats.un.org

onphilanthropy.com

gatesfoundation.org

worldofgood.org

usip.org

cofinteract.org

africagrantm

akers.org

unwomen.org

komen.org

sunlightfoundation.com

350.Org

cafonline.org

Entirety of entities involved in nonprofit evaluation

Interlanguage?

Frequency analysis: Terms used by 85% of entities Co-occurrence analysis: Non-particularity of terms, but used in combination with diverse languages

No random blending of discourses

Ubiquitous and multi-lateral interlanguage vocabulary

Interlanguage

Characteristics of the Interlanguage

Interlanguage is…

• Ubiquitous in being used by all contributors

• Composed of elements of original languages

• Distinctive to nonprofit evaluation discourse

AccountabilityCommitmentParticipatorySocial changeTransparency

Trust

AccountabilityQuantitativeAssessmentEvaluationFrameworkWhat worksIndicatorsMethodsSurvey

Lessons learnedM&E/Monitoring

(& evaluation)EffectivenessPerformanceTransparencyBest practiceCertification

OutcomesEvidence

Impact

39%38%

23%

Interlanguage enables communication and coordination across social and linguistic boundaries.

Explaining Interlanguage Use

• H1: Younger organizations use more interlanguage.

• H2: Larger entities, both in terms of a) scale (size) and b) (geographic) scope use more interlanguage.

• H3: Organizations engaged in activities which involve competition or coordination with organizations of a different legal form – specifically evaluation, networking and consulting - use more interlanguage.

• H4: Organizations with a) diverse revenue sources or b) a heterogeneous audiences use more interlanguage.

Interlanguage Use - Findings

• H1: Younger organizations use more interlanguage.

• H2: Larger entities, both in terms of a) scale and b) scope use more interlanguage.

• H3: Organizations engaged in activities which involve competition or coordination with organizations of a different legal form – specifically evaluation, networking and consulting - use more interlanguage.

• H4: Organizations with a) diverse revenue sources or b) a heterogeneous target audience use more interlanguage.

Interlanguage Use - Discussion

• The youngest organizations use less interlanguage, and more managerial.• Size does not affect interlanguage use, suggesting that structural features

do not condition language use.• Organizations engaged in evaluation, networking, consulting, and research

use more interlanguage.• Type of revenue source and audience affects interlanguage use.

– Organizations that speak to an audience of social service beneficiaries and the general public use less interlanguage, while those speaking to for profits use more interlanguage.

– Organizations that draw resources through grants or endowments use more interlanguage.

• Standard features (form, size, revenue sources) remain influential on use of original discourses, yet not on interlanguage use.

Observed Distribution of Interlanguage

Entirety of entities involved in nonprofit evaluation

Rela

tive

lang

uage

use

Conclusion

• Broad use of interlanguage across organizational forms• Interlanguage as a channel through which ideas get

transported across social boundaries

• Evidence of integration in combination and interlanguage use• Evidence of colonization in composition of interlanguage• Different discourses remain relevant: Rational – Normative

Interesting times ahead!

Future Plans

• Relational analysis of the application of keywords – Nonprofit evaluation context as a densely interconnected

reputation network among organizations and URLs– Identification of clusters

Topic clustersSome topics are clustered in specific regions of the reputation network.

Philanthropy

Positioning of clustersOther topics are positioned on the top and in the center.

Civil Society

Positioning of clusters… some connect center and periphery with an inverse distribution of size (marginalized term?)

Social Service

Civil Society

Philanthropy

Social Service

Thank youfor your attention!