co-exist, colonize, or combine?
DESCRIPTION
Professor Woody Powell, Stanford University Center on Philanthropy and Civil Society (with Carrie Oelberger, Achim Oberg, Karina Kloos, Valeska Korff). Academy of Management Presentation, Boston MA. August 7, 2012.TRANSCRIPT
Academy of ManagementCultural (Ac)counting: The rise of formal organization in social and cultural domains
7 August 2012, Boston, MA
Co-exist, Colonize, or Combine?Accounting for patterns of discourse on nonprofit evaluation
Carrie Oelberger, Achim Oberg, Karina Kloos, Valeska Korff, Woody Powell
Co-exist, Colonize, or Combine?
• Our study analyzes the current discussion of nonprofit evaluation with respect to contact between three different social worlds:– Civil Society– Science– Management
• Each of the three worlds have played varying roles throughout the history and development of the current nonprofit sector
Co-exist, Colonize, or Combine?
• The confluence of financially-driven managerial criteria, combined with the progressive era’s lasting focus on measurable impact, has led to a growing instrumental orientation for the nonprofit sector.
• At the same time, there is concern that what we measure will influence the shape of civil society.
• Our study analyzes these conversations and asks whether scientific and managerial language is co-existing, colonizing or combining with the more traditional associational language of civil society.
Three Distinct Social Domains
Trust Compassion
Transparency
Social Change
Commitment
Participatory
Three Distinct Social Domains
Justice
Civil Society
Trust Compassion
Transparency
Social Change
Commitment
Participatory
Methods Assessment
QuantitativeFramework
IndicatorsEvaluation
Survey
Three Distinct Social Domains
Justice
Civil Society
Data
Science
Trust Compassion
Transparency
Social Change
Commitment
Participatory
Methods Assessment
QuantitativeFramework
IndicatorsEvaluation
Survey
Impact
Lessons Learned
Effectiveness
Outcomes
Performance
M&E
Best practice
Three Distinct Social Domains
Justice
Civil Society
Data
Science
Efficiency
Management
Research Questions
Who is contributing what to nonprofit evaluation discourse?
1. Who is participating in online discourse regarding nonprofit evaluation?
2. What kind of discourse patterns form when different languages come into contact: co-existence, combination, or colonization?
3. What organizational features influence an entity’s discourse patterns?
Research Questions
Who is contributing what to nonprofit evaluation discourse?
1. Who is participating in online discourse regarding nonprofit evaluation?
2. What kind of discourse patterns form when different languages come into contact: co-existence, combination, or colonization?
3. What organizational features influence an entity’s discourse patterns?
Website Discourse
• Website discourse of entities talking about nonprofit evaluation– Open to the public, the information presented is not
tailored to one particular audience– Purposeful self-representations
Webcrawler Methodology
• Snowball sampling approach: Websites are added based on number of incoming references by identified members of the relevant sample.
• Inclusion/exclusion decision: Collective analysis of website content to appraise extent of contribution to non-profit evaluation discourse
• Website “Scraping”: The entire text from each website is “scraped” into our off-line database to enable analysis of discourse patterns
Research Questions
Who is contributing what to nonprofit evaluation discourse?
1. Who is participating in online discourse regarding nonprofit evaluation?
2. What kind of discourse patterns form when different languages come into contact: co-existence, combination, or colonization?
3. What organizational features influence an entity’s discourse patterns?
Sample Characteristics
• Our methodology produced 419 highly interconnected entities involved in nonprofit evaluation
• Entities were then coded for core organizational features including:– Demography– Institutional Properties– Resources and Constituencies
Sample Characteristics
Demography• Age: 2 - over 200 years old• Size -> Scale: one person blogs - 250,000 employee global organizations• Size -> Scope: local, regional, national and international
Institutional Properties• Form: 56% nonprofits, 13% for-profits, 3% branches of state or national government, 14%
transnational organizations, and 14% non-organizational forms• Activity: evaluation, funding, consulting, networking, media, advocacy, research, social
services
Resources and Constituencies• Revenue Streams: foundation grants, government grants, corporate funding, individual
donors, fee-for-services, membership fees, endowment, public equity market and taxes• Target Audiences: social service beneficiaries, donors, nonprofits, for-profits,
(transnational) government and the public
Research Questions
Who is contributing what to nonprofit evaluation discourse?
1. Who is participating in online discourse regarding nonprofit evaluation?
2. What kind of discourse patterns form when different languages come into contact: co-existence, combination, or colonization?
3. What organizational features influence an entity’s discourse patterns?
Co-exist
• Do entities co-exist in this field of nonprofit evaluation, but retain distinctly separate discourse – associational, scientific, or managerial – about how to approach nonprofit evaluation? – Similar to the “salad bowl” metaphor of immigration,
where individuals remain monolingual with their traditional language.
Combine
• Do we observe a combination of vocabularies and the emergence of some sort of shared language around nonprofit evaluation? – Similar to the “melting pot” story of immigration, where
we would find entities drawing equally on a combination of all of three languages, dissolving the boundaries that previously existed.
Colonize
• Has science or management colonized the nonprofit evaluation debate and crowded out the less powerful domain of civil society and its related associational discourse? – Similar to the classic story whereby immigrants “colonize”
indigenous languages.
Analyzing Discourse: Developing Vocabulary
Keywords are “significant, indicative words in certain forms of thought” that make up a distinctive, domain-specific vocabulary (Williams 1969: 14)
Iterative process of identifying keywords to develop a vocabulary of nonprofit evaluation:• Mined the discourse on the websites and consulted experts• Created word clusters that resulted in three different social domains• Fleshed out clusters based on extant knowledge of domain• Co-occurrence analysis to affirm validity of clusters
Process resulted in 196 terms categorized in 3 clusters
196 terms categorized into the following 3 clusters:
Analyzing Discourse: Clustering Terms
Social Domain Discourse
Civil Society Associational
Scientific Research Scientific
Management: Business & Government
Managerial
Analyzing Discourse: Counting Keywords
Examined each website to calculate the relative percentage of each of the three languages: associational, scientific, and managerial
• For example, if there were:– 50 occurrences of managerial terms– 30 occurrences of scientific terms– 20 occurrences of associational terms
• The entity would be 50% managerial, 30% scientific, and 20% associational
Observed Distribution of DiscourseRe
lativ
e la
ngua
ge u
se
thesroinetwork.org
swtgroup.net
cerise-m
icrofinance.org
mullagofoundation.org
efqm.org
corostrandberg.com
ladb.org
sphereproject.org
wdi.umich.edu
usaid.org
fbheron.org
rainforest-alliance.org
organizationalresearch.co m
eandco.org
compasspoint.org
arnova.org
iisd.org
robinhood.org
broadfoundation.org
joycefnd.org
gistfunders.org
gmfus.org
americanprogress.org
hfpg.org
alliance1.org
aecf.org
seechangeevaluation.co m
ncvo-vol.org.uk
nonprofitquarterly.org
unstats.un.org
onphilanthropy.com
gatesfoundation.org
worldofgood.org
usip.org
cofinteract.org
africagrantm
akers.org
unwomen.org
komen.org
sunlightfoundation.com
350.Org
cafonline.org
All entities involved in nonprofit evaluation
Does a coherent, shared language exist?Re
lativ
e la
ngua
ge u
se
thesroinetwork.org
swtgroup.net
cerise-m
icrofinance.org
mullagofoundation.org
efqm.org
corostrandberg.com
ladb.org
sphereproject.org
wdi.umich.edu
usaid.org
fbheron.org
rainforest-alliance.org
organizationalresearch.co m
eandco.org
compasspoint.org
arnova.org
iisd.org
robinhood.org
broadfoundation.org
joycefnd.org
gistfunders.org
gmfus.org
americanprogress.org
hfpg.org
alliance1.org
aecf.org
seechangeevaluation.co m
ncvo-vol.org.uk
nonprofitquarterly.org
unstats.un.org
onphilanthropy.com
gatesfoundation.org
worldofgood.org
usip.org
cofinteract.org
africagrantm
akers.org
unwomen.org
komen.org
sunlightfoundation.com
350.Org
cafonline.org
All entities involved in nonprofit evaluation
Does a coherent, shared language exist?
Interlanguage (Galison 1997)• pidgins and creoles that emerge in the interstices between
social domains
• facilitates local communication across social and linguistic boundaries
• enables coordination of action across place, time and context
Does a coherent, shared language exist?
To investigate whether a coherent interlanguage exists, we examined whether there are keywords that both occur:• frequently across the majority of entity websites
• in combination with keywords from the other two “parent” languages
This identified 24 (out of 196) terms that represent an interlanguage on nonprofit evaluation.
Operationalizing Interlanguage
• Nonprofit evaluation interlanguage spans the boundaries between the domains of civil society, scientific research, and management
• Approximate 2:1 ratio implies that there is slight colonization by managerial and scientific languages
TrustSocial changeCommitmentParticipatoryTransparency
Methods AssessmentAccountabilityQuantitativeEvaluationFrameworkWhat worksIndicatorsSurvey
ImpactOutcomes
Lessons learnedEffectivenessPerformanceTransparencyBest practiceCertification
Evidence
39%38%
23%
Including our Interlangauge in the analysis
• These 24 keywords become our interlanguage
• If we remove them from their “parent language” and create a fourth language cluster an interesting picture emerges Trust
Social changeCommitmentParticipatoryTransparency
Methods AssessmentAccountabilityQuantitativeEvaluationFrameworkWhat worksIndicatorsSurvey
ImpactOutcomes
Lessons learnedEffectivenessPerformanceTransparencyBest practiceCertification
Evidence
Observed Distribution of Interlanguage
Entirety of entities involved in nonprofit evaluation
Rela
tive
lang
uage
use
Conclusion
• A diverse array of entities discuss nonprofit evaluation
• Entities are multi-lingual, combining three distinct “parent” languages
– Furthermore, there is extensive use of a coherent interlanguage
– Managerial and scientific terms outnumber associational in interlanguage by a factor of 2:1
– Associational terms remain highly relevant, if less standardized, across all entities
Gerhard RichterBach (4)
1992300 cm x 300 cm
Oil on canvasCatalogue Raisonné: 788
Thank You!