Community Capacity Inventory1

• Gifts of the head: (things I know something about and would enjoy talking about or teaching other about, e.g., birds, local history, music).

• Gifts of the hands (things I know how to do and enjoy doing, e.g., carpentry, sports, planting, cooking, – be specific).

• Gifts of the heart (things I care deeply about, e.g., children, older people, community history, environment).

Adapted from World Learning, 1989

CORE Group Spring Meeting May, 2012

Strengthening and Measuring Community

Capacity for Sustained Health Impact

Gail Snetro-Plewman, Save the Childrengsnetro@savechildren.org

Background

Contributors:David Marsh, MD, Senior Child Survival Advisor, Save the ChildrenCarol Underwood, PhD, Johns Hopkins University, Center for Communication Programs (CCP)Mark Boulay, PhD, Johns Hopkins University, CCP

Definitions of Community Capacity

• “the characteristics of communities that affect their ability to identify, mobilize, and address social and public health problems (Goodman, et.al, 1998; Poole, 1997)

• “the set of assets or strengths that residents individually and collectively bring to the cause of improving the quality of life: (Easterling, Gallagher, Drisko, & Johnson, 1998.)

• “the ability of people and communities to do the work needed in order to address the determinants of health for those people in that place” (Bopp, GermAnn, et.al, 2000)

Why measure community capacity?

• We predict that increased capacity helps improve health & social change outcomes.

• We predict that this capacity will help sustain positive health & social change outcomes in the future.

• We know that SC projects increase community capacity (CC) through its community mobilization (and other) approaches - but how much?

Why measure community capacity?

• SC devotes a lot of effort to strengthening CC.• SC thinks we are good at strengthening CC.• But we cannot prove it because we lack

systematic way to measure CC.• Yet, it seems reasonable that strengthened CC:

– Helps increase the use of interventions.– Sustains this use.– Promotes “development” more broadly.

State of the Art - Measuring Community Capacity

• Limited research undertaken globally to date testing impact of capacity building on social change outcomes – but field is growing.*

• No agreed upon theoretical framework from social scientists.

• Few indicators from literature have ever been tested for validity and reliability.

• Myriad, unsystematic, project-specific indicators from programmers.

• ‘Who’ should develop & measure? institutional vs. community?

*Sources: “What is the evidence on effectiveness of empowerment to improve health.”, Feb. 2006, WHO/HENS; Johns Hopkins, Communication for Social Change, World Bank

Community-capacity interventionsas a means and as an end

COMMUNITYCONTEXT

- Material resources

- Social resources

- Disease burden

- Experience- etc.

COMMUNITY CAPACITY

SOCIAL CHANGEParticipationSocial capitalSocial cohesionEmpowerment Collective Efficacy

HEALTH COMPETENCEConsistent & appropriate health behavior

HealthStatus

Community capacity designed to improve health behavior and health status via social change

GOAL: Status Improved

SO: Use of Key Practices and Services Improved

IR 1Access and

Availability of Key Services and

SuppliesIncreased

IR 2Quality of Key

Services Increased

IR 3Knowledge and

Acceptance of Key Practices and

ServicesIncreased

IR 4 Social and

Policy Environment

Enabled

SC/US Generic Results Framework

GOAL: Status Improved

SO: Use of Key Practices and Services Improved

IR 1Access and

Availability of Key Services and

SuppliesIncreased

IR 2Quality of Key

Services Increased

IR 3Knowledge and

Acceptance of Key Practices and

ServicesIncreased

IR 4 Social and

Policy Environment

Enabled

SC/US Generic Results Framework

CM Terminology

Community Mobilization

Sustained Behavior ChangeSustained Social Change, Including Enhanced Community Capacity

Sustained Societal Change

Community Action Cycle

• Implemented by the community• Fosters equity through participation

of those most interested and affected by health issue

• Instills ownership and works towards sustainability

• Iterative – not linear• Builds community capacity for

managing health and development

Measuring Community Capacity –Efforts to Date

• Measuring Community Capacity - Partners Learning

Forum: Lusaka, Aug ‘06 - HCP– learning, change, planning

• Literature Review of measuring community capacity

research efforts to date

• Meta-Analysis of CC domains and indicators to validate

• Hypothesis-testing research testing the effect of CC on

achieving health outcomes in Uganda, Zambia,

Vietnam, Nicaragua

• Quantitative and Qualitative measurement tools

Literature Review: 30 C x 210 R matrixDomains of Community Capacity by Researcher*

ResearcherL W H G F R C B E S1 S2 S3 S4 S5 S6 S7 S8 S9 S10

1 Participation small groups, larger organizations X2 Participation strong participant base X X3 Participation diverse network to enable different interests to act X X4 Participation benefits over-ride costs of participation X X5 Participation citizen involvement in defining and resolving needs X X X6 Participation Citizen participation and control7 Participation Environement X8 Participation Attracting and keeping volunteers X9 Participation broad, representative range of community X

10 Participation engage diverse members of community in process of consultation, collective analysis & decision-making X

11 Participation, degree & equity of access to participation X12 Participation, degree & equity of extent and level of participation X X13 Culture of Openness & Learning openess between indiv; comfort to listen & speak X14 Sense of ownership importance of issue or program to participants X15 Sense of ownership sense of responsibility for program X16 Sense of ownership contribution to the program X17 Sense of ownership benefit from program X18 Sense of ownership sense of ownership of "credit/blame" for outcome X X19 Sense of ownership personal identification with program X20 Leadership organizes groups X21 Organizational level context Commitment to community Organizational culture and climate X22 Organizational level context Organizational structure, procedures and authority X23 Organizational level context Organizational effectiveness and/or sustainability X

*L=Laverack; W=World Bank; H=Hastings (proxy of Bopp); G=Goodman; F=Figueroa; R=Rifkin; C=Chaskin; B=Bopp; E=Easterling; S1= Study 1; S2=Study 2; S3= Study 3; S4=Study 4; S5=Study 5; S6=Study6; S7=Study 7; S8=Study8; S9=Study 9; S10=Study10; S11=Study11; T12= Tool12; T13=Tool 13; T14=Tool 14; T15= Tool 15; S16= Study 16; T17= Tool 17; T18= Tool 18

Sub-Domain# Domain

Long story short

CC Outcomes:10 domains with 54 sub-

domains

Long story short

CC Outcomes:10 domains with 54 sub-

domains

1. Community history2. Networks3. Participation4. Leadership5. Social cohesion6. Ownership7. Collective efficacy8. Resource

mobilization9. Information equity10.Critical thinking

Long story short

CC Outcomes:10 domains with 54 sub-

domains1. Community history2. Networks3. Participation4. Leadership5. Social cohesion6. Ownership7. Collective efficacy8. Resource mobilization9. Information equity10.Critical thinking

1. Extent2. Equity and diversity3. Flexibility4. Skills to maintain dialogue5. Vision and innovation6. Trustworthiness7. Exercise of power

Suggested SC Community Capacity Indicators – “Quantitative”

Explore &

Prioritize

Plan

ActEvaluate

Organize

Prepare to

Mobilize

Prepare for Scale

# (%) groups effectively* formed around health/development issue. *60/40 rule to include marginalized populations, including women; clear roles/responsibilities of members; rotating leadership…# (%) community

members exploring the health/development issue and setting priorities

# (%) written community action plans in place

# (%) communities completing 70% of action plan on time

# (%) communities using data for decision through community bulletin boards, or health records

Who should measure what? Community or externally measured indicators?

“Top-down”

vs.

“Bottom-up”

Measuring Community Capacity in Zambia - Context• Strengthening Community Capacity and Engendering Behaviour Change -

Health Communication Partnership (HCP) -Zambia

• Project Goal: To use strategic communication approaches at scale to support households and communities to take positive health action.

• 5 Year – 2004-2009, $31 million Cooperative Agreement with Johns Hopkins University – Save the Children Lead in Zambia - In’tl HIV/AIDS Alliance – sub.

• HCP focused on hard to reach districts; low health progress; inactive Neighbourhood Health Committees, weak community capacity

Community systems strengthened to focus on health priorities they identified through community-level dialogue, and application of Community Action Cycle as a mobilizing tool for collective action

Interventions integrated across health areas (Malaria, RH, Child health, HIV and AIDS, Maternal Health)

CM at Scale - Context Continued…

Total Population Covered: 2,848,520• 22 Districts out of 71 country-wide

(presence in all 9 provinces)• 22 District Level Health Center Partners• 1800 Community Core Groups’ -

Neighborhood Health Committees (NHCs)• Application of the Community Action Cycle• 1341 with Community Action Plans• 1063 communities completed at least one

activity from their action plan• 65 Safe Motherhood Action Groups formed as part of NHC’s.

HCP Program Framework (2005 – 2009)

• Mobilising and empowering individuals/families communities

• Engaging leaders• Empowering youth• Harmonising health

messages

• Increased community capacitySocial cohesionCollective efficacyConflict managementLeadershipEffective leadershipIndividual efficacy

• Increased exposure to health messages

DramaTV/videoPrint materials

• Increased knowledge• Reduced high risk

behaviours• Increased individual

/collective efficacy for health action

• More equitable gender and social norms

Illustrative:• FP use• Birth planning• Delivery at a health

facility• Exclusive breast-

feeding • ITN use• HIV protective

behaviours

HCP Enabling Intermediate

Health

Interventions

Environment

Effects Behaviour

Endline Evaluation assessed:

• The effect of the project activities to foster improvements in the community capacities required for social dialogue and action

• Increases in individuals’ knowledge, attitudes, and behaviors

• Associations between program exposure & knowledge, attitudes, and behaviors

• The reach of program messages through community-based and mass-media channels

Survey Design Con’tPhase I - Measuring Community Capacity Study• Qualitative approach for community-generated

domains and sub-domains• Most Significant Change technique*Phase II – Measuring Community Capacity Study• Survey to test and validate indicators• Principal Components Analysis (PCA) &

Cronback’s alpha (a) tests used to develop and assess the scales to measure community capacity domains (validate indicators)

Phase III – Quantitative Population Based Endline Evaluation

Significant Change Technique, Dart and Davies, 2003

Description of Community Capacity Domains and Indicators for Endline:

(1) Social Cohesion - Description of Domain:Seeks to measure the extent to which target

communities were able to work together towards a perceived common good.

Indicators:• Repay debts to others• Did not help each other in times of need

(reversed)• Did not trust one another (reversed)• Strong relationships• Able to discuss problems

Domains and Indicators…

(2) Collective Efficacy- Description of Domain

Seeks to measure the extent to which target groups shared belief in its conjoint capabilities to attain their goals and accomplish desired task. It involves the “belief or perception that an effective collective action is possible to address a social or public health problem”.

Indicators:• Work hard to accomplish a project • Confidence in community problem solving • Committed to the same collective goals • Solutions to problems

Domains and Indicators…

(3) Conflict Management – Description of Domain Seeks to measure the extent to which target

communities were able to handle conflicts in a way that was fair and allowed for continued participation of its members towards positive health action

Indicators:• Quick resolution to conflict• Trouble dealing with conflict (reversed)• Feuding for a long time (reversed)• Getting involved to resolve issue

Domains and Indicators…

(4) Leadership – Description of DomainSeeks to measure the extent to which target

communities had leaders with the capacity to engage the diversity of sectors and levels within community life in processes of learning and action for health.

Indicators:• Women leaders• Leaders treat people equally• Leaders listen• Leaders lead by example• Leaders are good at resolving

disagreements

Domains and Indicators….

(5) Effective Leadership – Description of Domain

Seeks to measure the extent to which the community has the capacity to engage the diversity of sectors and levels within community life in processes of learning and action for health.

Indicators:• Participation in meetings• Setting goals & objectives • Developing a plan • Assigning tasks fairly• Obtain money from outside

Domains and Indicators…

(6) Participation – Definition of DomainSeeks to measure the extent to which

target communities can engage its own diverse membership in constructive processes of consultation, collective analysis and decision making.

Indicators:• Skills and knowledge• Confidence to solve it• I can participate

% reporting that community worked together in past yearto solve a health problem - by number of capacities

Percentage of NHCs reporting having 50% or more female members by intevention and comparison districts (N=89)

73

54

0

10

20

30

40

50

60

70

80

Intervention District Comparison District

Pe

rce

nta

ge

Percentage of females in NHC leadership positions by intervention and comparison

districts

43

11

42

18

4

17

0

10

20

30

40

50

Chairperson V.Chaireperson Secretary

Key Positions

Per

cen

tag

e

Intervention District

Comparison Districts

Table 4: Adjusted odds ratios from logistic regression models predicting selected health behaviorsAdjusted Odds Ratios

Current use of a contraceptive

method1

Received HIV test in past year and know

results1

Youngest child under 5 years slept under a bed

net the past night2

Community worked to address health problem in past yearNoYes

1.002.14***

1.001.76***

1.001.54***

Capacity ScoreFirst quintile (lowest)Second quintileThird quintileFourth quintileFifth quintile (highest)

1.001.170.951.31*1.09

1.001.171.061.171.26

1.002.49***2.58***2.35***2.21***

Community TypeComparisonNon-intensive Intensive activity

1.000.971.05

1.000.870.92

1.000.931.07

GenderFemaleMale

1.001.08

1.000.46***

Age15-2425-3435 and over

1.001.39***1.38***

1.001.20*0.99

1.001.71***2.05***

EducationPrimary or lessSecondary or more

1.001.16

1.001.31***

1.000.96

Use media weeklyNoYes

1.001.42***

1.001.27**

1.001.39*

Type of residenceRural Urban

1.001.09

1.001.18*

1.001.05

ProvinceCentralCopperbeltEasternLuapulaLusakaNorthernNorthwesternSouthernWestern

1.001.201.69***1.281.80***0.790.44***1.210.57***

1.000.981.001.091.161.481.081.671.41

Source: 2009 HCP/Zambia Endline SurveyDifference from Referent Group: *p<0.05; **p<0.01; ***p<0.0011Among married respondents and unmarried respondents reporting sex in the past year (n=3642)2Among women with a child under 5 years of age (n=996)

Results! Community Capacity was measured through six domains:

- Participation- Collective Efficacy- Conflict Management- Leadership- Effective Leadership- Social Cohesion

Significant change in 6 domains of community capacity found in all intervention districts compared to comparison districts

Respondents living in an HCP Intervention District scored higher on more capacity scales than respondents in Comparison Districts.

Community capacity score was directly related to the community-led collective action.

Respondents living in communities with a greater level of community capacity were more likely to indicate that their community had worked together in the past year to solve a problem.

Communities that ‘worked together’ were: - 2 x use a modern contraceptive method - 1.8 times more likely to have received a HIV test and know results - 1.5 x more likely to have their youngest child sleep under a ITN

Results Con’t

• Overall 30% of community action was mediated by increases in community capacity (controlling for age; ed; media use. – Baron and Kenny, 1986).

• Increases in community capacity mediated the effect on health behaviors:

- 63% of contraceptive use was mediated by community action

- 11% of bed net use among young children was mediated by

community action

- No direct or indirect effect on HIV testing

Conclusions:

Community-generated capacity indicators were identified, validated and used to measure improved capacity

First time community capacity index validated and applied to a population based endline survey

Significant changes in community capacity measured in intervention areas over comparison areas

Changes in community capacity directly attributed to increased community collective action for health

Greater community capacity was significantly related to an increase in specific health outcomes!

Strengthening community capacity, in this instance, was both a means to an end – improved health behaviors and reported collective action for health – and an end-in-itself, both of which are vital to social development.

Still work to be done!

• Test these measures in other settings

• Simplify measurement tools and approaches

• Standardizing the integration of measuring community capacity into practice

Please join us in the challenge!

Thank you!

Web & Other Recent CM References

• How to Mobilize Communities for Health and Social Change – http://www.hcpartnership.org/Publications/Field_Guides/Mo

bilize/htmlDocs/cac.htm• “Rock Hop”

– http://www.comminit.com/strategicthinking/stcfscindicators/sld-1500.html

• Effects of a participatory interventions with women’s groups on birth outcomes (India), Lancet, April 3, 2010, T. Tripathy, et. al.

• Community Participation: Lessons Learned for Maternal and Child Health, M. Rosato, G. Laverack, L Howard-Grabman, et.al, Lancet, 2008: 372: 962-71

• An Evaluation of the Community Action Cycle, T.Kabore, HIV/AIDS Care and Support Program, Ethiopia, May, 2009.

More References: • Measuring Community Capacity for Better Health and Social

Outcomes - paper by Save the Children. Detailed overview of initiative to measure community capacity in Uganda, Nicaragua, Vietnam and Zambia.

•   Zambia Phase 1 PR Discussion Outline - qualitative tool to elicit community generated capacity indicators, Save the Children/HCP

• HCP/ZambiaEndlineSurvey - WomenQuestionnaire - quantitative endline household instrument. Section 1 A: Perception of Community includes the community generated indicators that had been validated, Save the Children/HCP

• Community Observation Checklist – to validate CC findings, Save the Children/HCP

What do you think?

1. What experience does your organization have on strengthening CC?

2. What tools/materials do you use?

3. What experience has your organization on measuring CC?

4. What tools/materials do you use?

5. Would you be interested in further measuring CC? 6. If yes, what support/materials/tool would be helpful?

•What experience does your organization have on strengthening CC?•What tools/materials do you use?•What experience has your organization on measuring CC?•What tools/materials do you use?

•Would you be interested in further measuring CC? If yes, what support/materials/tool would be helpful?

Endline Survey Design

– Baseline (2005); Endline (2009)– Sample Size: Baseline (3,000); Endline (4,000)– Endline fielded in 21 intervention districts

(Kalabo dropped) 13 comparison districts – External evaluation conducted by DCDM

(Baseline); Glow Consultancy (Endline) – Probability Sample – One male & female respondent from each

sample household; over 80% sample coverage rate

– Stratified into intervention and comparison districts; urban/rural

– Intervention districts refer to where HCP directly implemented programs