this work is licensed under a creative commons attribution ... · importance of a conceptual...
TRANSCRIPT
![Page 1: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/1.jpg)
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this site.
Copyright 2006, The Johns Hopkins University and Jane Bertrand. All rights reserved. Use of these materials permitted only in accordance with license rights granted. Materials provided “AS IS”; no representations or warranties provided. User assumes all responsibility for use, and all liability related thereto, and must independently review all materials for accuracy and efficacy. May contain materials owned by others. User is responsible for obtaining permissions for use from third parties as needed.
![Page 2: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/2.jpg)
Fundamentals of Program Evaluation Course 380.611
Developing a Conceptual Framework, and Introduction to Formative
Research
![Page 3: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/3.jpg)
Topics to cover today
Importance of a conceptual frameworkKey points from Earp and Ennett articleExamples of conceptual frameworksIn-class group discussion on CFsDiscussion of Assignment #1Introduction to formative research
![Page 4: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/4.jpg)
Importance of a conceptual framework (program theory)
Articulates the pathways by which an intervention is expected to cause the desired outcomesProvides evaluator with specific elements to assessOther names:
Logic model, program model, outcome line, cause map, action theory
![Page 5: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/5.jpg)
Conceptual Framework of Family Planning Demand and Program Impact on Fertility
Societal and Societal and IndividualIndividual
factorsfactors
Value and Value and Demand Demand
for Childrenfor Children
FP DemandFP Demand•• SpacingSpacing•• LimitingLimiting
DevelopmentDevelopmentProgramsPrograms
Family PlanningFamily PlanningSupply FactorsSupply Factors
Service Outputs:Service Outputs:•• AccessAccess•• QualityQuality•• Image/Image/
AcceptabilityAcceptability
ServiceServiceUtilizationUtilization
Other HealthOther Healthand Social and Social
ImprovementsImprovements
ContraceptiveContraceptivePracticePractice
OtherOtherIntermediateIntermediate
VariablesVariables
FertilityFertility•• WantedWanted•• UnwantedUnwanted
![Page 6: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/6.jpg)
Conceptual models: Earp and Ennett (1991)
Definition of a conceptual model: Diagram of proposed causal linkages among a set of concepts believe to be related to a particular public health problem.
Concepts = in boxesProcesses = shown by arrowsCan reflect factors at multi levels (macro to micro)
![Page 7: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/7.jpg)
Conceptual models
Often draw on:One or more theoriesEmpirical evidenceKnowledge specific to the particular case
Serve to:Summarize and integrate knowledgeProvide explanations for causal linkagesGenerate hypotheses
![Page 8: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/8.jpg)
Building a conceptual model
Start with the endpoint (dependent variable, outcome, or target point for intervention)Identify potential correlates, based on empirical or theoretical evidenceShow antecedent or mediating variables by proximity to dependent variables
![Page 9: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/9.jpg)
Conventions for drawing a conceptual model
1) Only include concepts that will be operationally defined and measured
2) Present left-to-right or top-to-bottom3) Use arrows to imply causality4) Label concepts succinctly5) Do not include operational definitions
or values of variables in the model
![Page 10: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/10.jpg)
How to “think through” a conceptual framework
Example: Tobacco Prevention and Control
Interventions to Reduce Exposure to Environmental Tobacco Smokehttp://www.thecommunityguide.org/tobacco/tobac.ppt
Tobacco use is the single largest cause of preventable premature mortality in the United States. It also represents an enormous cost burden to the nation. The question is, what works to make tobacco use prevention and control at the population or community level? The Guide to Community Preventive Services addresses the effectiveness of community-based interventions for three strategies to promote tobacco use prevention and control: 1) prevent tobacco product use initiation, 2) increase cessation and 3) reduce exposure to environmental tobacco smoke (ETS). The findings strengthen and complement existing guidelines (hyperlink table and text to existing guidelines) on tobacco prevention and control.
![Page 11: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/11.jpg)
Analytic FrameworkFor every intervention that we evaluate in the Community Guide, we develop an analytic framework, in which we postulate how we think the intervention works and what outcomes we think are important to capture information on. In many cases, our analytic frameworks change over the course of our reviews as we learn more about the intervention, the potential outcomes, and the body of the evidence in the literature.Let’s start with our intervention: Smoking bansAnd our goal: A reduction in morbidity and mortality.. Our analytic framework will connect these two.
![Page 12: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/12.jpg)
Analytic Framework: Smoking Bans
ReducedMorbidity
andMortality
SmokingBans
![Page 13: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/13.jpg)
Smoking Bans Might Result from Community Education Efforts
Now it’s important to recognize that smoking bans might be the result or outcome of interventions, such as a community-wide education and/or political campaign. The experience from the state of California with a broad smoking ban has been described quite well in the literature, giving you a blow by blow account of how that state managed to adopt and implement and extend a statewide clean indoor air laws.It’s also important to note that in many states, pre-emption legislation precludes local governments from strengthening clean indoor air laws. These efforts have been described as a industry-sponsored effort to obstruct efforts to extend protections from ETS to workers.
![Page 14: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/14.jpg)
Smoking Bans Might Result from Community Education Efforts
ReducedMorbidity
andMortality
SmokingBans
CommunityEducation
![Page 15: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/15.jpg)
Bans Might Reduce ETS Exposure
Now back to how smoking bans work. They work in one or three ways.First, directly by reducing exposure to ETS in the restricted environment. This alone will have health effects.
![Page 16: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/16.jpg)
Bans Might Reduce ETS Exposure
ReducedMorbidity
andMortality
ReducedExposure to ETS
SmokingBans
![Page 17: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/17.jpg)
Bans Might Increase Smoking Cessation
Second, smoking bans might work by affecting the tobacco use behaviors of smokers
In response to a smoking ban, they might think twice about continuing their habit. They might reduce their daily consumption of tobacco, and these two effects might increase the number who attempt to quit. Since smoking bans also reduce ques to smokers to relapse, more smokers attempting to quit will be successful. This will result in fewer tobacco users and a reduction in adverse health outcomes.
![Page 18: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/18.jpg)
Bans Might Increase Smoking Cessation
ReducedMorbidity
andMortality
SmokingBans
ReducedExposure to ETS
FewerTobacco
Users
Change In
Attitudes
ReducedConsumption
IncreasedQuit
Attempts
IncreasedCessation
![Page 19: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/19.jpg)
Bans Might Reduce Smoking Initiation
Finally, we acknowledge that smoking bans might directly affect tobacco consumption by youth or affect their impressions of the social desirability of smoking. These will reduce tobacco use prevalence among adolescents and contribute to fewer tobacco users.
![Page 20: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/20.jpg)
Bans Might Reduce Smoking Initiation
ReducedMorbidity
andMortality
SmokingBans
ReducedExposure to ETS
Change In
Attitudes
ReducedInitiation
FewerTobacco
Users
Change In
Attitudes
ReducedConsumption
IncreasedQuit
Attempts
IncreasedCessation
![Page 21: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/21.jpg)
Bans Might Increase ETS in the Home
One unintended effect described in the literature, at least initially, was a concern that smokers might respond to workplace smoking restrictions by compensating at home, smoking more and thus increasing ETS exposures in the home.We looked for evidence of this potential harm in our review.We also examined the evidence, not shown here, that smoking bans in restaurants and hotels adversely affects business revenue and tourism.
![Page 22: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/22.jpg)
Bans Might Increase ETS in the Home
ReducedMorbidity
andMortality
SmokingBans
ReducedExposure to ETS
Change In
Attitudes
ReducedInitiation
FewerTobacco
Users
Change In
Attitudes
ReducedConsumption
IncreasedQuit
Attempts
IncreasedCessation
DivertedConsumption
IncreasedHome Exposure
![Page 23: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/23.jpg)
Body of Evidence: Bans and Restrictions
We did a series of electronic database searches, and screened titles and abstracts and ended up with the following body of evidence:
56 studies were reviewed17 studies measured differences or changes in ETS exposure, of which 10 met our criteria for good or fair quality51 studies measured smoking habits of employees exposed to bans or restrictions, of which only 9 met our criteria for good or fair. In most cases, the excluded studies did not included concurrent comparison groups.
![Page 24: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/24.jpg)
Study Measurements by Outcome
If you take all of the qualifying studies, and plotted their outcomes to slots on our analytic framework this is what we found across this body of evidence.For example we have 12 measurements of differences or changes in exposure to ETS, 6 studies of changes in tobacco use prevalence among employees, 4 measurements of cessation by smoking employees etc.
![Page 25: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/25.jpg)
Study Measurements by Outcome
ReducedMorbidity
andMortality
SmokingBans
ReducedExposure to ETS
Change In
Attitudes
ReducedInitiation
FewerTobacco
Users
Change In
Attitudes
ReducedConsumption
IncreasedQuit
Attempts
IncreasedCessation
DivertedConsumption
IncreasedHome Exposure
12
6
4591
0*0*6
![Page 26: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/26.jpg)
Examples of different conceptual frameworks
![Page 27: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/27.jpg)
Model of Program Impact
SocioeconomicsStatus
Gender
Income
Education
PsychographicCharacteristics
FamilyCharacteristics
InterpersonalContacts
ProgramExposure
Attitude
Knowledge
Practice
![Page 28: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/28.jpg)
Determinants of Domestic Violence(no intervention)
Contextual andContextual andCommunity Factors
Household and IndividualHousehold and Individual--Level FactorsCommunity Factors Level Factors
•• SocioeconomicSocioeconomicdevelopmentdevelopment
•• Domestic violence Domestic violence normsnorms
•• Gender inequalityGender inequality
•• Crime levelsCrime levels
•• SocioeconomicSocioeconomicstatusstatus
•• Life cycle factorsLife cycle factors
•• IntergenerationalIntergenerationalexposure to violenceexposure to violence
•• Risk behaviorsRisk behaviors
WomenWomen’’s status/s status/autonomyautonomy
DomesticDomesticviolenceviolence
![Page 29: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/29.jpg)
Domains forCommunication Interventions
Communication to Strengthen the Social PoliticalEnvironment
Communication for Effective
Service DeliverySystems
Communicationto Create
Health Literate Communities
and Individuals
Conceptual FrameworkPathways to a Health-Competent Society
Conceptual FrameworkConceptual FrameworkCommunication Pathways to a HealthCommunication Pathways to a Health--Competent SocietyCompetent Society
![Page 30: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/30.jpg)
Domains forCommunication Interventions Initial Outcomes
Communication-Social PoliticalEnvironment
Communicationfor Service
DeliverySystem
Communication for Community/
Individual
Environment
Service Systems
Community
Individual
Conceptual FrameworkConceptual FrameworkCommunication Pathways to a HealthCommunication Pathways to a Health--Competent SocietyCompetent Society
![Page 31: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/31.jpg)
Domains forCommunication Interventions Initial Outcomes
BehavioralOutcomes
SupportiveEnvironment
Service Performance
Client Behaviors:
Community
Individual
Environment
Service Systems
Community
Individual
Communication-Social PoliticalEnvironment
Communicationfor Service
DeliverySystem
Communication for Community/
Individual
Conceptual FrameworkConceptual FrameworkCommunication Pathways to a HealthCommunication Pathways to a Health--Competent SocietyCompetent Society
![Page 32: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/32.jpg)
Domains forCommunication Interventions Initial Outcomes
BehavioralOutcomes
SustainableHealth Outcomes
USAID 5 SOs
SupportiveEnvironment
Service Performance
Client Behaviors:
Community
Individual
Environment
Service Systems
Community
Individual
Communication-Social PoliticalEnvironment
Communicationfor Service
DeliverySystem
Communication for Community/
Individual
Conceptual FrameworkConceptual FrameworkCommunication Pathways to a HealthCommunication Pathways to a Health--Competent SocietyCompetent Society
![Page 33: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/33.jpg)
UnderlyingConditions
Domains forCommunication Interventions Initial Outcomes
BehavioralOutcomes
SustainableHealth Outcomes
Context
Resources
USAID 5 SOs
SupportiveEnvironment
Service Performance
Client Behaviors:
Community
Individual
Environment
Service Systems
Community
Individual
Communication-Social PoliticalEnvironment
Communicationfor Service
DeliverySystem
Communication for Community/
Individual
Conceptual FrameworkConceptual FrameworkCommunication Pathways to a HealthCommunication Pathways to a Health--Competent SocietyCompetent Society
![Page 34: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/34.jpg)
UnderlyingConditions
Domains forCommunication Interventions Initial Outcomes
Envi
ronm
ent
Serv
ice
Syst
ems
Com
mun
ityIn
divi
dual
BehavioralOutcomes
SustainableHealth Outcomes
ContextDisease Burden
Social
Cultural
Economic
Communication Technology
Political
Legal
USAID 5 SOs
Reduction in:
Reduction in:Unintended/mistimed
Pregnancies
Morbidity/mortalityFrom pregnancy/
Childbirth
Infant/child morbidity/mortality
HIV transmission
Threat of infectious diseases
ResourcesHuman and Financial
Resources
Strategic Plan/Health
Priorities
Other Development Programs
Policies
SupportiveEnvironment:
• Multi-sectoralpartnerships
• Public opinion• Institutionalperformance
• Resource acquisition• Media support• Activity level
Service Performance:
• Access• Quality• Client volume• Client satisfaction
Client Behaviors:
Community• Sanitation• Hospice/PLWA• Other actionsIndividual• Timely service use• Contraception• Abstinence/partner reduction
• Condom use• Safe delivery• BF/nutrition• Child care/immuniz.• Bednet use
• Political will• Resource allocation• policy changes• Institutional capacitybuilding
• National coalition• National communication strategy
• Availability• technical competence• Information to client• Interpersonalcommunication
• Follow-up of clientsIntegration of services
• Leadership• Participation equity• Information equity• Priority consensus• Network cohesion• Ownership• Social norms• Collective efficacy• Social capital
• Message recall• Perceived socialsupport/stigma
• Emotion and values• Beliefs and attitudes• Perceived risk• Self-efficacy• Health literacy
Communication-Social PoliticalEnvironment
Communicationfor Service
DeliverySystem
Communication for Community/
Individual
Conceptual FrameworkConceptual FrameworkCommunication Pathways to a HealthCommunication Pathways to a Health--Competent SocietyCompetent Society
![Page 35: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/35.jpg)
Requirement for exercise #1
Present the diagram in terms of initial, intermediate, and long-term outcomes
Note: this is NOT a standard requirement of conceptual frameworks but it is a useful way to look at program effects.
![Page 36: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/36.jpg)
Criteria for grading conceptual framework on exercise #1
Diagram respects the 5 “conventions for drawing a conceptual model”The model presented is:
Conceptually clear (explains to the reader how you expect the program to achieve its objectives)Visually pleasingConcise but covers key factors (suggestion: include 10-15 concepts in your model)
![Page 37: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/37.jpg)
Rules relating to confounding and modifying variables (Earp & Ennett)
See page 169 of the article
Technically fine, but not necessarily used among all researchers
In exercise #1, don’t feel bound by these two rules.
![Page 38: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/38.jpg)
Formative evaluation
Guides the design of a program
Different types:Needs assessment (esp. in U.S.)Diagnostic (formative) research(Specific to media) Pretesting
![Page 39: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/39.jpg)
Needs assessment in the program cycle (McDavid)
Strategic PlanningEnvironment ScanningStakeholder Input
Program DevelopmentStakeholder Input
Program ImplementationProgram Evaluation
Stakeholder assessments of services/outputs in relation to needs (relevance)
Program Accountability
![Page 40: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/40.jpg)
Steps in conducting a needs assessment (McDavid & Hawthorn)
Become familiar with political contextIdentify users and usesIdentify target pop. (geographic, socio-dem)Inventory existing services (what gaps exist?)Identify needsPrepare document
Evidence, benchmarks, conclusions, recs
Communicate findings, implement
![Page 41: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/41.jpg)
Use of benchmarks in needs assessment
Compare current levels and types of services to benchmarks (or reference points)
Conceptions of human needs Moral/ethical values (“no child left behind”)Levels of service provided elsewhereService provider opinions/preferenceClient (current, prospective) opinions
![Page 42: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/42.jpg)
Sources of data: primary (new) & secondary (existing)
Lit reviewsSimilar studiesDemographic statistics Government reports
Surveys (mail, phone, in person)Focus groupsInterviewsDirect observation
![Page 43: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/43.jpg)
Diagnostic research (very similar to needs assessment)
Also called “formative research” or “formative evaluation”
Learn more about all aspects of the problem, population, and context
![Page 44: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/44.jpg)
Diagnostic research uses both quantitative & qualitative
Quantitative (demographic, epidemiological):
To quantify the extent of the problem To identify subgroups most affectedTo identify explain determinants
Qualitative:To understand problem from user perspective, identify barriers
![Page 45: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/45.jpg)
Great diversity in types of formative research
Examples:Formative research for Stop Aids Love Life
Louisiana study on teen smoking behavior
Investigation of places with “high rates of new partner acquisition” (PLACE methodology)
![Page 46: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/46.jpg)
Publication of formative research in peer-reviewed lit
Quite rareResults often presented in a report
Ex: Stop AIDS Love Life
More likely in form of baseline findingsLouisiana adolescent smoking study
If value goes beyond study locationPLACE methodology in S. Africa
![Page 47: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/47.jpg)
Key points from Louisiana smoking study
National surveys of adolescent smoking didn’t provide adequate data on target population
Survey of 4808 students provided data:Smoking patterns by ethnic group, genderSocial relationships related to smoking
Friends, family; smoking and alcohol
![Page 48: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/48.jpg)
Example of a baseline survey as a “two-fer” (two for one)
Formative research in form of baseline survey serves two purposes:
Establishing a baseline level against which to evaluate program after interventionProviding insights into the problem that help to guide the design of the program
![Page 49: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/49.jpg)
Findings from LA study useful in developing intervention
LA rate higher than national rate for adolescentsWho was most likely to smoke:
Among whites: no male/female differencesAmong blacks: males more likely to smokeBoth black and whites:
Discretionary $$Low academic achievers
![Page 50: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/50.jpg)
Findings from LA study useful in developing intervention
Strong relationship of smoking to:Smoking of family & friends, alcohol use
Authors discuss challenges of designing a program with these dynamics
Formative research doesn’t give all the answers to program design!
![Page 51: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/51.jpg)
PLACE (“priorities for local AIDS control activities) methodology
MEASURE Evaluation Project (UNC)
Identifies where to access sexual networks with individuals with “high rates of new partner acquisition”
Provides information on availability of preventive services (info, condoms)
![Page 52: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/52.jpg)
Methods of PLACE: 3 phases of data collection
Key informants: “where do people meet new sexual partners?”
Community leaders, health care providers, youth on street, taxi drivers, STD clients
Visit to sites compiled from interviewsType of site, patrons, AIDS prevention?Sites marked on aerial map
Interviews with people at these sites
![Page 53: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/53.jpg)
Useful information for designing an intervention
Key locations: taverns and shebeens<2% of sites had on-site anti-AIDS info<10% didn’t have condoms onsiteAlmost 60% of owners/managers would be willing to have condoms onsitePatrons at these locations – frequent visitors (“regulars”)
![Page 54: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/54.jpg)
Indicators useful for monitoring programs
# new sites identified – sexual activity% of sites with condoms (verified)Mean rate of new partnership formation at site in past 4 weeks, by genderPortion of patrons who ever used a condomPortion used condom at last sex
![Page 55: This work is licensed under a Creative Commons Attribution ... · Importance of a conceptual framework (program theory) Articulates the pathways by which an intervention is expected](https://reader030.vdocuments.site/reader030/viewer/2022040823/5e6e09f957189b0eab2ef91a/html5/thumbnails/55.jpg)
Use of qualitative research to inform quantitative
To learn vocabulary used by local population to describe problem
Yoder study on diarrhea: 9 different words
To identify new concepts that researchers hadn’t consideredTo generate hypotheses to be tested through subsequent research