arizona state university college of nursing and health innovation [email protected] linda k...
TRANSCRIPT
![Page 1: Arizona State University College of Nursing and Health Innovation Linda.larkey@asu.edu Linda K Larkey, PhD Scottsdale Healthcare Professor of Biobehavioral](https://reader034.vdocuments.site/reader034/viewer/2022051516/56649f325503460f94c4ecc8/html5/thumbnails/1.jpg)
Arizona State University College of Nursing and Health [email protected]
Linda K Larkey, PhDScottsdale Healthcare Professor of Biobehavioral Oncology Research
![Page 2: Arizona State University College of Nursing and Health Innovation Linda.larkey@asu.edu Linda K Larkey, PhD Scottsdale Healthcare Professor of Biobehavioral](https://reader034.vdocuments.site/reader034/viewer/2022051516/56649f325503460f94c4ecc8/html5/thumbnails/2.jpg)
overview
evaluation paradigms
what is an “evaluation theory”?
what does a theory buy you?
how do you build theoretical constructs into evaluation?
practice
![Page 3: Arizona State University College of Nursing and Health Innovation Linda.larkey@asu.edu Linda K Larkey, PhD Scottsdale Healthcare Professor of Biobehavioral](https://reader034.vdocuments.site/reader034/viewer/2022051516/56649f325503460f94c4ecc8/html5/thumbnails/3.jpg)
positivist approach
assumes there are “objective”, observable and measurable aspects of a program
preference for predominantly quantitative evidence needs assessment assessment of program theory assessment of program process impact assessment efficiency assessment
(Rossi, Lipsey and Freeman, 2004)
![Page 4: Arizona State University College of Nursing and Health Innovation Linda.larkey@asu.edu Linda K Larkey, PhD Scottsdale Healthcare Professor of Biobehavioral](https://reader034.vdocuments.site/reader034/viewer/2022051516/56649f325503460f94c4ecc8/html5/thumbnails/4.jpg)
three basic paradigms
positivist
Interpretive
critical-emancipatory
you don’t have to join a club--- feel free to blend
Potter, C. (2006). Program Evaluation. In M. Terre Blanche, K. Durrheim & D. Painter (Eds.), Research in practice: Applied methods for the social sciences (2nd ed.)
![Page 5: Arizona State University College of Nursing and Health Innovation Linda.larkey@asu.edu Linda K Larkey, PhD Scottsdale Healthcare Professor of Biobehavioral](https://reader034.vdocuments.site/reader034/viewer/2022051516/56649f325503460f94c4ecc8/html5/thumbnails/5.jpg)
interpretive approaches
purports it is essential that the evaluator develops an understanding of the perspective, experiences, needs and expectations of all stakeholders
key words, “understanding” “meaning” “rich description”
considered crucial before one is able to make judgments about the merit or value of a program.
The evaluator’s contact with the program is important, over time observation Interviews focus groups
![Page 6: Arizona State University College of Nursing and Health Innovation Linda.larkey@asu.edu Linda K Larkey, PhD Scottsdale Healthcare Professor of Biobehavioral](https://reader034.vdocuments.site/reader034/viewer/2022051516/56649f325503460f94c4ecc8/html5/thumbnails/6.jpg)
critical-emancipatory approaches based on action research for the
purposes of social transformation
particularly useful in developing countries or underserved populations
concerns: empowering populations to create change from within, providing “voice”
![Page 7: Arizona State University College of Nursing and Health Innovation Linda.larkey@asu.edu Linda K Larkey, PhD Scottsdale Healthcare Professor of Biobehavioral](https://reader034.vdocuments.site/reader034/viewer/2022051516/56649f325503460f94c4ecc8/html5/thumbnails/7.jpg)
so what?
it matters whether you think your goal is to achieve some measurable state or reality…
or, if you think that the process along the way and people’s opinions and perceptions are most important
or, if you see the program goals to empower, raise consciousness, give voice
![Page 8: Arizona State University College of Nursing and Health Innovation Linda.larkey@asu.edu Linda K Larkey, PhD Scottsdale Healthcare Professor of Biobehavioral](https://reader034.vdocuments.site/reader034/viewer/2022051516/56649f325503460f94c4ecc8/html5/thumbnails/8.jpg)
What is the result expected? Typical logic: Process: We will reach X number of
female, single heads of household with food stamp information by….
Short-term outcome: We will distribute $X within Maricopa County by…
Longer-term outcome: We will increase the use of food stamps by $X
Impact: We will reduce food shortage by X%
![Page 9: Arizona State University College of Nursing and Health Innovation Linda.larkey@asu.edu Linda K Larkey, PhD Scottsdale Healthcare Professor of Biobehavioral](https://reader034.vdocuments.site/reader034/viewer/2022051516/56649f325503460f94c4ecc8/html5/thumbnails/9.jpg)
what “causes” the desired end result? how would we… reduce food instability in our state
reduce flu epidemics
increase number of servings of F & V eaten by school age children in school lunch
increase cancer screening rates what else?
![Page 10: Arizona State University College of Nursing and Health Innovation Linda.larkey@asu.edu Linda K Larkey, PhD Scottsdale Healthcare Professor of Biobehavioral](https://reader034.vdocuments.site/reader034/viewer/2022051516/56649f325503460f94c4ecc8/html5/thumbnails/10.jpg)
Pick one “impact” level goal What causes it? (Needs assessment,
what is the problem, what contributes?)
What do you think might fix it? One thing or many?
How would you know if it were fixed?
![Page 11: Arizona State University College of Nursing and Health Innovation Linda.larkey@asu.edu Linda K Larkey, PhD Scottsdale Healthcare Professor of Biobehavioral](https://reader034.vdocuments.site/reader034/viewer/2022051516/56649f325503460f94c4ecc8/html5/thumbnails/11.jpg)
That is your Evaluation Theory Once you name the causes, purported
improvement strategies, and expected outcome, you have a LOGIC MODEL
Your construal of the problem (antecedents) is YOUR THEORY
This theory, PLUS your assumptions about what is important to change to get a result is your EVALUATION THEORY
![Page 12: Arizona State University College of Nursing and Health Innovation Linda.larkey@asu.edu Linda K Larkey, PhD Scottsdale Healthcare Professor of Biobehavioral](https://reader034.vdocuments.site/reader034/viewer/2022051516/56649f325503460f94c4ecc8/html5/thumbnails/12.jpg)
![Page 13: Arizona State University College of Nursing and Health Innovation Linda.larkey@asu.edu Linda K Larkey, PhD Scottsdale Healthcare Professor of Biobehavioral](https://reader034.vdocuments.site/reader034/viewer/2022051516/56649f325503460f94c4ecc8/html5/thumbnails/13.jpg)
Simpler “Logic Model”: ATM
![Page 14: Arizona State University College of Nursing and Health Innovation Linda.larkey@asu.edu Linda K Larkey, PhD Scottsdale Healthcare Professor of Biobehavioral](https://reader034.vdocuments.site/reader034/viewer/2022051516/56649f325503460f94c4ecc8/html5/thumbnails/14.jpg)
Low rates of colorectal
cancer screening
Patients can’t access
or are discouraged
from accessing services
Long Waits for services
Facilities Lacking
Not enough trained providers
Lack of funding for uninsured
(screening/ treatment)
Inadequate Lobbying
(for policy and/or funding) Existing health
care plans inadequate
Cultural factors
Providers not making referralsProviders not clear
on guidelines (who/what/when/ri
sk)
Patients not demanding screening
Patients not aware of need/importance/valu
e of screening
New and complicated
message(not your average bumper sticker)
Lack of office reminder and/or flagging systems
Conflicting research evidence, that
screening saves lives, or best method
Uncomfortable topic (distaste for “things
scatological”)
Inappropriate Utilization
Competing priorities during patient/provider appointment: too little
time
Mixed/unclear financial impact (will referred
patients return? reimbursement?)
![Page 15: Arizona State University College of Nursing and Health Innovation Linda.larkey@asu.edu Linda K Larkey, PhD Scottsdale Healthcare Professor of Biobehavioral](https://reader034.vdocuments.site/reader034/viewer/2022051516/56649f325503460f94c4ecc8/html5/thumbnails/15.jpg)
Such complex map of problems… what’s the solution?
Research Project DevelopmentAnd Evaluation
State/County DataPublic Messaging
Capacity for DeliveryCommunity-based Organizations
VolunteersCommunity Visibility
Legislative/PolicyDialogue Leadership
AZCC
(academic)
ADHS
(public
health)
ACS
(private)
Health Services Delivery
Figure 2. Transdisciplinary Model Approach for Increasing CRC Screening in Arizona
![Page 16: Arizona State University College of Nursing and Health Innovation Linda.larkey@asu.edu Linda K Larkey, PhD Scottsdale Healthcare Professor of Biobehavioral](https://reader034.vdocuments.site/reader034/viewer/2022051516/56649f325503460f94c4ecc8/html5/thumbnails/16.jpg)
![Page 17: Arizona State University College of Nursing and Health Innovation Linda.larkey@asu.edu Linda K Larkey, PhD Scottsdale Healthcare Professor of Biobehavioral](https://reader034.vdocuments.site/reader034/viewer/2022051516/56649f325503460f94c4ecc8/html5/thumbnails/17.jpg)
building your own evaluation plan… share example of one of your programs map “antecedent conditions”– what are
the causes? (theory of the problem) so, if those are the causes, what could
fix this problem? (theory of solution) what is the final outcome expected? then…
measure what along the way? At the end? things you can count? meaning/perception? what does the community say? Remember
the “critical” view
![Page 18: Arizona State University College of Nursing and Health Innovation Linda.larkey@asu.edu Linda K Larkey, PhD Scottsdale Healthcare Professor of Biobehavioral](https://reader034.vdocuments.site/reader034/viewer/2022051516/56649f325503460f94c4ecc8/html5/thumbnails/18.jpg)
holding your vision while negotiating the terrain…