“If you don’t know where you are going, how are you gonna’ know when you get there?”

Yogi BerraCommon Elements of Strong Program Evaluation

THEORY OF CHANGE: Cause/effect relationship of program

LOGIC MODEL: Identifies key program components and

their relationship to program outcomes

PERFORMANCE MEASUREMENT & IMPACT EVALUATION: Differences between the two

If you deliver the intervention as planned, it will bring about a measurable change/outcome in the

community/participants in relation to your identified community need.

Theory of Change Main Elements

LOGIC MODEL

INPUTS: What we investACTIVITIES: What we doOUTPUTS: Direct products from program activitiesEVIDENCE OF CHANGE/OUTCOMES

Short-Term Outcomes – one year Changes in knowledge, skills, attitudes

Medium -Term Outcomes – three yearsChanges in behavior or action resulting from new knowledge

Long-Term Outcomes – ten yearsMeaningful changes in condition

Birth & Beyond Evaluation PracticesA home visitation program at 9 Family Resource Centers serving parents

at risk of child abuse and neglect to prevent entry/re-entry into Child Protective Services

Common OutcomesSacramento County Commitment to CPS

Evaluation External Evaluator from the onsetShared Database across all 9 sitesMonthly Data Review distributed program wideQuarterly and Annual Reports Program Improvement Based on Data & Outcomes

Birth & Beyond Lessons Learned Integrated evidence-based practices into the model Intervention must align with assessments Analyze attrition of those served and ways to improve Member training aligns with intervention Since 2008, serving higher % of at-risk risk families Data facilitates peer to peer learning Track/monitor short-term outcomes to ensure meeting

medium-term outcomes Expanded services by adding one FRC (based on data)

Birth & Beyond Evaluation Challenge Impact Evaluation: strives for high rigor with scientifically-based

research design comparing people exposed to an intervention (“experimental” group) to people not exposed (“control” or “comparison” group)

How do you design an impact evaluation that does not deny services toparents at risk of child abuse & neglect?

Technical assistance from NORC Work in partnership with Sacramento County Child Protective Services to

enhance B&B evaluation conducting a quasi-experimental study Select a matched sample of families who have been referred to CPS

during the same time as Birth & Beyond families and did NOT receive Birth & Beyond services (“control” or “comparison” group)

20 Year Child Death Review Team Report

PERFORMANCE MEASUREMENT & IMPACT EVALUATION

Performance MeasurementProvides a view of how program is functioningDoes not “prove” that an intervention caused an outcomeStrives for high quality data, striking a balance between rigor & staff timeFocuses on shorter term changes, observed within a year

Impact Evaluation Strives for high rigor; with scientifically-based research designsSeeks to prove causal relationship; program is the specific cause of improvements within the target populationEvidence that intervention causes the outcomes Focuses on longer term changes as well as short term outcomes