Evidence-based Programming for SeniorsTips for Selecting, Implementing, and Evaluating EBPs

Marcia Ory, PhD, MPHAshley Wilson, MPHProgram on Healthy AgingTexas A&M Health Science Center

South Padre, TXMay 2012

Topic for Discussion

EBP rationale, definitions, and program elements

Steps in choosing an EBP for your organization

Local, State, and National Examples

Challenges and Successes

Questions & Answers

Audience Query

How many are currently delivering

evidence-based programs (EBPs)?

What types of programs are you currently delivering?

What are Evidence-based Programs?

• Evidence-Based Programs are interventions based on evidence that is generated by scientific studies published in peer-reviewed journals.

• Model Programs are a type of evidence-based program

What are evidence-based programs?

• Previously implemented• Found to make positive differences in the lives of

participants

Evidence-based Program have been:

Adapted from Brownson et al, 2003

Type 1:

Evidence that a health issue

exists

Type 2:

Evidence about design, context, and attractiveness of program

Type 3:

Evidence that a

program is effective

Three Types of Evidence

Why the Recent Interest in EBP’s?

Magnitude and seriousness of health and social problems in our communities

Awareness of preventability of many problems

Emergence of evidence-based practices and programs

Spend limited resources more efficiently

Question

What are some advantages and disadvantages of evidence-based programs versus “home grown” ones?

Perceived Disadvantages of EBPs

Requires knowing where to find and how to understand/judge the evidence

Feels like standardization of program rather than site-specific tailoring

Tools and processes are unfamiliar

Difficult to build community support—many prefer “home grown” to “off the shelf”

Can be expensive

Online Training Modules: Evidence-based Health Promotion Programs for Older Adults

Increase in the likelihood of positive outcomesLead to efficient use of resourcesFacilitate the spread of programsFacilitate the use of common performance measuresSupport continuous quality improvementHelp to establish partnershipsMake it easier to justify funding

Online Training Modules: Evidence-based Health Promotion Programs for Older Adults

Perceived Advantages to EBPs

Tiered Set of Criteria for Defining EBPs Implemented Through OAA

FY2012 Congressional appropriations

require Title IIID funding to be

Evidence-Based

http://www.aoa.gov/AoARoot/AoA_Programs/HPW/Title_IIID/index.aspx

Minimal Criteria

Demonstrated through evaluation to be effective

for improving the health and wellbeing or reducing

disease, disability and/or injury among older adults

Ready for translation, implementation and/or broad dissemination by

community-based organizations using

appropriately credentialed practitioners.

Intermediate Criteria

Published in a peer-review journal.

Proven effective with older adult population, using some form of a control condition (e.g. pre-post study, case control design, etc.).

Some basis in translation for implementation by community level organization.

Highest-level Criteria

Undergone Experimental

or Quasi-Experimental

Design

Level at which full translation has occurred

in a community

site.

Level at which dissemination products have

been developed and are available to the public.

How Does One Chose a EBP?

Choosing an EBP

Step 7: Sustain the program

Step 6: Evaluate the program

Step 5: Translate the intervention into a program

Step 4: Select an intervention

Step 3: Establish broad-based partnerships

Step 2: Identify effective interventions

Step 1: Identify an important health issue among the Seniors you serve

The tasks do not always follow a sequential progression. The list provides a guide, not a strict set of steps.

Step 1: Identify an important health issue among the Seniors you serve

Review health statistics and other data to identify relevant health conditions and risk factors• Information sources include the Texas Department of State

Health Services and county Community Needs Assessment documents

Ask questions of seniors in your constituency

Determine if the identified health issue affects all of the Seniors you serve or only a subset

Step 2: Identify effective interventionsAging Texas Well• http://www.agingtexaswell.org/ebased/index.cfm

National Council on Aging• http://www.healthyagingprograms.org/content.asp?sectionid=32

National Cancer Institute• http://rtips.cancer.gov/rtips/index.do

Substance Abuse and Mental Health Services Administration • http://nrepp.samhsa.gov/Search.aspx

Partnering for SuccessIdentify organizations with a common mission• Align goals• Tap into the

organizational cultures and core values of partners

Communicate • Encourage

honesty• Clarify

expectations• Make sure

everyone is given the same information

Work together• Share praise,

glory, and recognition

• Realize that no partner is perfect

• Think creatively

Share Resources• Enhance participant

recruitment• Maximize program-

to-program referrals and dissemination

• Share facilities and materials

• Cross train staff and volunteers

• Increase data collection and

• data analysis capacity

Step 3: Establish broad-based partnerships

Step 4: Select an intervention

Program Match

• Does the program fit your mission?

• Is leadership supportive of the program?

• Are clients interested in the program?

• Can you implement the program as designed?

Costs and Resources

• Is there a licensing fee?

• What type of facilitator is needed?

• Is facilitator training required?

• What equipment and supplies are needed?

• Is the program ongoing or periodic?

Cultural and Contextual Relevance

• Is the program culturally appropriate?

• Are materials written at the correct literacy level and in the correct language?

• Does the program have regional appeal?

Looking for Evidence-Based ProgramsBased on Best Practices and PrinciplesSubjected to rigorous research designProgram is efficacious on outcomes of interestProgram is manualized for replicabilityTested in multiple settings and populationsPublished in peer review literatureEndorsed by professional bodyFeasible to move to scale

Is Your Community Ready to Implement Evidence-Based Health Promotion Programs?

There is buy-in from leadership

There is access both to personnel with the expertise to do these programs and to the population that needs these programs

There is funding for the program

Agency/partner-ship is willing to do evidence-based health programs and stay true to the model being implemented

National Council on Aging Organizational Readiness Checklist

Step 5: Translate the intervention into a program

• The process of taking a program originally implemented in a controlled “laboratory-like” setting and making it suitable for implementation in the community

Translation

• The faithful and accurate adherence to the core elements of an intervention, must be maintained

• Preserves the evidence-base

Fidelity

Step 5: Translate the intervention into a program (cont.)

• Program components that generated the positive outcomes

• Goal setting, identifying barriers, monitoring behavior change, etc.

Core elements

• Attributes that make the program “fit” the population

• Size of fonts and reading level of materials, marketing techniques, etc.

• Key characteristics can be adapted, but core elements cannot be adapted (jeopardizes fidelity)

Key characteristics

Step 6: Evaluate the program

Plan goals for process and

outcome evaluation before program implementation

Create or find the instruments and

protocols needed for data collection

Monitor program operations and

make adjustments to evaluation plan

(if needed)

Evaluate program delivery and

outcomes to assess program success

Use findings in future program

planning

Step 7: Sustain the programElements of a sustainable infrastructure and delivery

system

1 •Effective leadership

2 •Adequate delivery infrastructure

3 •Partnerships

4 •Centralized and coordinated logistical processes

5 •Business planning and financial sustainability

6 •Quality assurance and fidelity to interventions

CDSMP Example

Local Level

East Texas Coalition (Brazos Valley) – 750 participants with 544 completers

State Level

Texas – enroll 4,098 participants with a 72.5% completion rate (3,975 completers)

National Level

By March 31, 2012, 50,000 older adults will participate in CDSMP type programs nationwide

Texas Healthy Lifestyles

Bexar AAA-San Antonio Area

Central Texas AAA-Temple, Belton, Killeen

East Texas AAA-45 counties

Rio Grande AAA-El Paso Area

Tarrant Count AAA-Fort Worth

Region Target Enrollees Target Completers

Tarrant County 960 697

Central Texas 900 653

Rio Grande 888 645

East Texas Coalition 750 544

Bexar 608 436

Texas Targets by Region

Reach

Are we reaching our intended population?

How do we attract underserved populations?

How do we minimize dropout?

Recruiting: What Works?

Quality Leaders & Classes

Go to where seniors WORK, PLAY, PRAY, LIVE & EAT

Perseverance

Incentives

Food

Recruiting Program Participants

Networking• Get out in your community• Presentations • Word of mouth

Senior Luncheons

Senior Expos, Health Fairs

City Senior Department

How to Recruit New Partners

Identify common organizational mission

Make clear what expectations are—MOU!

• Go beyond usual set of partners• Identify various contributions

Think creatively

Keep a log of partner contacts

Recruiting Community Partners

Approaching Potential Partners• Networking• Program can be mutually beneficial to existing organizations

What to expect from a partner• Space• Participants• Leaders

What to provide partner• Program• Leaders• Materials

Quality Assurance & Program Fidelity

Quality Assurance refers to the entire scope of a state’s/organization’s program implementation.• National Level• Texas• East Texas Coalition

Fidelity refers very specifically to the degree to which a particular program is delivered as intended.• Stanford Patient Education Center (CDSMP)

Both include: planning, monitoring, evaluating and making corrective actions• Have a plan

Steps to Program Fidelity4. Create a communications mechanism for feedback

3. Provide copies of plan and set training schedule

2. Prepare state-specific fidelity plan

1. Read Implementation Manual for each program

For reference, view NCOA Module 4: “Assuring Program Quality” at www.ncoa.org/chamodules

Example Fidelity Plan

Balancing Treatment Fidelity and Adaptation

• How to maintain fidelity to essential elements?

• What is needed to tailor programs to populations?

• Employ cultural, age or gender appropriate examples

• Keep the principles---Contextual the examples!

Considerations in Fidelity Monitoring and Enhancement Identify and operationalize

essential components of original program

Know if adaptations are consistent with original study

Simplify data collection

Maximize fidelity with “teach-back” where instructors have the opportunity to practice teaching during the trainings

With appreciation from Minnesota AAA

Challenges

Participant Attrition

Leader Attrition

Program Fidelity

Data Collection

Solutions • Recruit & provide quality classes in community• Provide transportation• Know community trends• Rural communities = fewer participants

Participant Attrition

• Provide leaders classes to teach immediately after training

• Involve leaders in scheduling classes• Stipends if available

Leader Attrition

• Dedicated Program Coordinator• Protocol for fidelity checksProgram Fidelity

• Academic partnersData Collection

Program Evaluation•What do we want to know?

•Create instruments•Data collection process?•Who collects & manages the

data?•Database creation

•How to provide feedback to community & stakeholders?

Why is this

important?

Program Evaluation TipsDo you have other data collection requirements?• AAA Client intake forms

Know target population’s abilities • Age• Reading Level• Eye sight

Don’t let it take too much time away from your program

KISS (Keep It Short & Sweet)

Texas Size Successes

Exceeded our goal of 4,000 enrolleesDeveloped training infrastructureSustainability mechanisms• Diabetes accreditation• Partnering with health care organizationsClustering of EBPs

Conclusions

EBPs have many benefits for improving population health

There are inventories of EBPs—no need to start from scratch

Partnerships are critical for recruitment, adoption, maintenance

For maximum impact—consider the context!

For more information:

Program on Healthy Aging 1266 TAMU

College Station, TX 77843-1266

Phone: 979-458-4202Fax: 979-458-4264

www.programonhealthyaging.org