Working on Program Implementation and Fidelity: The DEBI Experience

Jonny F. Andia, Ph.D Behavioral Scientist

2011 National HIV Prevention Conference – Atlanta, GA August 14th – 17th 2011

National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention Division of HIV/AIDS Prevention

Background

• Evidence-based interventions (EBIs) are part of the National HIV/AIDS Strategy to reduce the incidence of HIV in the U.S. and territories.

• The CDC/DEBI project disseminates over 23 EBIs with multiple national partners.

• Correct adoption of EBI programs, fidelity and quality assurance (QA) are of paramount importance.

Purpose of the Presentation

• Describe implementation tools and quality assurance procedures related to EBIs and other Public Health Strategies (PHS)

• Describe the EBI dissemination process and methods used by CBA (capacity building assistance) partners to help implement and assess fidelity among CBOs (community-based organizations)

The DEBI Project

• This project disseminates evidence-based HIV

prevention interventions into prevention practice. • DEBI’s goal is achieved by development and

coordination of a national-level strategy providing quality assurance related to: – Training – Technical assistance – Providing national infrastructure and support for CBA

activities to state health departments and CBOs

* Charles Collins, et al., The diffusion of Effective Behavioral Interventions Project: Development, Implementation and Lessons Learned. Moving Science Into Practice, AIDS Education and Prevention (SS) Vol. 18, Spl A. August 2006

STRATEGIC PLANING

HEALTH MARKTING STRATEGY

INTERVETNION PROGRAM

PACKAGE DESIGN

TRAINING

POLICY AND INCENTIVES

CAPACITY BUILDING

ASSISTANCE, TA & FIDELITY

EVALUATION & QUALITY CONTROL

DEBI Model of Dissemination, Implementation and Fidelity*

The Intervention Package Design

• CDC develops and evaluates prototype training materials with members of the target population and in collaboration with original researchers to ensure transfer of science-based technology into practice.

– Promotional videos – Instructional videos – Detailed protocols – Evaluation guides, including sample evaluation tools – Technical assistance guides – Sample health education materials

Note: These should be living documents which may be periodically updated so

as to improve implementation.

Training Delivery Strategy

• A range of pedagogical methods and learning strategies based on adult learning practices are used to transfer science-based information to CBOs and health departments, among them: – Face-to-face training – Virtual classrooms – Web-conference technology – Individual on-line instruction

Training Delivery

Institutions, organizations and individuals that partner with CDC to disseminate EBIs:

• STD/HIV Prevention Training Centers (PTCs) • Capacity Building Assistance (CBA) providers • AIDS Education Training Centers (AETCs) • The original researchers • Danya International, Inc. and or other contractual

services

DEBI Training Logistics and Planning

• Activities provided by DEBI contractors (i.e., Danya International) to plan, coordinate, support and monitor training services and outcomes of diffusion of EBIs

DEBI Training Results (26 Interventions)

# of trainings # Participants # Employees CBOs

# Employees HDs

1142 18,531 4,410 1,027

# of trainings # Participants # Employees CBOs

# Employees HDs

169 2,540 664 114

Trainings From 9/1/2010 to 6/30/2011 (8 months)

Trainings From 1/1/2002 to 5/31/2011 (approximately 8 years)

Capacity Building and Technical Assistance

Capacity building assistance includes training, TA, and information and technology transfer

Types: • CDC project officers • CDC science application team • CBA Request Information System (CRIS) ** • Behavioral and Social Science Volunteer Program • Capacity building assistance organizations (CBAs) • PTC trainers may offer phone technical assistance

CRIS – Technical Assistance

The CRIS system is a web-based system that can be accessed 24 hours, 7 days a week.

Key features include: • Monitoring and tracking TA requests • Provides ongoing status reports and a model of

communication among the CBA provider, program consultants, project officers and behavioral experts.

• Allows requests for additional services from other CBA providers

• Assesses the impact and effectiveness of the TA provision

Other related trainings/TEC (Nov 2009 to Jun 2011)

Total TA requests

TA for CBOs TA for EBIs and PHS

EBI/PHS implementation

General evaluation

515 337 (65%) 166 40

44

Total TEC trainings

EBI/PHS support courses

Courses on implementation

of EBI/PHS

EBI/PHS M&E

General evaluation

302 74 (15%) 13 1

15

Technical Assistance (CRIS) and Other Related Trainings Events (TEC)

Technical Assistance (CRIS) for CBOs (Nov 2009 to Jun 2011)

Fidelity Tools

• Fidelity tools are usually created by TA experts to assess proper implementation of an intervention in multiple/different conditions

• Fidelity tools are an integral part of any needs assessment process (what is needed to maximize efficiency during implementation)

• Issues of adaptation are identified early in the implementation process

Example 1: Safety Counts Fidelity Tool - NCHP

Example 2: SHIELD Fidelity Tool - JSI

Example 3: Community PROMISE

These inputs are ‘required’ to implement an EBI!

• Organization (is the agency ready for the intervention?)

• Governance (proper organizational support for the intervention)

• Programmatic infrastructure (who is doing what in the intervention?)

• Workforce and professional development • Resources and support

Fidelity Tools Assessment Areas

Evaluation Field Guides for CBOs & HDs

STAGE 1 (MACRO) STAGE 2 (JSI) STAGE 3 (PEB)

1. SISTA 13. WILLOW 21. SHIELD

2. RAPP 14. START 22. CLEAR

3. VOICES 15. CONNECT

4. HEALTHY RELATIONSHPS 16. SIHLE

5. PROMISE 17. CRCS

6. POL 18. MIP (English & Spanish)

7. MPOWERMENT 19. PfH

8. 3MV 20. PARTNER SERVICES

9. FOCUS ON YOUTH

10. STREET SMART

11. SAFETY COUNTS

12. SISTA TO SISTA

Evaluation: Specialized Capacity Building Tools

Results/Conclusions

• Capacity building and technical assistance experts (CDC and CBA/PTC providers) develop capacity building tools and other fidelity tools to assess the proper implementation of interventions

• Provision of CBA during dissemination and implementation of EBIs is fundamental to ensure optimal levels of fidelity and outcomes

Co-authors

• Juli Powers, MPH. JSI Research and Training Institute, Inc.

• Ricky Tavares, MPH. National Community Health Partners

• Naima Cozier, MSPH. JSI Research and Training Institute, Inc.

• Charles Collins, Ph.D. NCHHSTP/CBB

Acknowledgements

• Rashad Burgess, MPH. CDC/CBB • Aisha Gilliam, Ed.D. CDC/CBB • Ashley Gamayo, MPH. CDC/CBB (ASPH Fellow) • JSI Research & Training Institute, Inc. • National Community Health Partners (NCHP)

Thank you!

For more information please contact Centers for Disease Control and Prevention 1600 Clifton Road NE, Atlanta, GA 30333 Telephone: 1-800-CDC-INFO (232-4636)/TTY: 1-888-232-6348 E-mail: cdcinfo@cdc.gov Web: http://www.cdc.gov The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention Division of HIV/AIDS Prevention