Barbara SimsBrenda Melcher

Dean FixsenKaren Blase

Michelle Duda

Washington, D.C.

July 2013

Keep Dancing After the Music Stops

OSEP Project Directors’

Conference

Keep Dancing After the Music Stops

WHAT:Effective

Interventions

HOW, WHO, WHEN:

EffectiveImplementation

WHERE:EnablingContexts

WHY:Socially

Significant Outcomes

Formula For Success

WHO

Teams

WHEN

Stages

HOW

Drivers

HOW

Cycles

WHAT

UsableInterventions

Active Implementation FrameworksActive Implentation frameworks

USABLE INTERVENTIONS

An intervention needs to be teachable, learnable, doable, and be readily assessed in practice.

UsableInterventions

Usable Interventions

Operational Definitions

ClearDescription

PerformanceAssessment

EssentialFunctions

Usable Interventions

Usable InterventionsTools You Can UseHexagon Tool Practice Profiles

UsableInterventions

The HexagonAn EBP Exploration Tool

NEED

FIT

RESOURCES

EVIDENCE

CAPACITY

READINESS

Fit with current Initiatives• School, district , state priorities• Organizational structures

Community values

Need in school, district, state• Academic & socially significant Issues• Parent & community perceptions of need• Data indicating need

Resources and supports for:• Curricula & Classroom• Technology supports (IT dept.)• Staffing• Training• Data Systems• Coaching & Supervision• Administration & system

Evidence• Outcomes – Is it worth it?• Fidelity data• Cost – effectiveness data• Number of studies• Population similarities• Diverse cultural groups• Efficacy or Effectiveness

Capacity to Implement• Staff meet minimum qualifications• Able to sustain Imp Drivers

• Financially • Structurally

• Buy-in process operationalized• Practitioners • Families

Readiness for Replication• Qualified purveyor• Expert or TA available• Mature sites to observe• Several replications• How well is it operationalized?• Are Imp Drivers operationalized?

The “Hexagon” can be used as a planning tool to evaluate evidence-based programs and practices during the Exploration Stage of Implementation.

Download available at:http://implementation.fpg.unc.edu/resources/hexagon-tool-exploring-context

EBP:

5 Point Rating Scale:High = 5; Medium = 3; Low = 1.Midpoints can be used and scored as a 2 or 4.

High Med Low

Need

Fit

Resource Availability

Evidence

Readiness for Replication

Capacity to Implement

Total Score

© 2009 Karen Blase, Laurel Kiser, & Melissa Van Dyke Adapted from work by Laurel J. Kiser, Michelle Zabel, Albert A. Zachik, and Joan Smith at the University of Maryland

An EBP Exploration Tool

Practice Profile Tool

How: Effective Implementation

• Changing the behavior of educators and administrators

• Creating the setting conditions to facilitate these changes

• Creating the processes to maintain and improve these changes in both setting conditions and behavior of well-intentioned adults

• So that students benefit

IMPLEMENTATION DRIVERS

Key components of capacity and infrastructure that influence a program’s success.

Drivers

Performance Assessment(Fidelity)

Coaching

Training

Selection

Systems Intervention

Facilitative Administration

Decision Support Data System

Com

pete

ncy

Driv

ers

Com

pete

ncy

Driv

ers

Organization D

rivers

Organization D

rivers

Implementation Drivers

© Fixsen & Blase, 2008Leadership Drivers

Characteristic Practice Mean Effect Size

Introduction Out-of-class learner activities/self-instructionClassroom/workshop presentationsPre-class learner exercises

0.640.630.54

Illustration Trainer role playing/simulationsLearner informed input

0.550.53

Practicing Real-life learner applicationReal-life learner application/role-playing

0.940.86

Evaluation Self assessment of strengths/weaknesses 0.94

Reflection Identify performance-improvement goalsJournaling/behavior suggestions

1.270.82

Mastery Standards-based assessment 0.86

Dunst, C.J., Trivette, C.M., & Hamby, D.W. (2010). Meta-analysis of the effectiveness of four adult learning methods and strategies. International Journal of Continuing Education and Lifelong Learning, 3(1), 91-112.

Research Synthesis of Adult Learning StudiesResearch Synthesis of Adult Learning studies

Training and CoachingImplementation Drivers

 OUTCOMES% of Participants who Demonstrate Knowledge, Demonstrate

New Skills in a Training Setting, and Use new Skills in the Classroom

TRAINING

COMPONENTS

Knowledge SkillDemonstration

Use in theClassroom

Theory and Discussion 10% 5% 0%

..+Demonstration in Training 30% 20% 0%

…+ Practice & Feedback in Training 60% 60% 5%

…+ Coaching in Classroom

95% 95% 95%

—Joyce and Showers, 2002

Intervention Training

• Develop Intervention Training Plan

Identify initial trainers

» Qualifications for trainers

If outside experts are used you need to assess

» Innovation Fluency

» Use of Training Best Practices

» Data from people they have ‘trained’

Intervention Training Plan

• Aspects requiring new knowledge Description

Purpose

Supporting data

Examples

Critical components

Intervention Training Plan

• Aspects requiring new skills

Key behaviors

Behavior rehearsals

Identify and prepare behavior rehearsal leaders and confederates

Intervention Training Plan

• Establish Coaching System Selection Criteria Policy/Procedures Accountability structure Feedback loops

Coaching

Training

Selection

Systems Intervention

Facilitative Administration

Decision Support Data SystemC

ompe

tenc

y D

river

s

Com

pete

ncy

Driv

ers O

rganization Drivers

Organization D

rivers

LeadershipLeadership

Improved outcomes

Interventions meet

Implementation

Consistent Use of

Programs

© Fixsen & Blase, 2008

PerformanceAssessment

(Fidelity)

Implementation DriversTools You Can UseTraining Plan TemplateCoaching System Worksheet

Drivers

Performance Assessment(Fidelity)

Coaching

Training

Selection

Systems Intervention

Facilitative Administration

Decision Support Data System

Com

pete

ncy

Driv

ers

Com

pete

ncy

Driv

ers

Organization D

rivers

Organization D

rivers

Implementation Drivers

© Fixsen & Blase, 2008Leadership Drivers

Practice-Policy Communication Cycle

Policy

Practice

Po

licy En

ables P

ractices

Plan

DoEx

tern

al I

mp

lem

enta

tio

n S

up

po

rt Policy

Practice

Structure

Procedure

Pra

ctic

e In

form

s P

oli

cy

Fee

db

ack

Stu

dy - A

ct

FORM SUPPORTS FUNCTION

Practice-Policy Communication Cycle

Children & Families

Children & Families

Building Teachers and Staff

Building Teachers and Staff

Building Leadership and Implementation

Team

Building Leadership and Implementation

Team

District Leadership and Implementation

Teams

District Leadership and Implementation

Teams

Regional Implementation

Teams

Regional Implementation

Teams

State Transformation

Team

State Transformation

Team

StateDepartment of

Education Leadership

StateDepartment of

Education Leadership

Polic

y Ena

bled

Pract

ice

(PEP)

Pract

ice

Info

rmed

Polic

y (P

IP)

Practice/Policy Communication

Loops

Practice/Policy Communication Loops

Communication CyclesTools You Can UseCommunication Protocol Worksheet

Drivers

Free 24/7 Implementation Training for You and Your Team

SISEP’s newly launched “Active Implementation Hub” is a free, online learning environment for use by any stakeholder — practitioners, educators, coaches, trainers, purveyors — involved in active implementation and scaling up of programs and innovations. The site goal is to increase the knowledge and improve the performance of persons engaged in actively implementing any program or practice.”

http://implementation.fpg.unc.edu

The AI Hub is an initiative of the State Implementation & Scaling-up of Evidence-based Practices Center (SISEP) and The National Implementation Research Network (NIRN) located at The University of North Carolina at Chapel Hill’s FPG Child Development Institute.

Free Training!

Get Connected!

www.scalingup.org

SISEP @SISEPcenter

For more on Implementation Sciencehttp://nirn.fpg.unc.edu

www.implementationconference.org