ensuring rti’s successful implementation

Dean L. Fixsen, Karen A. Blase,

Sandra F. Naoom, Melissa Van Dyke,

Frances WallaceNational Implementation Research Network

Louis de la Parte Florida Mental Health Institute

Ensuring RtI’s Successful Implementation

Virginia RtI Meeting 2007

Copyright © Dean L. Fixsen and Karen A. Blase, 2007

Outcomes for Children, Families,

& Communities

A Functional System

Bureaucracy

Practitioners

Policies

Agencies


Evidence-Based Movement

The “evidence-based movement” is an international experiment to make better use of research findings in typical service settings.

The purpose is to produce greater benefits to students and society.


Education

65 million kids

6 million teachers and staff

140,000 schools

3,143 counties

60 states & U.S. jurisdictions


Science to Service

SCIENCE SERVICEGAPIMPLEMENTATION


Science to ServiceScience to Service Gap

What is known is not what is adopted to help students, families, and communities

Implementation Gap What is adopted is not used with fidelity

and good outcomes for consumers.

What is used with fidelity is not sustained for a useful period of time.

What is sustained is not used on a scale sufficient to impact societal outcomes.


Making use of Science

Letting it happen

Recipients are accountable

Helping it happen

Recipients are accountable

Making it happen

Implementation teams are accountable

Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004


0

100

200

300

400

500

600

700

800

900

≤1972 1974 1976 1978 1980 1982

0

50

100

150

200

250

300

CumulativeCouples

CumulativeHomes

Fixsen, Blase, Timbers, & Wolf (2001)

Teaching–Family Replications


Follow Through Programs

Figure 1: This figure shows the average effects of nine Follow Through models on measures of basic skills (word knowledge, spelling, language, and math computation), cognitive-conceptual skills (reading comprehension, math concepts, and math problem solving) and self-concept. This figure is

adapted from Engelmann, S. and Carnine, D. (1982), Theory of Instruction: Principles and applications. New York: Irvington Press.


School Wide PBS

SWPBS Schools

0

1000

2000

3000

4000

5000

6000

7000

Aug 04 Aug 05 Aug 06 Aug 07

Year

Tota

l Num

ber

Sch

ools

4.2% of all schools


Implementation Reviews

Human service prevention and treatment programs (e.g. education, substance abuse, adult / children’s MH, justice, health)

Advanced manufacturing technologies

AMA clinical guidelines

Engineering: bridge maintenance

Hotel service management

National franchise operations

Cancer prevention & treatment


Ineffective MethodsExcellent experimental evidence for

what does not work

Diffusion/dissemination of information by itself does not lead to successful implementation (research literature, mailings, promulgation of practice guidelines)

Training alone, no matter how well done, does not lead to successful implementation


Ineffective MethodsExcellent evidence for what does

not work

Implementation by edict by itself does not work

Implementation by “following the money” by itself does not work

Implementation without changing supporting roles and functions does not work Paul Nutt (2002). Why Decisions Fail


What Works

Effective intervention practices

+

Effective implementation practices

=

Good outcomes for consumers


Implement Innovations

Effective NOT Effective

Effective

NOT Effective

IMPLEMENTATION

INT

ER

VE

NT

ION Student Benefits

PLACEBO: Something of no intrinsic remedial value that is used to appease or reassure another

Placebo


Implementation

An effective intervention is one thing

Implementation of an effective intervention is a very

different thing


EBPs & Implementation

From an implementation perspective, what do we need to know about innovations such as evidence-based programs?



The usability of a program has little to do with the quality or weight of the evidence regarding that program

Evidence on intervention effectiveness for specific populations helps us choose what to implement

Evidence on the effectiveness of the intervention does not help implement the program or practice successfully



Core intervention components

Clearly described (who/what)

Practical measure of fidelity

Fully operationalized (do/say)

Field tested (recursive revision)

Contextualized (org./systems fit)

Effective (worth the effort)


Implementation

What do we need to know about successful implementation methods?


Stages of Implementation

Implementation is not an event

A mission-oriented process involving multiple decisions, actions, and corrections



Exploration

Installation

Initial Implementation

Full Implementation

Innovation

Sustainability

Implementation occurs in stages:

Fixsen, Naoom, Blase, Friedman, & Wallace, 2005

2 – 4 Years



Exploration

Installation


Full Implementation

Innovation

Sustainability



Intervention Outcomes

0% 100%

Implementation Outcomes


INTEGRATED & COMPENSATORY

CONSULTATION & COACHING


STAFF PERFORMANCE EVALUATION


FACILITATIVE ADMINISTRATIVE

SUPPORTS


SUPPORTS

RECRUITMENT AND SELECTION


PRESERVICE TRAINING

PRESERVICE TRAINING

SYSTEMSINTERVENTIONS


Implementation Drivers

DECISION SUPPORT DATA SYSTEMS


OUTCOMES(% of Participants who Demonstrate Knowledge, Demonstrate

new Skills in a Training Setting, and Use new Skills in the Classroom)

TRAININGCOMPONENTS

KnowledgeSkill

DemonstrationUse in the Classroom

Theory and Discussion

10%

5% 0%

..+Demonstration in Training

30%20%

0%

…+ Practice & Feedback in Training

60% 60% 5%

…+ Coaching in Classroom

95% 95% 95%

Joyce and Showers, 2002

Integrated & Compensatory


Who does the work?

Implementation Teams

Develop effective, flexible, adaptable capacity to initiate and manage continual change

Requires new roles, functions, and skill sets that do not exist currently


Implementation Team

A group that knows the innovation very well (formal and practice knowledge)

A group that knows how to implement that innovation with fidelity and good effect

A group that accumulates data & experiential knowledge -- more effective and efficient over time (information economics, K. Arrow)


Implementation Team

Policy members (change policy, barrier busters, facilitators)

Practice members (do the innovation, test policies, feedback)

Families and stakeholders

Management members (roles and functions)

Daily / Weekly / Monthly Meetings (urgent, unfiltered, goal focused)


Implementation Team

School

Management (leadership, policy)

Administration (HR, structure)

Supervision (nature, content)

Teacher

State and Community Context

District

Imp

lem

enta

tio

n T

eam

Simultaneous, Multi-Level Interventions


Implementation TeamChange the behavior of adult education professionals

“Systems don’t change, people do” (J.W.)

Change organizational structures, cultures, and climates

Change the thinking of system directors and policy makers

Successful and sustainable implementation of evidence-based programs always requires organization and systems change.


RtI


What is RtI??

Core intervention components

Clearly described (who/what)

Practical measure of fidelity

Fully operationalized (do/say)

Field tested (recursive revision)

Contextualized (org./systems fit)

Effective (worth the effort)

Response to Intervention: PhilosophyCore Principles (NASDSE, 2006)

We can effectively teach all children

We need to identify the curricular, instructional and environmental conditions for learning.

Intervene early

Solving small problems early is both more efficient and more successful.

Use a multi-tier model of service delivery

Needs-driven, resourced deployment systems to match instructional resources with student need.

Dona Meinders, Silvia DeRuvo; WestEd, California Comprehensive Center

RtI Definition: Components

RtI is the practice of providing:

high-quality instruction and intervention

matched to student need,

monitoring progress frequently to

make decisions about change in instruction or goals and

applying child response data to important educational decisions. (NASDSE, 2005)


RtI: Components (NASDSE, 2006)

Monitor student progress to inform instruction

Use assessment to collect information on how student is progressing

Use data to make decisions

Ongoing data collection systems in place and used to make informed decisions

Use assessment for three different purposes

Screening

Diagnostics

Progress monitoring



Response to Intervention

Shifts accountability for outcomes from the children to the teachers and their supporting schools & education systems



Exploration

Installation


Full Implementation

Innovation

Sustainability



2 – 4 Years


RtI: Oregon Group

District level entry point; system change focus

Information, what it is, is not

Stakeholder buy-in,

Application/ selection process

Informed agreement, understand and defend the initiative

The Tigard-Tualatin District; Carol Sadler et al


RtI: Oregon Group

District level system change

Give lots of rationales

Not a project, not patchwork

Focus on RtI functions

Establish a common vocabulary to ease communication

Build on what folks are doing already – help them get ready for change


RtI: Oregon Group

Demand far exceeds capacity

Select the willing

Be overly strict at the start

Work with others to help them become “RtI ready”

Develop capacity: Yr 3 sites now helping with newbies


INTEGRATED & COMPENSATORY






SUPPORTS


SUPPORTS



PRESERVICE TRAINING

PRESERVICE TRAINING



Implementation Drivers




RtI: Oregon Group

Guided development

Leadership involvement (will require more in the future)

Year of training with on-going coaching (will require more on-site visits in the future)

Include leaders in the training (well informed, able to explain and defend, willing to do what is required)


RtI: Oregon Group

Infrastructure development

DOs need information to guide the changes

Top down and bottom up approach

Never done, always changing, 3-ring binders and updated websites


Systems Change

Organizational changes (schools and districts)

System changes (state and federal)


RtI: Oregon Group

Infrastructure development

Designate funds to support implementation efforts right from the start (costs associated with making changes)


RtI: Oregon Group

Issues

Teacher education does not support RtI work (philosophy, values, skills)

Teacher certification may need to change


A Sobering Observation

"All organizations [and systems] are designed, intentionally or unwittingly, to achieve precisely the results they get."

R. Spencer DarlingBusiness Expert


Systems Change

New practices do not fare well in old organizational structures and systems

Develop new position descriptions and job functions in state departments and districts focused on implementation (effective use) of policies and innovations

Academic Systems Behavioral Systems

1-5% 1-5%

5-10% 5-10%

Intensive, Individual Interventions•Individual Students•Assessment-based•High Intensity•Of longer duration

Intensive, Individual Interventions•Individual Students•Assessment-based•Intense, durable procedures

Targeted Group Interventions•Some students (at-risk)•High efficiency•Rapid response


80-90% 80-90%Universal Interventions•All students•Preventive, proactive

Universal Interventions•All settings, all students•Preventive, proactive

Multi-tier Model


Academic Systems Behavioral Systems

1-5% 1-5%

5-10% 5-10%

Intensive, Individual Interventions•Individual Students•Assessment-based•High Intensity•Of longer duration

Intensive, Individual Interventions•Individual Students•Assessment-based•Intense, durable procedures



80-90% 80-90%Universal Interventions•All students•Preventive, proactive

Universal Interventions•All settings, all students•Preventive, proactive

Multi-tier Model

Attention, Effort,

Precision


“Making it happen”

Implementation: Active involvement of implementation teams that work at the intersection of practices, programs, systems, communities, & scientists

Implementation teams are accountable for assuring use of innovations with fidelity and good outcomes

Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004


Implementation Team

Implementation Team

Prepare Communities

Prepare schools faculty, staff

Work with Researchers

Assure Implementation

Prepare Districts Assure Student Benefits


Systems Change

Teachers / staff impact students

It is the job of administrators, managers, and funders to align policies and structures to facilitate effective teacher / staff practices

There is no such thing as an “administrative decision” – they are all education decisions


Systems Change

State Department

Districts

Schools

Teachers/ Staff

Effective Practices

AL

IGN

ME

NT

Federal Departments

Imp

lem

enta

tio

n T

eam

s

FORM FOLLOWS FUNCTION


Creating Capacity for Competent Change

New OSEP Center

State Implementation and Scaling up of Evidence-based Practices (SISEP)


Creating Capacity for Competent Change

State Transformation Team that learns complex skills related to creating and sustaining RITs

Regional Implementation Teams that learn complex skills related to system, school, and teacher change

Capacity = knowledge, skills, and experience; self-correcting, self-sustaining knowledge utilization


State Transformation

Team

RegionalImplementation

TeamN = 50 - 200


TeamN = 50 - 200


TeamN = 50 - 200


Outcomes for Children, Families,

& Communities

Functional Education

Bureaucracy

Teachers

Policies

Schools


Thank YouWe thank the following for their support

Annie E. Casey Foundation (EBPs and cultural competence)

William T. Grant Foundation (implementation literature review)

Substance Abuse and Mental Health Services Administration (implementation strategies grants; NREPP reviews; SOC analyses of implementation; national implementation awards)

Centers for Disease Control & Prevention (implementation research contract)

National Institute of Mental Health (research and training grants)

Juvenile Justice and Delinquency Prevention (program development and evaluation grants

Office of Special Education Programs (Capacity Development Center contract)

Agency for Children and Families (Child Welfare Leadership Development contract)


For More Information

Dean L. Fixsen

813-974-4446

[email protected]

Karen A. Blase

813-974-4463

[email protected]

National Implementation Research Network

At the Louis de la Parte Florida Mental Health Institute

University of South Florida

http://nirn.fmhi.usf.edu


For More Information

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).

Download all or part of the monograph at:http://nirn.fmhi.usf.edu/resources/publications/Monograph/index.cfm

Implementation Research: A Synthesis of the Literature

ensuring rti’s successful implementation

Documents