the national center on response to intervention and implementation science: building capacity for...

National Center on Response to

Intervention

The National Center on Response to Intervention and Implementation Science

“Building Capacity for Equity and Excellence for All Students”

Tessie Rose, PhD


Intervention

Agenda

What is Response to Intervention (RTI)? What is the National Center on Response to

Intervention (NCRTI)? Implementation Science

Implementation DriversStages of Implementation Capacity Building


Intervention

What is Response to Intervention? An instructional framework aimed at

improving the skills of ALL students RTI is “preventative” and provides immediate

support to students who are at risk for academic failure

Two primary goals: prevent future academic problems assist in identifying students with SLD

4

Primary Intervention (~80%)School-/Classroom-wide Systems for All Students,

Staff and Settings

Secondary Intervention (~15%)Specialized Group

Systems for Students with At Risk Performance

Tertiary Intervention (~5%)Specialized Individualized

Systems for Students with Intensive Needs

~80% of Students

~15%

~5%

Continuum of School-wide Support

Adapted from”What is School-Wide PBS?”


Intervention

Essential Features of RTI

Core Curriculum Universal Screening Tiered-Interventions Progress Monitoring Data-Based Decision Making Learning Disability Determination

1. Core Curriculum

Research-based curriculum delivered to all students

Differentiated curriculum with evidence-based instructional methods

High-quality instruction in the general education classroom setting

Delivered with fidelity

~80% of Students

~5%

~15%


Intervention

2. Universal Screening

PURPOSE: Identify students who are at risk of academic failure

Conducted with all or targeted groups of students

Administered more than 1x per year Fall, Winter, Spring

Involves brief assessments that are valid, reliable, and evidence-based

NCRTI Screening Tools Chart

www.rti4success.org

3. Tiered-Interventions 3+ levels of increasingly

intense scientific, research-based interventions

Intensity addressed through duration, frequency and time of interventions, group size, and instructor skill level

Individual problem solving model or standardized intervention protocol for intervention levels

Delivered with fidelity

~80% of Students

~5%

~15%


Intervention

4. Progress Monitoring

Repeated measurement of academic performance

Conducted at least monthly to estimate rates of improvement, students who are not demonstrating adequate

progress and/or compare the efficacy of different forms of instruction

to design more effective, individualized instruction. Technically adequate measures administered with

fidelity (accurate data)

0

10

20

30

40

50

60

70

80

90

Wor

ds C

orre

ct P

er M

inut

eSample Progress Monitoring Chart

Words Correct

Aim Line

Linear (Words Correct)

Progress Monitoring Tools Chart


Intervention

5. Data-Based Decision making

Conducted at all levels of implementation (e.g., district, school, grade/content)

Explicit decision rules for assessing learners’ progress (e.g., level and/or rate)Based on evidence based criteria

Follows established routines and procedures


Intervention

6. Learning Disability Determination

Component of a comprehensive evaluationConsensus that dual discrepancy is the best

way to operationalize “responsiveness/non-responsiveness”. Significantly below grade level peersInadequate rate or growth of performance


Intervention

What does RTI look like?Who decides?

USDOE does not endorse a specific RTI model/ framework

National Center on RTI promotes essential components

Washington provides guidance for LEAs Using Response to Intervention (RTI) for Washington’s Students (2006)

Some LEAs are implementing RTI ahead of their SEA

THE NATIONAL CENTER ON RESPONSE TO INTERVENTION

5-year Technical Assistance Center

Funded through a cooperative agreement to American Institutes for Research from OSEP

Part of OSEP’s National Technical Assistance and Dissemination Network

About NCRTI


Intervention

RTI Center Partners

American Institutes for ResearchMaurice McInerney and Nancy Safer, Co-Principal

InvestigatorsMaurice McInerney and Darren Woodruff, Co-Project

DirectorsAmy Elledge, Deputy Project Director

Vanderbilt University ResearchersLynn Fuchs, Doug Fuchs

University of Kansas Center for Research on LearningDon Deshler, Daryl Mellard


Intervention

Center Definition of RTI

Response to intervention integrates student assessment and intervention within a multi-level prevention system to maximize student achievement and reduce behavior issues. With RTI, schools identify students at risk for poor learning outcomes, monitor their progress, provide evidence-based interventions and adjust the intensity and nature of those interventions based on a student’s responsiveness, and identify students with learning disabilities or other disabilities


Intervention

Our Mission

To build state capacity and support for implementing RTI in local districts and schools by serving as a central source of knowledge, expertise, and research-based information for educators, administrators, and parents.


Intervention

Achieving the Mission…

Strategy 1 – identify and evaluate RTI components for identifying and serving students with or at risk for a learning disability identification

Strategy 2 – provide ongoing technical assistance to states and to support the implementation of RTI in classrooms, schools, and local districts nationally

Strategy 3 – disseminate information about proven and promising RTI models to interested stakeholders across the country


Intervention

Strategic Activities

Knowledge Production

Technical Assistance Support

Information Dissemination


Intervention

Strategy 1:Knowledge Production

Technical Review Committees (TRCs): Identify tools and interventions that are

grounded in a rigorous scientific evidence base

Three TRCs(1) Screening

(2) Progress Monitoring

(3) Tiered Instruction Winter 2009


Intervention

Strategy 1: Knowledge Production

RTI WorkgroupsSEA ImplementationELL and RTIDisproportionalityLearning Disability Identification Secondary RTI (High and Middle School)

State Performance Plan (SPP) AnalysesFocusing on Indicators 9 & 10


Intervention

Strategy 2: Technical Assistance Support

In personDirect Technical Assistance

At a distanceTA Resource DevelopmentSharing Communities and WebinarsTraining Modules


Intervention

Strategy 2: Technical Assistance Support

Available to all SEAs at varying levels of intensity

May include a variety of support activities, such as but not limited to:Development of state resources (e.g., guidance

documents, readiness checklists)Training and coaching Development of implementation plans


Intervention

Strategy 3: Information Dissemination

Sharing information through a variety of methodsThe NCRTI website – www.rti4success.org Monthly newsletters – the RTI ResponderEmail blasts about new resources or productsQ&A through email inbox and toll-free number

http://www.rti4success.org/

http://twitter.com/home


Intervention

NCRTI State Database

Database contains RTI information and resources from states, such as:• Policies• Guidance

documents• Briefs• Handbooks• Presentations• Tools• Checklists

Monthly Webinars and Podcasts

Training Modules

IMPLEMENTATION SCIENCE


Intervention

“To implement – is easier said than done.”


Intervention

What is Implementation?

Greenhalgh et al. (2005) active and planned efforts to mainstream an

innovationOther definitions of the verb “to implement” are: introduce and put new ideas into use, establish and use a method in practice, realize, apply or put plans, ideas, models,

norms or policies into operation.Greenhalgh T, Robert G, Bate P, Macfarlane F, & Kyriakidou O. (2005) Diffusion of innovations in health service organisations. A systematic

literature review. Oxford: BMJ Books, Blackwell Publishing.See Guldbrandsson, 2008


Intervention

What is Implementation?

Fixsen et al. (2005) a specified set of activities designed to put

into practice an activity or program of known dimensions

Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. (2005). Implementation research: A synthesis of the literature. Tampa, Florida: University of South Florida, Louise de la Parte. Florida Mental Health Institute, The National Implementation Research Network.


Intervention

Implementation

Intervention Activities

Intervention Outcomes

Implementation Activities

Implementation Outcomes

Two Sets of Activities

Two Sets of Outcomes


Intervention

Why is implementation important?IMPLEMENTATION

EFFECTIVE NOT EFFECTIVE

INTERVENTIONEFFECTIVE

Improved Consumer Outcomes

Poor Outcomes

NOT EFFECTIVE Variable, often ineffective;

sometimes harmful

As cited in Fixsen, 2008 (Institute of Medicine, 2000; 2001; New Freedom Commission of Mental Health, 2003; National Commission of Excellence in Education, 1983; Department of Health and Human Services, 1999)


Intervention

Evidence: What Doesn’t Work?

• Dissemination alone• Training/ professional

development alone

• Laws and policies alone• Special funding alone

Most Common Approaches Used

to Support Implementation!

(see Ager & O´May, 2001; National Implementation Research Network, n.d.; Paul Nutt, 2002; Rogers, Wellins, & Conner, 2002).


Intervention

Evidence: What Works?

• Combination of several implementation measures leads to better results– e.g., distributing guidelines for innovations,

offering education, practical training, coaching, feedback and consultation.

• Quality of support is more important than quantity

(See Guldbrandsson, 2008)


Intervention

Implementation Support

Letting it Happen

Helping it Happen

Making it Happen

Benefits to Consumers

Effective Implementation

(Greenhalgh et al., 2005)


Intervention

Common Perceived Barriers

Scheduling (time)State/district/school policiesStaff attitudes/lack of knowledgeFunding“The system”

Is this really true? Can these things really not be changed?

Evidence: What Doesn’t Work?• Attempt to fit innovation into existing service

delivery system• OUTCOMES:– Rarely fully implemented in a reasonable time– Often viewed as incompatible – Eventually disappears

Service Delivery System

Innovation: Component #1






Intervention

Example: What Doesn’t Work?


Core Curriculum

Universal Screening

Progress Monitoring

Tiered Interventions (some)

Data Based Decision Making


Intervention

Evidence: What Works?

• Evaluate current infrastructure and identify and address potential barriers to implementation

• Structure technical assistance and service delivery system to support innovation


Changed to Fit Innovation







Intervention

Example: What Works?


• Changes in PD• Schedule in teaming

time• Establish

assessment/ intervention times in schedule• Restructure priorities

Core Curriculum

Universal Screening

Progress Monitoring

Tiered Interventions

Data Based Decision Making


Intervention

Supporting Implementation

Identify potential barriers prior to implementation

Prepare for potential barriersBuild capacity of implementation teams to

identify and address issues immediately

The biggest mistake is to ignore or hope they will resolve themselves.

Something to Think About: Implementation Matters

IMPLEMENTATION

EFFECTIVE NOT EFFECTIVE

INNOVATION EFFECTIVE80% Full

Implementation3.6 Years

14% Full Implementation

17 Years

NOT EFFECTIVE

(Fixsen, 2008)


Intervention

Things to Remember

75 - 85% of LEA/SEA policies support innovations Remaining need to be changed

Benefit will be seen when 60% are in full implementation

Things don’t always work at firstKeep the entire system in mind – even if you

are only responsible for training in one area


Intervention

System AlignmentFederal

TA /PD System

State

Local Education Agencies

Teachers/ Staff

Effective Practices(Fixsen, 2008)


Intervention

What are Implementation Drivers?

Methods to develop, improve, and sustain competent use of innovations

Methods to create and sustain effective organizational and systems environments for effective services

(National Implementation Research Network)


Intervention

Implementation Drivers

Staff Selection Preservice and Inservice Training Consultation and Coaching Staff and Program Evaluation Program Evaluation Facilitative Administrative Support

Systems Interventions

Developing Competency

Creating Supportive

Systems and Environments


Implementation Components or Drivers

National Implementation Research Network


Intervention

Driver 1: Staff Recruitment & Selection

Selection may be a key ingredient of implementation at every level: selection of practitioners, selection of organization staff (trainers,

coaches, evaluators, administrators), and selection of staff for purveyor groups.



Intervention

Sample Staff Selection Questions

Who is qualified to carry out the evidence-based practice or program that a program wants to implement?

Beyond academic qualifications or experience factors, what practitioner characteristics are essential for carrying out the evidence-based practice “on the ground?” What characteristics or abilities will not or cannot be addressed through training and coaching?

Do organizational staff members have a comprehensive understanding of the practices being implemented?

Are organizational staff members prepared to support practitioners in carrying out the evidence-based practices that are slated to be implemented?



Intervention

Driver 2: Pre-service or in-service training

Training appears to be a core implementation component for practitioners, agency staff, and purveyor staff.

Includes activities related to providing Specialized informationInstructionSkill development

Most skills can be introduced in training but really are learned on the job with the help of a consultant/coach (Driver 3)


Intervention

Research indicates that effective training involves:

Providing practitioners with the background information, theory, philosophy, and values of the new program or practice

Introducing and demonstrating the components and rationales of key practices;

Providing opportunities to practice specific skills related to the new way of work and receive feedback in a safe training environment

Providing staff with opportunities for quality interaction


Intervention

Driver 3: Consultation and Coaching

Coaching and mentoring include activities for either individuals or groups, on-the-job observation, instruction, modeling, feedback, or debriefing of practitioners and other key staff in the program.

Avoid “train and hope”Ensure coaching and mentoring are included as

part of the training processGOAL: Behavior change


Intervention

Driver 4: Staff Evaluation

Assesses use and outcome of essential skillsComponent of overall program evaluation

(Driver 5)OUTCOMES:

Assess progress of implementation efforts for building staff competency

Assess usefulness of training and coaching


Intervention

Driver 5: Program Evaluation

Ensure data system supports decision makingAssess key aspects of overall performance

Numerous UsesImprove quality professional development, training,

and coaching Assess fidelity of implementation of interventionAllocate resources based on identified needsIdentify and address innovations or barriers to

implementationAssess cost efficiency


Intervention

Driver 6: Facilitative Administrative Supports

Proactive, vigorous and enthusiastic attention by the administration to…Reduce implementation barriers Create an administratively hospitable environment for

practitioners. Facilitative administration includes…

Internal policy analyses and decisionsProcedural changesFunding allocationsCulture focused on what it takes to implement with

fidelity


Intervention

Integrated and Compensatory

Behavior change must occur among professionals AND within the system

An infrastructure (e.g., implementation drivers) is needed to support and sustain such changes

These implementation components (drivers) must be integrated and can be compensatoryinternal consistency among selection variables,

skills training, coaching, staff evaluation, etc.


Intervention

Implementation Drivers: In a Nut ShellLe

ader

ship

Staff Competency

Organization

Improved Outcomes

Integrated and Compensatory


Intervention

Stages of Implementation

Exploration and Adoption Program Installation Initial Implementation Full Implementation Innovation Sustainability

2-4 Years


Intervention

Stage 1: Exploration and Adoption

Assess match between innovation and needsCommon components

identify the need considering the information available acquire information via interactions with one another assess fit between the intervention and needs prepare the organization, staff, and resources by

mobilizing information and support (e.g., social marketing)

Concludes with decision to move toward implementation



Intervention

Stage 2: Program Installation

Active preparation prior to implementationAcquiring Required Resources and MaterialsStaff SelectionStructural Supports (e.g., policies, funding

Includes all start-up costs



Intervention

Stage 3: Initial Implementation

Initial attempt to implement the innovation (e.g., smaller scale, pilots)

Opportunity to re-assess potential barriers and identify additional needs (e.g., training, coaching) for full implementation

WARNING: Implementation could end here unintentionally!



Intervention

Stage 4: Full Implementation

Considered “business as usual” or “accepted practice”

Occurs when 60% of those who would benefit have full and effective accessNot just those who want the intervention

NOTE: Evaluation of student outcomes can only begin once full implementation is reached.



Intervention

Stage 5: Innovation

Occurs only after innovation has been implemented with fidelity

Opportunity to refine and expand both the innovation and the implementation practices and programs



Intervention

Stage 6: Sustainability

OUTCOMES:Long-term survival of innovationContinued effectiveness within changing world



Intervention

Stages of Implementation: In a Nut Shell

Decide you want to do ‘it’ Plan for ‘it’ Try ‘it’ out Just do ‘it’ Improve ‘it’ Sustain ‘it’


Intervention

NCRTI Capacity Building

According to Fullan (2001), “effective approaches to managing change call for combining and balancing factors that do not apparently go together--(such as) fidelity and adaptivity. More than anything else, effective strategies require an understanding of the process (of change), a way of thinking that cannot be captured in any list of steps to be followed” (p. 71).


Intervention

NCRTI Capacity Building Domains

Vision Leadership Needs Assessment Training and Outreach Evaluation


Intervention

Vision

Identify the state’s vision for RTI. The dialogue includes discussions about (a) state’s goal for RTI, (b) roles and responsibilities for different agencies in meeting the goal, and (c) timeline for implementing its RTI initiative on a multi-district or statewide level.

1. Develop overall vision and goals/objectives for RTI implementation

2. Develop vision for initial implementation3. Develop vision for full implementation4. Develop procedures for revising and

communicating vision


Intervention

Leadership

Establish RTI leadership or implementation teams. The dialogue includes discussions about membership, roles and responsibilities, and implementation plans.

1. Identify members, establish roles and responsibilities and establish administrative procedures

2. Identify potential barriers and ensure infrastructures are in place

3. Develop and implement plan (s) (PD, evaluation, ect.) for initial implementation

4. Develop and implement plan (s) for full implementation


Intervention

Needs Assessment

Conduct ongoing needs assessment (data collection). The dialogue includes discussions about assessing needs of the infrastructure, data system, students and teachers, and other key stakeholders.

1. Assess SEA/LEA need for RTI2. Assess infrastructure3. Conduct ongoing needs assessment for initial

implementation4. Conduct ongoing needs assessment for full

implementation


Intervention

Outreach and Training

Provide outreach and training. The dialogue includes discussions about broad outreach and communication about RTI, recruiting and training coaches, and scheduling and implementing a series of coordinated training events (including workshops and follow-up activities) to support LEAs 1. Engage in RTI social marketing activities 2. Build competency of leadership and implementation teams 3. Implement training and coaching plan for initial

implementation4. Implement training and coaching plan for full

implementation


Intervention

Evaluation

Conduct ongoing evaluation. The dialogue includes discussions about how to measure the efficiency with which the is implementing as well as the effectiveness.1. Evaluate the need and ability to implement RTI2. Evaluate infrastructure and develop comprehensive

data system3. Evaluate efficacy and efficiency of initial

implementation4. Evaluate efficacy and efficiency of full

implementation and evaluate RTI effectiveness


Intervention

Capacity Building Dialogue Guide

PURPOSE: Guide dialogue discussions with LEAs and SEAs

OUTCOMES:Identify strengths and weaknessesDevelop an action plan Assess progress for implementation of RTI

Questions or Comments?


Intervention