results driven accountability and intensive intervention: using mtss to improve outcomes for...
TRANSCRIPT
Results Driven Accountability and Intensive Intervention: Using MTSS to Improve Outcomes for Students with Disabilities
Sarah Arden, Laura Berry Kuchle, Christopher Lemons, T. Chris Riley-Tillman
This document was produced under U.S. Department of Education, Office of Special Education Programs, Award No. H326Q110005. Celia Rosenquist serves as the project officer. The views expressed herein do not necessarily represent the positions or policies of the U.S. Department of Education. No official endorsement by the U.S. Department of Education of any product, commodity, service or enterprise mentioned in this document is intended or should be inferred.
A note about questions…Please type
questions related to technical issues in
the Chat box.
Please type questions related to webinar content in
the Q&A box.
PanelistsSarah V. Arden, Ph.D. – Researcher and Technical Assistance Liaison at American Institutes for Research
Laura Berry Kuchle, Ph.D. – Researcher and Technical Assistance Liaison at American Institutes for Research
Chris Lemons, Ph.D. – Senior Advisor to the National Center on Intensive Intervention and Assistant Professor in the Department of Special Education at Peabody College of Vanderbilt University
T. Chris Riley-Tillman, Ph.D. – Trainer for the National Center on Intensive Intervention and Professor at the University of Missouri College of Education
Session Overview What is the vision of RDA How is intensive intervention part of RDA Overview of National Center on Systemic Improvement Introduction to DBI Why Do We Need to Assess DBI Implementation? DBI Implementation Rubric Lessons Learned: Strategies for Improving Technical
Assistance and DBI Implementation NCII Resources to Support Implementation
5
Results-Driven Accountability: VisionAll components of an accountability system will be
aligned in a manner that best support states in improving results for infants, toddlers, children, and
youth with disabilities and their families.
Shift from Compliance to Results + ComplianceSlide adapted from: OSEP Slides to Explain Results Driven Accountability (RDA) Retrieved from http://www2.ed.gov/about/offices/list/osers/osep/rda/index.html
6
SSIP
• Conduct root cause analysis (including infrastructure) to identify contributing factors
• For each contributing factor, identify both barriers and leverage points for improvement
• Search/evaluate evidence-based solutions (Exploration Phase)
• Develop action steps (address barriers/use leverage points)
• Develop Theory of Action• Develop Plan for Improvement
(Implementation Framework)
• Initiate Data Analysis• Conduct broad
Infrastructure Analysis• Identify problem area
• Evaluation of progress annually• Adjust plan as needed
How well is the solution
working?What is the problem?
Why is it happening?
What shall we do
about it?
SSIP
SSIP Phase I
SSIP Phase I and II
SSIP Phase III
SSIP Phase I
Slide from: OSEP Slides to Explain Results Driven Accountability (RDA) Retrieved from http://www2.ed.gov/about/offices/list/osers/osep/rda/index.html
7
State Systemic Improvement PlanYear 1—FFY 2013Delivered by April 2015
Year 2—FFY 2014Delivered by Feb. 2016
Years 3–6—FFY 2015–18Feb. 2017–Feb. 2020
Phase I Analysis Phase II Plan Phase III Evaluation Data analysis Infrastructure
analysis State-identified
measureable result Coherent
improvement strategies
Theory of action
Multiyear plan addressing:
• Infrastructure development
• Support early intervening services program and local education agencies in implementing evidence-based practices
• Evaluation plan
Reporting on progress including:
• Results of ongoing evaluation
• Extent of progress Revisions to the State
Performance Plan
Slide from: OSEP Slides to Explain Results Driven Accountability (RDA) Retrieved from http://www2.ed.gov/about/offices/list/osers/osep/rda/index.html
8
State-identified Measurable Result(s) State-identified Measurable Result(s) (SiMR)
• A child-level (or family-level, for Part C) outcome
• Not a process or system result
• May be a single result or a cluster of related results
Identified based on analysis of data
9
On What Are States Focusing?• Graduation: AK, DC, FL, GA, MN, MT, NC, ND, NJ, PA, RMI, VA, WV
• Reading/ELA: AR, AS, AZ, CNMI, CO, CT, DE, FSM, GU, HI, IA, ID, IL, IN, KS, LA, MI, MS, NE, NV, NM, NY, OH, OK, OR, Palau, SC, SD, TN, TX, VI, WA, WI, WY
• Math: KY, MD, ME, PR, RI, UT, VT
• Reading and Math: CA, MO
• Early Childhood Outcomes: MA, NH
• Post-school Outcomes: AL, BIE
Variations: Disability category; race/ethnicity; gender; grades; English learner; poverty status; subset of districts
10
Collaboration between Centers Led by WestEd Partners:
• AIR
• NASDSE
• SRI
• CCSSO
• Parent Center Network
• Meadows Center for Preventing Educational Risk (evaluator)
Learn more at http://ncsi.wested.org/
Ask NCSI: http://ncsi.wested.org/contact/
NCSI Resources: http://ncsi.wested.org/resources/
11
Targeted Outcomes of NCSI1. Increase the capacity of state education agencies (SEAs) and lead
agencies (LAs) to develop, implement, and evaluate their State Systemic Improvement Plans (SSIPs)
2. Increase SEAs’ and LAs’ knowledge, selection, and utilization of evidence-based practices (EBPs)
3. Improve SEA and LA infrastructure and coordination within SEAs and LAs for delivering effective technical assistance (TA) on implementing and scaling-up effective strategies, stakeholder engagement, resource mapping and allocation, and instructional collaboration
12
Targeted Outcomes of NCSI Cont.4. Increase the use of effective dissemination strategies by SEAs
and LAs to ensure that local education agencies (LEAs) and early intervention service (EIS) programs and providers have access to EBPs and can select and implement those EBPs in a sustainable manner
5. Increase the effectiveness of SEAs and LAs to meaningfully engage State and local stakeholders in the development and implementation of the SSIP
13
NCSI Approach to TA
14
Why Is This Important? Meeting SiMR goals will require a focus on improving
instruction. States will be in need of support on how to provide
intensive intervention for the kids who need it the most, including: • Evidence-based intervention strategies
• Overcoming implementation barriers
• Making connections to other state, district, and school initiatives
15
What Does This Mean for the Field? Data Use
• Knowing what data to look at…accessing those data…root cause analysis to make sense of the data
Knowledge Utilization
• Selecting coherent/aligned evidence-based practices…implementing them with fidelity…scaling up and sustaining them
Systems Change
• Infrastructure considerations…resource mapping and alignment…policies that support the goals
Communication & Collaboration
• Problem solving and planning with diverse stakeholders, in the right settings, at the right time…and working together for implementation to achieve results
16
What is Intensive Intervention and Why Do We Need to Assess Fidelity of Implementation?
17
What is Intensive Intervention?Intensive intervention addresses severe and persistent learning or behavior difficulties. Intensive intervention should be Driven by data Characterized by increased intensity (e.g., smaller group,
expanded time) and individualization of academic instruction and/or behavioral supports
18
What Intensive Intervention… Is… Individualized based on
student needs More intense, often with
substantively different content AND pedagogy
Comprised of more frequent and precise progress monitoring
Is Not… A single approach A manual A preset program More of the same Tier 1
instruction More of the same Tier 2
instruction
19
Why Do We Need Intensive Intervention?
Low academic achievement
Dropout rates
Arrest rates
20
What is NCII’s Approach toIntensive Intervention?Data-Based Individualization (DBI): A systematic method for using data to determine when and how to provide more intensive intervention: Origins in data-based program modification/experimental teaching
were first developed at the University of Minnesota (Deno & Mirkin, 1977).
It is a process, not a single intervention program or strategy. It is not a one-time fix, but an ongoing process comprising intervention
and assessment adjusted over time.
21
DBI: Integrating data-based decision-
making across academics and social behavior
22
Is DBI the same as MTSS?Is DBI special education? Many components of DBI are consistent with elements of
special education and tiered service delivery systems
Tiered Interventions (RTI, MTSS, PBIS)
• Universal, secondary, and tertiary interventions
• Progress monitoring• Team-based decisions based
on data
Special Education
• Individualized program• Progress monitoring• Team-based decisions
based on data
23
Who Needs DBI? Students with disabilities who are not making adequate
progress in their current instructional program
Students who present with very low academic achievement and/or high-intensity or high-frequency behavior problems (typically those with disabilities)
Students in a tiered intervention system who have not responded to secondary intervention programs delivered with fidelity
24
What Is Fidelity? Extent to which an intervention’s core components have
been implemented as planned (Nelson et al., 2012) For DBI, this includes the instructional platform, adapted
iterations of intervention, ongoing progress monitoring, and decision-making procedures.
See IRIS module for additional information: http://iris.peabody.vanderbilt.edu/module/fid/
25
Levels of Fidelity Student level
• For a given student, are assessment and intervention being carried out as planned?
• Student-level fidelity may be assessed through teacher logs, observations, or record review
Systems level (school or team)• Are essential components of DBI being implemented consistently?
• Are there systems-level problems that hinder DBI implementation?
• A systems-level rubric or interview might examine team meeting checklists, logs, or other records
26
Why Does Fidelity Matter? Fidelity assessment provides evidence that DBI is being
implemented as intended• Assessment, intervention, and decision-making processes
• Student and systems levels
Provides guidance on how to improve DBI implementation• Does the interventionist need additional training or support?
• Is an intervention adaptation needed?
• Is there a systems-level problem? (e.g., scheduling prevents sufficient intervention time, staff do not have access to evidence-based instructional platforms)
27
Developing a Rubric to Measure Implementation Fidelity
28
Developing a Systems-Level Rubric Based on the structure of the Center on Response to
Intervention’s RTI Essential Components Integrity Rubric http://www.rti4success.org/resource/essential-components-rti-integrity-rubric-and-worksheet
Identify structures, resources, and practices needed for quality school-level implementation of DBI
Reflect lessons learned from NCII knowledge development sites
29
Measuring School-Level Implementation NCII’s DBI Implementation Rubric and Interview: http://
www.intensiveintervention.org/resource/dbi-implementation-rubric-and-interview
Aligned with the essential components of DBI and the infrastructure that is necessary for successful implementation in Grades K–6
30
Key Findings From Knowledge Development Sites Purpose: to learn about strategies for implementing
intensive intervention from sites that have demonstrated positive outcomes for students with disabilities (SWDs)
Implementing Intensive Intervention: Lessons Learned From the Field (NCII, 2013) http://www.intensiveintervention.org/resource/implementing-intensive-intervention-lessons-learned-field
31
Lessons Learned1. Intensive intervention is most likely to be facilitated when
implemented as a component of a multi-tiered system of support (MTSS)
2. Family engagement supports implementation
3. Intensive behavioral intervention is often more challenging because of limited progress monitoring
4. Inconsistent decision rules for intensifying supports can hinder intervention planning and resource allocation
5. Hidden inefficiencies may overtax staff unnecessarily
32
Components/Infrastructure Required for DBI Implementation1. System Features
2. Data and Decision Making
3. Intervention
4. DBI Process
5. DBI Evaluation
33
1. System Featuresa) Leadership
b) School Schedules
c) Resources
d) Cultural and Linguistic Responsiveness
e) Communication With and Involvement of Parents
f) Communication With and Involvement of All Staff
34
2. Data and Decision Makinga) Process to Identify Students in Need of Intensive
Intervention
b) Academic Progress-Monitoring Tools
c) Behavior Progress-Monitoring Tools
d) Progress-Monitoring Procedures
e) Diagnostic Assessment
f) Fidelity of Assessment Implementation
35
3. Interventiona) Intensive Intervention
b) Adaptation of Intervention
c) Fidelity of Implementation of Intervention
d) Interventionist Characteristics
e) Relationship to Core Instruction
36
4. DBI Processa) Team or Appropriate Support Mechanism
b) Regular Meetings
c) Data Review
d) Documentation
e) Decision Rules
f) Overall Implementation of DBI Process
37
5. DBI Evaluationa) Evaluation
38
Rating Scale1–5 points with anchors for 1 point = little or no implementation 3 points = partial or inconsistent implementation 5 points = complete and consistent implementation
39
DBI Implementation Interview Script and note-taking template for gathering information to
evaluate rubric Sample questions for each rubric item
40
Lessons Learned: Initial use in NCII Intensive Sites Strategies for Improving DBI Implementation
41
Highest Rated Items Trained, experienced interventionists Teams to support decision making for DBI, with a regular
meeting schedule Aligning intervention to student need and core
expectations, addressing prerequisite skills as appropriate District and school leadership support for DBI Technically rigorous academic progress-monitoring tools
that are sensitive to student improvement
42
Lowest Rated Items Evaluation Schools may informally review implementation without
setting specific goals Evaluation may be difficult or a low priority if other pieces
are not clearly articulated and in place
Behavior progress monitoring Fewer validated tools available compared to academics New to many schools
43
Lowest Rated Items Continued…. Fidelity Few schools systematically monitor
Decision rules May not be clearly articulated in one or more areas Application may not be consistent
Overall implementation May be inconsistent or poorly documented
44
Essential Elements of DBI Implementation Staff commitment Student plans Student meetings Valid, reliable data Inclusion of students with disabilities
Staff CommitmentKey Element Flexibility Within
Implementation
Commitment of: Principal Intervention staff Special educators
Specific intervention staff involved including staff who work with students with intensive needs in the area(s) of concern. (e.g., reading specialists, social workers)
46
Student PlansKey Element Flexibility Within
ImplementationStudent plans are developed and reflect: Accurate and timely student
data Goal(s) for the intervention
based on valid, reliable assessment tools
Timeline for executing and revisiting the intervention plan
Content area(s) Number of student plans Grade level(s)
47
Student MeetingsKey Element Flexibility Within
Implementation
Student meetings are data driven
There is a regularly scheduled time to meet
Meetings are structured to maximize efficiency and focused problem solving
Frequency Schedule Team members
48
Progress Monitoring
Key Element Flexibility Within Implementation
Valid, reliable progress monitoring tools are used.
Data are graphed. Data are collected at
regular intervals.
Choice of tool Use of progress-
monitoring data at other tiers
49
Students With Disabilities
Key Element Flexibility Within Implementation
Students with disabilities must have access to intensive intervention.
Who delivers intervention for students with disabilities
Inclusion of students with and without IEPs
50
NCII & NCSI Resources to Support Implementation
51
Addressing Common Barriers to Implementation Systems level considerations—infrastructure and staff
commitment• Module to assess and support readiness to implement DBI
Identifying intervention and assessment tools for mathematics and behavior• Tools charts
• Sample lessons and activities
Consistent procedures and documentation• Student meeting tools
52
Getting Ready to Implement Intensive Intervention: Infrastructure for DBI Module introducing schools to DBI and infrastructure
needed for implementation: http://www.intensiveintervention.org/resource/getting-ready-implement-intensive-intervention-infrastructure-data-based-individualization
Handouts and activities to assess readiness and begin action planning
53
Fidelity Resources
http://www.intensiveintervention.org/fidelity-resources
54
Fidelity Resources Continued
http://www.rti4success.org/resource/essential-components-rti-integrity-rubric-and-worksheet
Tools Charts
Academic Progress Monitoring http://www.intensiveintervention.org/chart/progress-monitoring
Academic Intervention http://www.intensiveintervention.org/chart/instructional-intervention-tools
Behavioral Progress Monitoring http://www.intensiveintervention.org/chart/behavioral-progress-monitoring-tools
Behavioral Intervention http://www.intensiveintervention.org/chart/behavioral-intervention-chart
56
Mathematics: Sample Lessons & Activities
http://www.intensiveintervention.org/resources/sample-lessons-activities/mathematics
57
Implementing Behavioral Strategies: Considerations and Sample Resources
http://www.intensiveintervention.org/implementing-behavioral-strategies-considerations-and-sample-resources
58
Individual Student Meeting Tools
http://www.intensiveintervention.org/tools-support-intensive-intervention-data-meetings
59
Learn More About DBI NCII website Examples of Standards-Aligned Instruction Across Tiers DBI Training Series Webinars Connect to NCII
60
Find Out What NCII Has to Offer
www.intensiveintervention.org
61
Examples of Standards-Aligned Instruction Across Tiers
http://www.intensiveintervention.org/illustration-standards-relevant-instruction-across-levels-tiered-system
62
DBI Training Series Eight training modules focusing on components of DBI for
academics and behavior One module focused on
readiness and action
planning Include
Slides and speaker notes Activities Coaching guides
http://www.intensiveintervention.org/content/dbi-training-series
63
Webinars View archived webinars and look for announcements about the next live webinar:
www.intensiveintervention.org
64
Connect to NCII Sign up on our website
to receive our newsletter and announcements
Follow us on YouTube and Twitter• YouTube Channel:
National Center on Intensive Intervention
• Twitter handle: @TheNCII
65
Connect to NCSI This website provides immediate, easy
access to foundational NCSI information, previously developed technical assistance resources, a snapshot of the NCSI team, and contact information.
www.ncsi.wested.org/contact/
66
Ask the NCSI Ask the NCSI is intended to support
states to (1) obtain information about evidence-based practices;
(2) develop, implement and evaluate State Systemic Improvement Plans (SSIPs);
(3) learn about practices being implemented in other states; and
(4) find out what current research says about “what works” to improve results for children with disabilities.
www.ncsi.wested.org/contact/
67
Questions
68
ReferencesNational Center on Intensive Intervention. (2013). Implementing intensive
intervention: Lessons learned from the field. Washington, DC: U.S. Department of Education, Office of Special Education Programs. http://www.intensiveintervention.org/sites/default/files/Lessons_Learned_From_Field_0.pdf
Nelson, M. C., Cordray, D. S., Hulleman, C. S., & Sommer, E. C. (2012). A procedure for assessing intervention fidelity in experiments testing educational and behavioral interventions. The Journal of Behavioral Health Services & Research, 39(4), 374–396.
69
NCII & NCSI Disclaimers This presentation was produced under the U.S. Department of
Education, Office of Special Education Programs, Award No. H326Q110005 for which Celia Rosenquist serves as the project officer and No. H326R140006 for which Perry Williams and Shedeh Hajghassemali serve as project officers. The views expressed herein do not necessarily represent the positions or polices of the U.S. Department of Education. No official endorsement by the U.S. Department of Education of any product, commodity, service, or enterprise mentioned in this website is intended or should be inferred.
Although permission to redistribute this webinar is not necessary, the citation should be: National Center on Intensive Intervention and National Center on Systemic Improvement (2015). Results Driven Accountability and Intensive Intervention: Using MTSS to Improve Outcomes for Students with Disabilities. Washington, DC: U.S. Department of Education, Office of Special Education Programs, National Center on Intensive Intervention.
Contact UsNational Center on Intensive Intervention1000 Thomas Jefferson Street NWWashington, DC 20007-3835
www.intensiveintervention.org
[email protected]@TheNCII
National Center on Systemic Improvement730 Harrison StSan Francisco, CA 94107
http://ncsi.wested.org/