1 implementing a three-tiered state evaluation structure bob putnam the may institute karen childs...
TRANSCRIPT
![Page 1: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/1.jpg)
1
Implementing a Three-Tiered State Evaluation Structure
Bob Putnam The May Institute
Karen ChildsUniversity of South Florida
2009 National PBIS Leadership Forum
![Page 2: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/2.jpg)
Goals of Session
• Present the purpose of a state-wide evaluation
• Provide exemplars of the core components of three tier state-wide evaluations
• Provide an ongoing state-wide evaluation system
![Page 3: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/3.jpg)
Purpose of Statewide SWPBS Evaluation
• Evaluation is the process of gathering information for decision-making.
• SWPBS decisions focus on levels of adoption, adaptation, and replication. – Efficiency – Effectiveness – Sustainability– Wide-spread adoptability
3
![Page 4: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/4.jpg)
Funding Visibility PolicyPoliticalSupport
Training CoachingBehavioral Expertise
Evaluation
LEADERSHIP TEAM(Coordination)
Local School/District Implementation Demonstrations
![Page 5: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/5.jpg)
Core Indicators• These indicators include considerations about the
– context in which implementation of SWPBS is to occur – inputs available to guide and assist with
implementation – fidelity with which core elements of SWPBS are put in
place – impact of those core elements on the social and
academic behavior of students.
• Assess the breadth of adoption, and sustainability of implementation.
5Algozzine, Horner, Sugai, Barrett, Eber, Kincaid & Lewis, 2009
![Page 6: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/6.jpg)
Context
• What are/were the goals and objectives for SWPBS implementation?
• Who provided support for SWPBS implementation?
• Who received support during SWPBS implementation?
6
![Page 7: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/7.jpg)
Context
Who received support
during SWPBS implementation?
![Page 8: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/8.jpg)
Input
• What professional development was part of SWPBS implementation support?
• Who participated in the professional development?
• What was the perceived value of the professional development?
8
![Page 9: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/9.jpg)
Participating Schools2004 Schools (21)2005 Schools (31)2006 Schools (50)
2000 Model Demonstration Schools (5)
2007 Schools (165)2008 Schools (95)
Who Is Receiving Training and Support?
![Page 10: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/10.jpg)
Training/Technical Assistance
More than three-fourths of the counties in the state have at least one school participating in the North Carolina Positive Behavior Support Initiative.
Who Is Receiving Training and Support?
![Page 11: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/11.jpg)
Tier 2 & Tier 3 Who Is Receiving Training and Support?
![Page 12: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/12.jpg)
Tier 2 & Tier 3 Who Is Receiving Training and Support?
![Page 13: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/13.jpg)
Tier 3 Who Is Receiving Training and Support?
![Page 14: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/14.jpg)
Fidelity
• To what extent was SWPBS implemented as designed?
• To what extent was SWPBS implemented with fidelity?
14
![Page 15: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/15.jpg)
School-Wide Evaluation Tool35 Coaches trained as SET assessors
15 Contractual SET assessors
• 97 SETs completed 2004• 154 SETs completed 2005• 157 SETs completed 2006• 104 schools have at least two SET scores• 80% Total score is considered Maintenance
Phase (IPI)• All regions met 80% criterion across schools• 69% increase after one year of implementation
To What Extent Have Practices Changed?
![Page 16: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/16.jpg)
SET Scores by Region
42.52
25.00 29.0042.00
88.44 92.9086.60
95.58 91.04
0
20
40
60
80
100
Central Eastern Southern Special Western
Pre Post
To What Extent Have Practices Changed?
![Page 17: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/17.jpg)
% at phase
11.32%
9.47%
23.16%
56.05%
Preparation Initiation Implementation Maintenance
To What Extent Have Practices Changed?
![Page 18: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/18.jpg)
To What Extent Have Practices Changed?
www.pbisillinois.org
![Page 19: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/19.jpg)
Tier 3To What Extent Have Practices Changed?
www.pbisillinois.org
![Page 20: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/20.jpg)
Tier 3To What Extent Have Practices Changed?
www.pbisillinois.org
![Page 21: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/21.jpg)
Impact
• To what extent is SWPBS associated with changes in student outcomes?
• To what extent is SWPBS associated with changes in other areas of schooling?
21
![Page 22: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/22.jpg)
What extent is SWPBS associated with changes in student behavior?
www.pbisillinois.org
![Page 23: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/23.jpg)
What extent is SWPBS associated with changes in student behavior?
www.pbisillinois.org
![Page 24: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/24.jpg)
Tier 2 & 3
What extent is SWPBS associated with changes in student outcomes?
www.pbisillinois.org
![Page 25: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/25.jpg)
Percent of Students at DIBELS Benchmark (Spring) and Major Discipline Referrals per 100 Students
To What Extent has Behavior and Academic Performance Changed?
![Page 26: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/26.jpg)
MiBLSi Schools and Reading MEAP:Average Total Office Discipline Referrals per
100 Students per Day 2004-2005
To What Extent has Academic Performance Changed?
![Page 27: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/27.jpg)
MEAP- 4MEAP- 4thth grade Reading Assessment grade Reading Assessment29 Elementary Schools In Michigan29 Elementary Schools In Michigan
Schoolwide: Over 55% of major discipline referrals from classroom
Schoolwide: Under 55% of major discipline referrals from classroom
Probability of scoring below 75% proficiency on 4th grade MEAP (Reading): .78
Probability of scoring above 75% proficiency on 4th grade MEAP (Reading): .75
To What Extent has Academic Performance Changed?
![Page 28: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/28.jpg)
North CarolinaPositive Behavior Support Initiative
[A]chievement causes
[B]ehavior?
[B]ehavior causes
[A]chievement?
[Context causes [A]chievement
and [B]ehavior?
To What Extent has Academic Performance Changed?
![Page 29: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/29.jpg)
Replication, Sustainability, and Improvement
• To what extent did SWPBS implementation improved capacity for the state/region/district to replicate, sustain, and improve behavior and other outcomes?
• To what extent did SWPBS implementation change educational/behavioral policy?
• To what extent did SWPBS implementation affect systemic educational practice?
29
![Page 30: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/30.jpg)
What is the breadth of adoption, and sustainability
of implementation?
www.pbisillinois.org
![Page 31: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/31.jpg)
What is the breadth of adoption, and sustainability of implementation?
465
030 42
6181
133
183
283
383
1434
64
118
178
263
368
559
0
100
200
300
400
500
600
700
FY 99 FY 00 FY 01 FY 02 FY 03 FY 04 FY 05 FY 06 FY07(P rojected)
Schools Trained Coaches Trainedwww.marylandpbis.org
![Page 32: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/32.jpg)
Conclusions
• Increase of states reporting on the implementation of SWPBS
• Information on implementation and outcomes of Tier 1 interventions more available than Tier 2 and 3 interventions
• Research is needed on the types of and presentations of data included in state evaluations that will have best outcome on funding, visibility, political support and policy.
![Page 33: 1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership](https://reader035.vdocuments.site/reader035/viewer/2022070415/56649ccf5503460f9499b604/html5/thumbnails/33.jpg)
For More Information
Bob Putnam, Senior Vice President of School ConsultationMay InstitutePhone: (781) 437-1207)Email: [email protected]: http:/mayinstitute.org
Karen Elfner Childs, Research & Evaluation CoordinatorFlorida’s PBS ProjectPhone: (813) 974-7358Email: [email protected]: http://flpbs.fmhi.usf.edu