pdf7
TRANSCRIPT
ORI GIN AL PA PER
Comparing Individual Behavior Plans from SchoolsWith and Without Schoolwide Positive BehaviorSupport: A Preliminary Study
Natasha S. Medley Æ Steven G. Little ÆAngeleque Akin-Little
Published online: 2 October 2007
� Springer Science+Business Media, LLC 2007
Abstract School-wide positive behavior support (SWPBS) has been proposed
as a proactive and preventive method to reduce problematic behavior in schools.
Under this approach, educators and administrators seek to create a school
environment that fosters prosocial behavior and attempts to systematically deter
problem behaviors before they happen. To date, the relationship between
SWPBS and individualized positive behavior support (PBS) plans has not been
examined. Specifically, it is unclear whether an atmosphere of SWPBS facilitates
the functional behavioral assessment process and the design of PBS plans for
students exhibiting severe behavior problems. The purpose of the present study
was to investigate whether behavior support plans created in schools employing
SWPBS systems were more technically adequate than support plans created in
schools utilizing traditional approaches to behavior problems. Results indicated
that support plans created at schools with SWPBS systems were more technically
adequate than support plans produced at non-SWPBS schools as measured by the
Behavior Support Plan-Quality Evaluation (BSP-QE). However, support plans
from schools with SWPBS systems were still considered underdeveloped.
Limitations and future research are discussed.
Keywords Positive behavior supports � Behavior support plans � BSP-QE
Schools have become increasingly interested in identifying strategies to reduce
disruptive and violent behaviors and raise prosocial behaviors in students. School-wide
N. S. Medley (&)
Graduate School of Education, University of California, Riverside, Riverside, CA 92521, USA
e-mail: [email protected]
S. G. Little � A. Akin-Little
Walden University, Minneapolis, MN, USA
123
J Behav Educ (2008) 17:93–110
DOI 10.1007/s10864-007-9053-y
positive behavior support (SWPBS) is a proactive model that utilizes preventive
strategies at three levels to reduce problematic behavior within school settings (Scott
et al. 2002; Anderson and Kincaid, 2005; Eber et al. 2002; Safran and Oswald 2003;
Sugai and Horner 2002). The three-tier model employs empirically-based interven-
tions to promote prosocial behavior in (a) the general school population of students
who lack chronic behavior problems (primary prevention); (b) students who are at risk
of developing chronic behavior problems (secondary prevention); and (c) students
with major behavioral problems (tertiary prevention). In an effective SWPBS system,
changes in attitudes and behavior occur across both students and staff. Data suggest
that an effectual SWPBS system will foster a school climate where appropriate
behavior is acknowledged and appreciated by all staff, within-child psychopathology
is not viewed as an unchangeable entity, and expectations of staff are universal
(Horner and Carr 1997; Crone and Horner 1999–2000).
At the core of this model is the problem-solving team whose purpose, at the
tertiary prevention level, is to identify the environmental factors supporting the
student’s problem behavior and produce strategies and interventions that will
decrease those behaviors (Colvin et al. 1993). The functional behavior assess-
ment (FBA) process plays a key role in the problem-solving team’s ability to
uncover the function of the behavior, and the antecedents and consequences that
facilitate its occurrence. In the context of a SWPBS system, collaborative teams
should be comprised of individuals that are knowledgeable in the area of
behavior support. Furthermore, it can be expected that a comprehensive problem-
solving team will have a team leader who has expertise in applied behavior
analysis (Sugai et al. 1999–2000; Crone and Horner 1999–2000). When these
foundational components of the SWPBS system are in place, it can be
anticipated that behavior teams are equipped to navigate through the FBA and
Behavior Support Plan (BSP) process. It may be further expected that these
teams will create BSPs that will lead to more effective behavioral interventions
than BSPs created by teams utilizing a more traditional approach and/or lacking
in specific PBS-FBA training. For example, Benazzi et al. (2006) found that
BSPs formulated by a team with a behavior specialist were rated higher in
technical adequacy and contextual appropriateness than BSPs created by a team
without a behavior specialist. Further, other researchers have found that
availability of FBA information does not necessarily lead to more effective
BSPs (Hsiao and Albin 2000) and that teams that use FBA information
effectively need at least one member specially trained in behavioral theory
(Mitachi and Albin 2001). Consequently, the assumption is that teams with more
behavioral training will produce better quality BSPs than teams without that
training. This study attempts to ascertain the veracity of that assumption.
The success of a BSP clearly depends on the quality of the plan (Horner et al.
2000; Sugai et al. 1999). Producing accurate and technically sound BSPs is
important not only because it is best practice, but also because it is mandated by
federal law. IDEA (1997) clearly mandates that when disciplinary action is taken
by school, the Individualized Education Plan (IEP) team should conduct an FBA
and develop a related BSP. The 2004 IDEA revision continues the emphasis on
BSPs and use of the FBA process (20 U.S.C.§ 1400 et seq.). Although inadequate
94 J Behav Educ (2008) 17:93–110
123
plans may result in behavior change, this generally is not the case. Furthermore,
when a BSP is inadequate it may actually exacerbate student problems (e.g.,
reactionary methods are only included, focus on hypothesized within-child
psychopathology), leading to higher rates of problem behavior (e.g., Mayer et al.
1983).
In recent years, several measures have been developed to judge the adequacy of
the BSP (e.g., Lewis-Palmer et al. 2004). The Behavior Support Plan Quality-Evaluation (BSP-QE) scoring guide (PENT 2003) is the only instrument that
evaluates the quality of the BSP (the purpose of the present study), offers concrete
definitions and examples for each component of the BSP, and allows the evaluator
to calculate a total score that indicates the overall quality of the plan. The evaluator
rates the support plans across 12 concepts, ranging from the definition of problem
behavior to the relationship of functional assessment results to intervention
strategies.
The purpose of the present study was to evaluate whether individualized BSPs
created in schools employing SWPBS were more technically adequate than those
created in schools without SWPBS training, utilizing traditional approaches to
behavior problems. It was hypothesized that individualized BSPs in the context of
SWBS would be more technically adequate than those created in non-SWPBS
schools, given that historically, interventions for students with behavior problems
that rely on reactive procedures have not had positive outcomes (Walker et al. 1995;
Walker and Shinn 2002). To date, it has been assumed that individuals with training
in assessment-based intervention should develop improved BSPs due to a better
understanding of the principles of behavior change and the FBA process. The intent
of this study was to test this hypothesis.
A second hypothesis was that the quality of support plans in SWPBS schools
would be related to length of time the system had been in place, with the longer
duration of SWPBS resulting in a higher score on technical adequacy of SWPBS
plans. These two hypotheses are based on two assumptions: (a) a systemic view of
the importance of proactive and preventative approaches to addressing problem
behavior permeating to all levels of support, and (b) behavioral expertise of the
collaborative team in schools with SWPBS will lead to higher quality plans. It is
assumed that because staffs at SWPBS schools are required to receive training on
PBS strategies, they should display a better understanding of behavior, its function,
and environmental factors that maintain the problem behavior and thus produce
more technically adequate BSPs.
Method
Participants
Schools
Nine middle schools from a single district in an urban community located in
Southern California participated in this study. The study included support plans
J Behav Educ (2008) 17:93–110 95
123
from all nine middle schools in the district. Each school served between 1,300 and
2,200 students. The ethnic breakdown of the district is as follows: (a) Hispanic or
Latino—62%, (b) African-American—19%, (c) White—14%, (d) Asian—2%, and
(e) other—3%. On average, 89% of students qualify for free or reduced lunch. At
the time of the study, two of the nine schools had implemented SWPBS systems.
SWPBS support plans were obtained from the two SWPBS schools and the seven
middle schools not employing a SWPBS system. The demographics of the schools
from which support plans were obtained were similar.
The implementation dates of SWPBS system varied for both schools. At the time
of the study School 1 had a SWPBS system in place for 22 months, while School 2
had a SWPBS system in place for 10 months. The composition of team members for
both schools was similar; however, School 1 had a psychology intern on the team
with training in applied behavior analysis.
SWPBS System and Training
Each school implementing SWPBS established two collaborative teams. The
Behavior Team (BT) was trained to lead the new school-wide policies and the
Student Study Team (SST) assumed the role of handling behavioral concerns for
nonresponsive students. Under the model, the SST was responsible for the
implementation of behavior goals and writing support plans for at-risk students.
Generally, the SST and BT were composed of the school psychologist, general
education and special education teachers, and the vice principal. Some team
members, such as general and special education teachers, participated in the
meetings based on their involvement with the student under evaluation.
Schools implementing a SWPBS system received four full days of onsite
trainings focused on the framework for a Positive School-Wide Discipline program
and the creation of the Tier 1 and Tier 2 interventions. Both SST and behavioral
team members received a half-day training on the fundamentals of BSP writing,
based on the Behavior Support Plan-QE that was developed by Positive
Environments, Network of Trainers (PENT 2003). During these trainings, teams
were taught how to use the information provided by the functional assessment to
create a BSP. Team members were explicitly trained on the differences between an
effective and poor BSP. The training addressed each component of the BSP and
gave examples of how various answers would be scored. During the session,
trainees were given the opportunity to practice writing a BSP and were given
feedback. Additionally, after the training, team members were allowed to submit
sample BSPs for critique and feedback.
In order to ensure that all staff was familiar with the principles of SWPBS, an
overview of the program was presented on site for all school employees. These
trainings were separate from the 4-day training attended by collaborative teams and
school administration. Furthermore, throughout the year district coaches offered a
variety of positive discipline trainings for all staff to attend. Both SWPBS schools
adopted three main rules on campus. The three broad rules were: (a) be safe, (b) be
responsible, and (c) be respectful.
96 J Behav Educ (2008) 17:93–110
123
District-wide Behavior Support Plan Training
A basic BSP training is offered quarterly to all district employees. The training is
offered as an option for professional development hours. These workshops are
generally attended by school psychologists, resource specialists, and special
education teachers. Typical attendance for each workshop is approximately 20
staff. Staff from SWPBS and non-SWPBS schools was welcomed to attend the basic
training. The 4-h training is voluntary and takes place over a 2-day period. The
training focuses on the purpose of the BSP and explains the various components of
the BSP. The rationale behind the BSP and its role in special education placement
are discussed. Trainees are provided with a blank template created to mirror the
layout of the BSP-QE rubric. While each component of the BSP is mentioned, the
training centers on teaching educators how to identify environmental correlates and
function of problem behavior. Definitions of each construct are given and trainees
are provided with four examples to show the possible functions of the behavior and
environmental factors that increase the likelihood of the behavior occurring.
However, trainees are not shown the types of responses that would be included in a
technically adequate BSP. No additional assistance for BSP writing is provided after
the workshop.
Materials
All BSPs were evaluated using the BSP-QE scoring guide. The BSP-QE was
created in 2003 by Diana Browning-Wright and Dru Saren, with input from Rob
Mayer (PENT 2003). In its development, the rubric was used by over 200
behavior specialists and was revised to improve the educator’s ability to
effectively evaluate BSPs and produce scores that accurately indicate the quality
of the BSP plan. The purpose of the rubric is to establish whether the BSP
developed by the team aligns with the principles of behavioral change found in
applied behavior analysis. The BSP-QE does not assess the appropriateness of
the BSP in relationship to the developmental needs of the students. Specifically,
the BSP-QE does not determine whether the identified function of the behavior is
correct or whether the interventions selected were appropriate or implemented as
intended.
The guide is based on six key concepts posited to be important in the creation
of an effective BSP. The six key concepts, as outlined by Positive Environments,
Network of Trainers (2003) are as follows: (a) all behavior, including problem
behavior serves a purpose for the student; (b) the behavior is related to the context
or environment in which it occurs; (c) in order to change behavior, the
environment must be changed such that the problem behavior is no longer
effective and a functionally-equivalent replacement behavior must be taught;
(d) to increase maintenance of behavior over time, the new behavior must be
reinforced; (e) implementers must have a uniform method regarding how problem
behavior will be addressed if it reoccurs; and (f) frequent two-way communication
between all stakeholders must occur and staff training must be ongoing.
J Behav Educ (2008) 17:93–110 97
123
The BSP-QE measures the key components over 12 categories. These 12 factors
are: (a) problem behavior, (b) predictors of problem behavior, (c) the relationship
between environmental changes and problem behaviors, (d) the logical relation-
ship between environmental changes and events supporting the problem, (e) the
relationship of predictors to the function of the behavior, (f) the relationship of
function to replacement behavior, (g) the relationship of teaching strategies to
replacement behavior, (h) the quality of reinforcers to be used during the
intervention, (i) the adequacy of reactive strategies to be utilized when the child
exhibits the problem behavior, (j) goals and objectives of the intervention,
(k) team coordination and implementation, and (l) outline of communication. Each
category can be scored as zero, one, or two, with two indicating the objectives of
the category were met and zero meaning the objectives were either minimally met
or absent. The rubric operationally defines the characteristics required to
accompany scoring for each category. The total BSP-QE score ranges from 0 to
24. BSPs yielding fewer than 12 points are categorized as weak. The PENT cadre
suggests that while this plan may affect some change in problem behavior, the
written plan only weakly expresses the principles of behavior change. It is
suggested that any plan scoring in this category should be rewritten. BSPs
receiving scores between 13 and 16 represent underdeveloped plans. While there
is the possibility that this plan could produce some change in behavior, it would
require many modifications to embody best practice. Plans producing scores
between 17 and 21 points are categorized as good. Plans rated as good are likely
to produce positive changes in behavior and incorporates elements of best
practice. BSP yielding scores of 22 points or more are considered superior. The
PENT cadre postulates that this plan is likely to produce positive changes in
behavior and embodies best practice.
Recent research suggests adequate reliability and validity of the BSP-QE. Cook
et al. (in press) indicated a Cronbach alpha greater than .80 and an interrater
reliability estimate that exceeded .80. Furthermore, content validity was reviewed
by experts in PBS and applied behavior analysis, who reported that the BSP-QE had
adequate content validity (Cook et al. in press).
Procedure
Behavior support plans from SWPBS and non-SWPBS schools were obtained from
the district. Several steps were taken to identify support plans for this study. First, a
list was generated with the names of students for whom support plans had been
created at each school. Next, students were randomly selected from the list and their
BSPs were obtained from the schools. Two support plans for randomly selected
students could not be found despite reports indicating that support plans had been
developed.
Forty support plans (21 SWPBS; 19 non-SWPBS) were evaluated in this study.
Prior to evaluation, all identifying information was removed from the plans. In order
to control for response bias, any information indicating whether the support plan
98 J Behav Educ (2008) 17:93–110
123
was from a SWPBS or non-SWPBS school was removed. The only coding
information on the support plan was the student identification number.
Each plan was evaluated by the first author using the BSP-QE. The standards
outlined by the rubric were applied to each item. The score and the rationale for the
score were recorded for each item. After all components were rated, the item scores
were summed and assigned a categorical evaluation based on the total score. This
process was repeated for all support plans.
Prior to rating the plan, the first author received 4 h of training on the BSP-QE and how to evaluate plans by two behavior specialists. Training focused on
each component of the BSP, offering examples of the types of responses that
would yield a score of zero, one, or two. During this training the author
practiced applying the principles of the rubric to a sample BSP. Then, she
created her own support plan and evaluated it using the BSP-QE. The trainers
concluded that the first author adequately applied the guidelines of the rubric to
the evaluation of the BSP.
Interobserver Agreement
In order to obtain an interobserver agreement estimate, 25% (10 support plans) of
the sample support plans were evaluated by a second reviewer. The second reviewer
received the same 4-h training as the first author. The support plans were compared
on each of the 12 components outlined on the BSP-QE. Interobserver agreement
was calculated by dividing the number of exact numeric agreements on each
component by the total number of items and multiplying by 100%. Reliability was
calculated using Cohen’s kappa (Cohen 1960). Agreement across all components
was .61. According to Landis and Koch (1977), the strength of agreement based on a
score of .61 is good.
Data Analysis Procedures
An independent samples t-test was used to determine whether or not there were
significant mean differences between SWPBS and non-SWPBS total scores.
Specifically, support plans were compared on how closely they adhered to the
guidelines outlined on the BSP-QE. Support plans were compared on each of the 12
components of the BSP-QE and the total score. A t-test was employed because there
were only two levels of the independent variable. Furthermore, t-tests are robust to
minor departures of normality and homogeneity of variance. Graphs of the total
scores did not reveal any outliers in the distributions of SWPBS and non-SWPBS
scores. Additionally, Fisher’s z’ transformation was used to identify any significant
differences in correlations amongst variables based on whether the school was a
SWPBS or non-SWPBS school. Finally, a Pearson correlation was calculated to
determine whether there was a relationship between length of SWPBS implemen-
tation and BSP quality.
J Behav Educ (2008) 17:93–110 99
123
Results
Mean Comparisons
Forty support plans (21 SWPBS; 19 non-SWPBS) were evaluated. An independent
samples t-test was preformed to compare the total scores yielded by SWPBS and
non-SWPBS schools. A comparison of mean total score of SWPBS schools and
non-SWPBS schools resulted in a mean of 13.95 for SWPBS schools (SD = 6.67)
and a mean of 7.84 for non-SWPBS schools (SD = 3.76). A Levene’s test for the
equality of variances for the total BSP score yielded an F-value of 16.769
(p \ .001) therefore equal variances were not assumed for the t-test. Analysis
revealed a t of –3.606 with 32.175 degrees of freedom and a two-tailed p-value of
.001. This indicates that there was a significant difference between the total scores
yielded by SWPBS and non-SWPBS schools, such that SWPBS schools received
significantly higher total scores on the BSP-QE. When total scores were
differentiated by school type and BSP-QE effectiveness categories, very different
patterns of score distributions emerged. In SWPBS schools the following pattern
emerged: a) nine of 21 support plans were rated as weak, b) three of 21 were rated as
underdeveloped, c) 6 of 21 were rated as good, and d) three of 21 were rated as
superior. In non-SWPBS schools, 16 of 19 support plans were categorized as weak
and three of the 19 non-SWPBS support plans were categorized as underdeveloped.
Correlational Analyses
We hypothesized that there would be a positive relationship between the length of
SWPBS implementation and support plan total scores—that is longer the
implementation of a three-tier SWPBS model, the higher the total scores. A
Pearson correlation was utilized to assess the relationship between length of time
and total scores. Results did not confirm this hypothesis. A significant correlation
was not found for length of SWPBS implementation and total scores, r = 0.169.
Analysis of SWPBS schools yielded moderate to large correlations between the
BSP total score and problem behaviors (r = .491), predictors of behavior (r = .834),
environmental changes (r = .618), predictors related to function (r = .536),
replacement behaviors (r = .792), teaching strategies (r = .856), reinforcement
(r = .952), reactive strategies (r = .871), goals and objectives (r = .918), and
communication (r = .820). The variable ‘‘predictors of behavior’’ yielded moderate
to strong correlations with environmental changes (r = .476), replacement behaviors
(r = .678), teaching strategies (r = .736), reinforcement (r = .782), reactive strat-
egies (r = .691), goals and objectives (r = .775), and communication (r = .616).
Results are listed in Table 1.
Bivariate correlations were calculated for SWPBS and non-SWPBS schools.
According to Cohen’s (1988) guidelines, analysis yielded moderate to large
correlations. Results for non-SWPBS schools are listed in Table 2. Moderate to
large correlations were found for the BSP total score and problem behaviors
(r = .492), predictors of behavior (r = .564), environmental changes (r = .697),
100 J Behav Educ (2008) 17:93–110
123
Ta
ble
1In
terc
orr
elat
ion
sam
on
gco
mp
on
ents
and
tota
lsc
ore
on
the
BS
P-Q
Efo
rS
WP
BS
Sch
oo
ls
Mea
sure
12
34
56
78
91
01
11
21
3
1.
Pro
ble
mb
ehav
ior
2.
Pre
dic
tors
of
beh
avio
r.2
93
3.
Su
pp
ort
ing
pro
ble
mb
ehav
ior
.16
0.3
53
4.
En
vir
on
men
tal
chan
ges
.21
2.4
76*
.16
5
5.
Pre
dic
tors
rela
ted
tofu
nct
ion
.327
.378
–.2
37
.256
6.
Rep
lace
men
tbeh
avio
rs.3
94
.678**
.160
.212
.462*
7.
Tea
chin
gst
rate
gie
s.3
27
.736**
.389
.464*
.285
.702**
8.
Rei
nfo
rcem
ent
.34
5.7
82*
*.2
64
.57
8*
*.4
87*
.77
1*
*.8
50*
*
9.
Rea
ctiv
est
rate
gie
s.4
26
.691**
.144
.568**
.536*
.757**
.673**
.857**
10
.G
oal
san
do
bje
ctiv
es.5
09*
.77
5*
*.3
82
.61
1*
*.5
09*
.66
1*
*.7
24*
*.8
32*
*.8
34
**
11
.T
eam
coo
rdin
atio
n–
.175
–.2
09
.20
0–
.132
–.1
75
–.1
75
–.0
14
–.0
23
–.2
88
–.1
16
12
.C
om
mu
nic
atio
n.3
02
.61
6*
*.1
71
.47
7*
.48
8*
.63
6*
*.6
69*
*.8
47*
*.6
17
**
.67
3*
.18
1
13
.B
SP
tota
lsc
ore
.49
1*
.83
4*
*.3
86
.61
8*
*.5
36*
.79
2*
*.8
56*
*.9
52*
*.8
71
**
.91
8*
*–
.070
.82
0*
*
14
.L
eng
tho
fP
BS
imp
lem
enta
tio
n–
.159
.23
2–
.20
0.2
24
.16
8.0
37
.06
4.2
12
.38
3.2
47
–.2
13
.12
4.1
69
*p
\.0
5,
two
-tai
led
;*
*p
\.0
1,
two-t
aile
d
J Behav Educ (2008) 17:93–110 101
123
Ta
ble
2In
terc
orr
elat
ions
amo
ng
com
po
nen
tsan
dto
tal
sco
reo
nth
eB
SP
-QE
for
No
n-S
WP
BS
Sch
oo
ls
Mea
sure
12
34
56
78
91
01
11
2
1.
Pro
ble
mb
ehav
ior
2.
Pre
dic
tors
of
beh
avio
r.4
70*
3.
Su
pp
ort
ing
pro
ble
mb
ehav
ior
–.1
20
.04
1
4.
En
vir
on
men
tal
chan
ges
.19
7.5
07*
.12
3
5.
Pre
dic
tors
rela
ted
tofu
nct
ion
.243
.218
.007
.308
6.
Rep
lace
men
tb
ehav
iors
.35
1.3
05
–.1
60
.35
4.5
17*
7.
Tea
chin
gst
rate
gie
s–.1
22
–.2
39
.418
.070
.171
.224
8.
Rei
nfo
rcem
ent
.43
6.5
76*
*–
.220
.42
8.0
65
.14
7–
.34
1
9.
Rea
ctiv
est
rate
gie
s.0
23
.215
.474*
.190
.262
.469*
.208
–.1
79
10
.G
oal
san
do
bje
ctiv
es.2
24
.10
5.2
05
.09
3.0
54
.05
6.2
31
–.1
67
.10
2
11
.T
eam
coo
rdin
atio
n–
.014
–.2
48
.22
3.1
82
–.1
02
.42
5.3
93
–.0
38
.21
7.0
55
12
.C
om
mu
nic
atio
n.1
49
.21
7–
.233
.06
2–
.062
–.3
17
–.3
62
.24
0–
.190
.24
4–
.422
13
.B
SP
tota
lsc
ore
.49
2*
.56
4*
.24
5.6
97*
*.5
81*
*.7
65
**
.30
3.3
37
.56
2*
.30
3.3
69
–.0
79
*p
\.0
5,
two
-tai
led
;*
*p
\.0
01,
two-t
aile
d
102 J Behav Educ (2008) 17:93–110
123
predictors related to functions (r = .581), replacement behaviors (r = .765) and
reactive strategies (r = .562). A moderate correlation was found between problem
behavior and predictors of behavior (r = .470). A large correlation was found
between predictors of behavior and environmental changes (r = .507) and
reinforcement (r = .576). A moderate correlation was found between supporting
problem behaviors and reactive strategies (r = .474). Lastly, a strong correlation
was suggested between predictors related to function and replacement behaviors
(r = .517).
Since the Pearson correlation coefficient of SWPBS schools was not being
compared to zero, but to a known r (non-SWPBS), a Fisher’s z transformation was
used to compare the correlations produced by non-SWPBS and SWPBS schools.
Results indicated significant differences in correlations between total BSP scores
and predictors of behavior (Z = 2.352), teaching strategies (Z = 4.085), reinforce-
ment (Z = 6.288), reactive strategies (Z = 2.966), goals and objectives (Z = 5.284),
and communication (Z = 4.563). Results were in the predicted direction. Schools
with SWPBS systems yielded higher positive correlations than non-SWPBS
schools. Interestingly, the correlation for reactive strategies and supporting problem
behavior was higher for non-SWPBS schools (r = .474) than SWPBS schools
(r = .144). This was the only correlation in which the difference between the
schools displayed different results such that the correlation for a non-PBS school
was higher. Results are listed in Table 3.
Discussion
The purpose of this study was to determine if there were differences in the technical
quality of BSPs written at non-SWPBS and SWPBS schools. Specifically, the study
sought to investigate whether schools employing a SWPBS system produced better
support plans than schools utilizing traditional methods to address students
displaying problematic behaviors. It was hypothesized that SWPBS schools would
yield significantly higher total scores as measured by the BSP-QE than non-SWPBS
schools. Mean comparisons between the two types of school supported this
hypothesis. BSP total scores on average were higher for SWPBS schools than non-
SWPBS schools. However, despite yielding results in the hypothesized direction,
the scores still indicated that over half of the plans in the SWPBS schools were rated
as underdeveloped or weak according to the BSP-QE rubric.
Amongst the two SWPBS schools, it was expected that length of implementation
would influence the quality of the support plan. It was presupposed the school that
had SWPBS in place longer would produce higher BSP total scores than the school
with the shorter duration. The difference in implementation dates of the school-wide
PBS system for the two SWPBS schools was 1 year. Results indicated that there
was not a significant relationship between length of implementation and overall
score on the BSP-QE. This was also true for individual components of the BSP-QE.
No significant correlations were found between length of implementation and the 12
components of the BSP. These results seem counterintuitive given that according to
the district’s schedule for training SWPBS schools, the school with the longer
J Behav Educ (2008) 17:93–110 103
123
implementation date would have received more specialized training on support plan
writing, which should have led to higher proficiency in BSP writing. The anticipated
result may not have been observed for two reasons. First, there may have been a
difference between the school staff in the overall acceptance of the SWPBS system.
It has been recommended that at least 80% of school staff must be willing to
participate and adhere to the policies of the SWPBS system (Sugai et al. 2003). In
places where the school climate is not supportive of the SWPBS system, the
development of the program may be stifled, therefore limiting collaborative team
growth. A second factor that may account for the results were the previous skills of
the collaborative team members. The effect of length of implementation may have
Table 3 Item-total score product-moment correlations for SWPBS and non-SWPBS schools
Variables SWPBS Non-SWPBS Z’
1. Replacement behaviors-predictors of behavior .679 [.305 2.178*
2. Teaching strategies-predictors of behavior .736 [–.239 2.945**
3. Reactive strategies-predictors of behavior .691 [.215 2.669**
4. Goals and objectives-predictors of behavior .775 [.105 3.932**
5. Communication-predictors of behavior .616 [.217 2.114**
6. BSP total score-predictors of behavior .834 [.564 2.352*
7. Reactive strategies-supporting problem behavior .144 \.474 –2.119*
8. Goals and objectives-environmental changes .611 [.093 2.623**
9. Reinforcement-predictors related to function .487 [.065 1.992*
10. Goals and objectives-predictors related to function .509 [.054 2.144*
11. Communication-predictors related to function .488 [–.062 1.992*
12. Replacement behaviors-teaching strategies .702 [.224 2.725**
13. Reinforcement-replacement behaviors .771 [.147 3.703**
14. Reactive strategies-replacement behaviors .757 [.469 2.051*
15. Goals and objectives-replacement behaviors .661 [.056 3.120**
16. Reinforcement-teaching strategies .850 [–.341 3.822**
17. Reactive strategies-teaching strategies .673 [.208 2.555*
18. Goals and objectives-teaching strategies .724 [.231 2.856**
19. BSP total score-teaching strategies .856 [.303 4.085**
20. Reactive strategies-reinforcement .857 [–.179 4.648**
21. Goals and objectives-reinforcement .832 [–.167 4.326**
22. Communication and reinforcement .847 [.240 4.208**
23. BSP total score-reinforcement .952 [.337 6.288**
24. Goals and objectives-reactive strategies .834 [.102 4.610**
25. Communication-reactive strategies .617 [–.190 2.225*
26. BSP total score-reactive strategies .871 [.562 2.966**
27. Communication-goals and objectives .673 [.244 2.398*
28. BSP total score-goals and objectives .918 [.303 5.284**
29. Communication-BSP total score .820 [–.079 4.563**
* p \ .05, two-tailed; ** p \ .01, two-tailed
104 J Behav Educ (2008) 17:93–110
123
been muted if the members of the collaborative team of the school with the shorter
implementation date had a strong behavioral foundation prior to training. Because
data were not collected on the professional background of the team members or
staff, this explanation can only be viewed as speculation.
The total score on the BSP-QE is based on 12 different factors. However, many
of these factors are interrelated, such that incorrect assessments on one factor may
negatively affect subsequent ratings. Given these interrelations, it can be expected
that several of the individual components will be highly correlated. Overall, the
correlations in SWPBS schools followed the anticipated pattern; however, some
factors did not correlate that should have. For example, the BSP-QE the component
‘‘predictors of behavior’’ is supposed to directly impact recommendations in
‘‘environmental changes’’ and ‘‘predictors of function.’’ This expectation was
supported for environmental factors, but not for predictors related to function.
Environmental changes referred to environmental, curriculum, and/or interaction
changes that would alleviate the need to engage in the problem behavior. These
environmental modifications were supposed to be related to the identified
antecedents reported in predictors of behavior. The correlation yielded for these
variables was moderate, suggesting that there was a positive relationship between
these variables. However, the expected relationship between predictors of behavior
and predicted function was not observed. The antecedents noted in predictors of
behavior should have informed the team’s perception of why the student was
engaging in the target behavior. A nonsignificant relationship between these
variables suggests that teams failed to integrate these two concepts. For example, in
one BSP the identified predictors of behavior were: (a) academic tasks involving
cursive writing, (b) timed math assignments, (c) dictation tasks, and (d) whole group
instruction. In order to gain the highest score of two on the BSP-QE the
environmental changes must be logically related to the predictors. The environ-
mental changes recommended on the BSP were: (a) seat student next to the door for
access to alternative work areas, (b) allow the student to select a ‘‘lunch buddy’’ and
leave class early for lunch, (c) consider the student’s sensory sensitivities and need
for movement, and (d) front-loading or telling the student what the next activity will
be and what he needs to do. Upon review, only one of the proposed environmental
modifications was logically related to the reported predictors of behavior. Although
these types of errors were less frequent on the support plans produced at SWPBS
schools, it is important to note that they also had problems integrating the various
components of the BSP.
Fewer correlations were observed between variables on support plans written at
non-SWPBS schools. This partially explains why the total scores of non-SWPBS
schools were lower than SWPBS school. As mentioned earlier, many of the items on
the BSP-QE build on each other so that is there is a relationship between many of
the variables. In order to achieve a high total score, several of the items must be
logically related to one another. Only five correlations between variables were
observed in non-SWPBS schools. Interestingly, contrary to SWPBS schools, a
strong correlation was observed between predictors of behavior and environmental
factors; however, the correlation amongst predicted function and predictors of
behavior was not significant. In non-SWPBS schools, the predictors identified often
J Behav Educ (2008) 17:93–110 105
123
centered around internal or within-child psychopathology as opposed to environ-
mental triggers; therefore, functions were typically unrelated to antecedents. For
example a support plan listed mood as an antecedent and hypothesized the function
of the behavior to be a ‘‘place of belong.’’ Furthermore, the strong correlation for
environmental changes and predictors of behavior may have been found because of
the logical link between medicine and psychopathology. Often the recommended
environmental change would be to ensure that the child receive his or her
medication, which despite not being a school factor, is related to a predictor stating
the child has bipolar disorder and refuses to take medication.
Significant differences were observed between correlations from SWPBS and
non- SWPBS schools. Overall, significant differences were found in 29 correlations
for SWPBS and non-SWPBS schools. These differences suggest that SWPBS
schools were better at integrating the components the BSP, therefore producing
stronger relationships amongst the items. Differences in non-SWPBS and
SWPBS ?tul?> correlations were statistically significant for: (a) function of
behavior, (b) reinforcement, (c) reactive strategies and (d) communication. The
positive relationship observed between these variables indicates that SWPBS
schools were better at identifying the function of behavior, and were found to use
appropriate reinforcement and reactive strategies. The significant difference in
reactive strategies between the two types of schools, suggest that SWPBS schools
produced more strategies that were supportive, corrective, and assisted in
deescalating the child than reverting to punishment procedures. Non-SWPBS
schools often used traditional disciplinary methods, such as time out, following
student problem behavior.
Limitations
Several limitations existed in this study. First, the training and implementation of
the SWPBS systems were not directly observed. All of the information about
training was obtained from interviews with the training facilitators and reviews of
training manuals. Although it is assumed that all trainings were implemented as
reported, no documentation exists showing exactly what took place or who attended
the trainings. Furthermore, due to other district commitments, several promised
supports such as BSP evaluation and feedback by district trainers, were not offered
on a consistent basis to SWPBS schools.
Second, the actual composition of the collaborative teams was unknown for all of
the schools included in the study. Similar to the limitations identified by Scott et al.
(2005a), the author had no knowledge of the team’s skill level prior to training;
therefore, it cannot be assumed that the differences observed in quality of support
plans was due solely to training and presence of SWPBS. It is possible that
collaborative team members in SWPBS schools had expertise in behavioral
assessment and intervention construction preceding implementation of the school-
wide program. Additionally, the extent to which SWPBS was being implemented in
the two SWPBS schools was not formally evaluated. Future research should use the
106 J Behav Educ (2008) 17:93–110
123
School-wide Evaluation Tool (Horner et al. 2004) to evaluate the adequacy of
SWPBS implementation.
Third, although the BSP-QE has been widely used to evaluate BSPs, very little
research regarding its validity and reliability exist. Currently, there is only one
known study indicating high reliability and validity for the BSP-QE (Cook et al. in
press). The interobserver agreement reliability (Kappa) yielded for this study was
only .61. The raters found several areas of the rubric to be unclear, which led to
discrepancies in scores. In order to effectively apply the rubric extensive training is
necessary. However, the BSP-QE is the only metric available to date that
operationally defines and quantifies the quality of BSPs.
Lastly, this study focused solely on the technical adequacy of support plans and
did not address treatment integrity or effectiveness. Technical adequacy is not
analogous to treatment integrity; therefore, research needs to be conducted
investigating the link between technical adequacy and integrity. Gresham (1989)
suggested that treatment integrity may be increased by writing out the intervention
plan. The underlying assumption of this study is that a technically adequate BSP
will most likely lead to higher integrity because it clearly identifies the problem
behavior and the proposed intervention procedures. In addition, a technically
adequate plan is presumably more effective, also increasing the likelihood of
implementation with integrity.
Relevant Findings and Implications
The overall hypothesis that schools employing SWPBS systems would produce
more technically adequate BSPs than SWPBS schools was confirmed. This supports
assertions suggesting that SWPBS systems promote increased understanding of
behavior and environmental factors that can support the maintenance of the problem
behavior (Walker et al. 1996, Sugai et al. 2003). However, it is important to note
that even with higher total scores; the support plans produced at the SWPBS schools
were still evaluated as underdeveloped. These findings support previous research
suggesting that acquisition level trainings are not sufficient to produce accurate FBA
based support plans or interventions (Scott et al. 2005a, b). Scott et al. (2005a)
found that despite extensive training, functional behavior assessment teams still did
not consistently connect assessed function of behavior with logically corresponding
strategies related to function. The same problem was observed in this study. Despite
training, collaborative teams were not consistent in their integration of the BSP
components. Inconsistency across support plans may be the result of an inadequate
functional behavioral assessment or lingering attitudes related to traditional methods
of discipline.
The findings of this study suggest that ongoing staff training is necessary to
increase the quality of BSPs in both SWPBS and non-SWPBS and schools. Given
that many of the components of the BSP are interrelated, strategically focusing on
items that inform other variables may assist in increasing the overall quality and
consistency of the plan. For example, since ‘‘predictors of behavior’’ informs the
variables ‘‘predicted function of the behavior’’ and ‘‘environmental modifications,’’
J Behav Educ (2008) 17:93–110 107
123
by ensuring that staff understands how to correctly identify predictors of behavior,
the scores in subsequent areas may increase.
Future Research
This study focused solely on the quality of BSPs produced at SWPBS and non-
SWPBS schools. Future research should conduct evaluations of the functional
behavioral assessment process, in addition to the quality of resulting BSPs. Perhaps
the problems observed in this study were related to a flawed FBA process. Future
studies also should focus on determining the amount of training necessary to
produce consistency across assessments and support plans. In addition, specific
factors that lead to team success in the FBA-BSP process should be studied.
Another important measure of effectiveness is how well the BSP translates into
effective interventions in SWPBS and non-SWPBS schools. Future research should
investigate how the technical adequacy of the BSP influences the implementation
and success of behavioral interventions in the presence or absence of SWPBS.
Treatment fidelity also may be of interest. An interesting research question would be
whether treatment fidelity is higher in SWPBS schools. Also, additional research is
needed on the BSP-QE. For example, research should seek to uncover whether the
categories yielded by the BSP-QE (i.e., weak, underdeveloped, good, superior)
differentially predict the success of the intervention.
The importance of function-based interventions for children with behavioral
problems has been well established (Ingram et al. 2005; Anderson and Kincaid
2005, Horner and Carr 1997). With IDEA (1997), the federal government
acknowledged the importance of creating individualized, functionally based
intervention plans. Given these requirements, problem-solving collaborative teams
must possess the knowledge and skills to appropriately navigate through the FBA
process and gather meaningful information to produce an effective BSP. As
observed in this study and previous research, these skills are not easily acquired.
Ongoing research in training is necessary if educators are expected to use FBA and
PBS strategies to decrease problem behavior.
References
Anderson, C. M., & Kincaid, D. (2005). Applying behavior analysis to school violence and discipline
problems: Schoolwide positive behavior support. Behavior Analyst, 28, 49–63.
Benazzi, L., Horner, R. H., & Good, R. H. (2006). Effects of behavioral support team composition on the
technical adequacy and contextual fit of behavior support plans. The Journal of Special Education,40, 160–170.
Cohen, J. (1988). Set correlation and contingency tables. Applied Psychological Measurement, 12(4),
425–434.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and PsychologicalMeasurement, 20, 37–46.
Colvin, G., Kameenui, E. J., & Sugai, G. (1993). Reconceptualizing behavior management and school-
wide discipline in general education. Education & Treatment of Children, 16, 361–381.
108 J Behav Educ (2008) 17:93–110
123
Cook, C. R., Crews, S. D., Wright, D. B., Mayer, G. R., Gale, B., Kraemer, B., & Gresham, F.M. (in
press). Establishing and evaluating the substantive adequacy of positive behavior support plans.
Journal of Behavioral Education.
Crone, D. A., & Horner, R. H. (1999–2000). Contextual, conceptual and empirical foundations of
functional behavioral assessment in schools. Exceptionality, 8, 161–172.
Eber, L., Sugai, G., Smith, C. R., & Scott, T. M. (2002). Wraparound and positive behavioral
interventions and supports in the schools. Journal of Emotional and Behavioral Disorders, 10,
171–180.
Gresham, F. M. (1989). Assessment of treatment integrity in school consultation and prereferral
intervention. School Psychology Review, 18, 37–50.
Horner, R. H., Todd, A. W., Lewis-Palmer, T., Irvin, L. K., Sugai, G., & Boland, J. B. (2004). The
School-wide Evauation Tool (SET): A research instrument for assessing school-wide positive
behavior support. Journal of Positive Behavior Interventions, 6(1), 3–12.
Horner, R. H., Albin, R. W., Sprague, R. R., & Todd, A. W. (2000). Positive behavior support. In M. E.
Snell & F. Brown (Eds.), Instruction of students with severe disabilities (pp. 47–83). Columbus:
Merrill.
Horner, R. H., & Carr, E. G. (1997). Behavioral support for students with severe disabilities: Functional
assessment and comprehensive intervention. Journal of Special Education. Special Issue: Researchin Severe Disabilities, 31(1), 84–109.
Hsiao, Y., & Albin, R. W. (2000, May). The effects of functional assessment information on thebehavioral support recommendations for school personnel. Paper presented at the Association for
Behavior Analysis Convention. Washington, DC.
Individuals with Disabilities Education Act Amendments of 1997, Public Law 105–17, 20, U. S. C.
Chapter 33, Section 1415 et seq..Individuals with Disabilities Education Act Reauthorization of 2004, 20 U.S.C.§ 1400 et seq.
Ingram, K., Lewis-Palmer, T., & Sugai, G. (2005). Function-based intervention planning: Comparing the
effectiveness of FBA function-based and non-function-based intervention plans. Journal of PositiveBehavior Interventions, 7, 224–236.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data.
Biometrics, 33, 159–174.
Lewis-Palmer, T., Todd, A. W., Horner, R. H., Sugai, G., & Sampson, N. (2004). Individual studentsystem evaluation tool. Eugene: Educational Community Support, University of Oregon.
Mayer, G. R., Butterworth, T., Nafpaktitis, M., & Sulzer-Azaroff, B. (1983). Preventing school vandalism
and improving discipline: A three year study. Journal of Applied Behavior Analysis, 16, 55–369.
Mitichi, M., & Albin, R. W. (2001, May). The effects of functional assessment information on thebehavioral support recommendations of school personnel. Paper present at the Association of
Behavior Analysis Convention, New Orleans.
Positive Environments, Network of Trainers. (2003). http://www.pent.ca.gov/behBbspqe.htm
Safran, S. P., & Oswald, K. (2003). Positive behavior supports: Can schools reshape disciplinary
practices? Exceptional children, 69, 361–373.
Scott, T. M., Liaupsin, C., Nelson, C. M., & McIntyre, J. (2005a). Team-based functional behavior
assessment as a proactive public school process: A descriptive analysis of current barriers. Journalof Behavioral Education, 14, 57–71.
Scott, T. M., McIntyre, J., Liaupsin, C., Nelson, C. M., Conroy, M., & Payne, L. D. (2005b). An
examination of the relation between functional behavior assessment and selected intervention
strategies with school-based teams. Journal of Positive Behavior Interventions, 7, 205–215.
Scott, T. M., Nelson, C. M., Liaupsin, C. J., Jolivette, K., Christle, C. A., & Riney, M. (2002). Addressing
the needs of at-risk and adjudicated youth through positive behavior support: Effective prevention
practices. Education & Treatment of Children, 25, 532–551.
Sugai, G., & Horner, R. (2002). The evolution of discipline practices: School-wide positive behavior
supports. Child & Family Behavior Therapy, 24, 23–50.
Sugai, G., Horner, R. H., & Gresham, F. M. (2003). Behaviorally effective school environments. In M. R.
Shinn, H. M. Walker, & G. Stoner (Eds.), Interventions for academic and behavior problems II:Preventative and remedial approaches (pp. 315–351). Betheseda: National Association of School
Psychologists.
Sugai, G., Horner, R. H., & Sprague, J. R. (1999). Functional-assessment-based behavior support
planning: Research to practice to research. Behavior Disorders, 24, 253–257.
J Behav Educ (2008) 17:93–110 109
123
Sugai, G., Lewis-Palmer, T., & Hagan-Burke, S. (1999–2000). Overview of the functional behavioral
assessment process. Exceptionality, 8, 149–160.
Walker, H. M., Colvin, G., & Ramsey, E. (1995). Antisocial behavior in schools: Strategies and beastpractice. Pacific Grove: Brooks-Cole.
Walker, H. M., Horner, R. H., Sugai, G., Bullis, M., Sprague, J. R, Bricker, D., & Kaufman, M. (1996).
Integrated approaches to preventing antisocial behavior patterns among school-age children and
youth. Journal of Emotional and Behavioral Disorders, 4, 194–209.
Walker, H. M., & Shinn, M. R. (2002). Structuring school-based interventions to achieve integrated
primary, secondary, and tertiary prevention goals for safe and effective schools. In M. R. Shinn, H.
M. Walker, & G. Stoner (Eds.), Interventions for academic and behavior problems II: Preventativeand remedial approaches (pp. 315–351). Betheseda: National Association of School Psychologists.
110 J Behav Educ (2008) 17:93–110
123
