3/29/18

1

What’s Your Function: Making CICO Work For All Students Evan H. Dart, University of Southern Mississippi David A. Klingbeil, University of Wisconsin-Milwaukee Jonathan Jagemann, Milwaukee Public Schools Amber L. Schramm, University of Wisconsin-Milwaukee

Acknowledgements

The research reported here was supported in part by the Institute of Education Sciences, U.S. Department of Education, through Grant R305L160013 made to the Board of Regents to the University of Wisconsin-Milwaukee. The opinions expressed here are those of the authors and do not represent the views of the Institute or the U.S. Department of Education.

Check-in/Check-Out (CICO)

Widely used

Integrated into behavioral MTSS (Tier 2)

Evidence-based (Maggin et al., 2015)

Critical Components of Tier 2 Interventions

1.  Continuously available 2.  Quick, easy implemented (2-3

days) 3.  Minimal time commitment from

teachers 4.  Aligned with school-wide

strategies 5.  All staff understand the

intervention and their roles 6.  Consistent Implementation

Mitchell et al. (2016)

Check-in/Check-Out (CICO)

Four core Components

Check-in with mentor

Regular feedback from teacher

Check-out with mentor

Contingency management

Behavior Function Consequences that maintain behavior

Escape and attention are very common

Access to edibles/tangibles and automatic reinforcement less common

Functional Behavior Assessment (FBA)

Often not assessed until Tier III in behavioral MTSS

3/29/18

2

Problem

CICO is differentially effective depending on behavior function

Great for attention-maintained behavior

Possibly ineffective for escape-maintained behavior

FBA and functional modifications necessary at Tier II

Support in literature, no large scale evidence

“A promising approach includes a quick screening for function of problem behavior, such as the FACTS for students in general education…and then one of two options: (a) select the best match from a number of Tier 2 interventions, or (b) modify the intervention to provide effective support for students with escape-maintained behavior.”

- McIntosh et al. (2009), p. 90

Systematic Review of Function-Based CICO

Klingbeil, Dart, & Schramm (under review)

Systematic Review of Function Based CICO Found 11 studies (out of 48 possible) that ●  investigated a modified version of CICO ●  changes made to CICO were explicitly linked to the hypothesized function of

the participants’ problem behavior

Ten studies used single-case design, majority met WWC standards with (k = 5) or without reservation (k = 3). All 11 evaluated effects on problem behaviors, 6 also included measures of academic engagement.

Design and Intervention Information ●  6 studies conducted in Elementary schools, 3 in middle schools

●  In 8 studies, standard CICO was delivered prior to the function-based version.

●  In 8 studies, the modified version was standardized across participants

●  The four core components of CICO were retained in all studies.

Methods of Determining Behavior Function ●  All 11 studies included a teacher interview, 10 used the Functional

Assessment Checklist for Teachers

●  10 studies incorporate direct observation data in their FBA

○  Conducted three to six, 20-minute observations

●  Rough estimate…about 2 hours per student

3/29/18

3

Hypothesized Behavior Functions

●  Reported in all 10 single-case design studies (n = 31)

●  Six students had multiple functions reported

Escape from tasks/demands = 21 Access to peer attention = 9 Access to adult attention = 7 No other hypothesized functions reported

Modifications to Standard CICO Procedures ●  Few modifications to structured feedback from teachers or Daily Progress

Report forms

●  One study targeting escape-maintained behavior taught students how to appropriately request breaks throughout the day

●  Majority of modifications occurred to some aspects of check-out.

○  More frequent check-outs (and access to contingent reinforcement)

○  Checking out with peer

○  Adding supplemental assignment that could be escaped from (Kilgus et al., 2016)

Additions to Standard CICO procedures ●  Modifications to academic tasks (shortened, easier)

●  Time to complete work during school day

●  Preferential seating

●  Precorrection of inappropriate behaviors

●  Incorporation of academic instruction along with CICO

Modifications to Reinforcers Adult attention: ●  Increased frequency of contingent praise, increased frequency of check-outs,

earned extra time with preferred adult

Peer attention: ●  Sit with peers, earned free time with peers, eat lunch with a peer, bring peer

to check-out. Escape ●  Access desired activity contingent on task completion, request breaks, earn

break passes, finish work at home.

Summary ●  Approximately 90 to 120 minutes to estimate behavior function

●  Most studies used standardized modifications to CICO, after standard CICO was ineffective

●  Standard core components of CICO all maintained

●  Reinforcers often linked to behavior function

“A promising approach includes a quick screening for function of problem behavior, such as the FACTS for students in general education…and then one of two options: (a) select the best match from a number of Tier 2 interventions, or (b) modify the intervention to provide effective support for students with escape-maintained behavior.”

- McIntosh et al. (2009), p. 90

3/29/18

4

Clustered RCT of Function-Based CICO

Purpose Modify CICO to address behavior function

Evaluate compared to traditional CICO

Setting

Milwaukee Public Schools

●  161 schools ●  7,436 staff members ●  76,933 students ●  20.5% students with disability ●  81.4% economically disadvantaged ●  14.5% English learners

MPS Demographics PBIS Rollout Model

3/29/18

5

Documented Behavior Interventions Teaming Systems

Learning Team ●  Academics Tier 1

PBIS Team ●  Behavior/Attendance Tier 1

Building Intervention Team (BIT) ●  Academics & Behavior/Attendance Tier 2 and 3

Tier 2 Entrance Criteria Recommendations

●  Students with 1+ ODRs in 20 school days ●  Students with 1+ suspensions in 90 school days ●  Teachers use Risk Behavior Screening

○  When ODR data is not there for particular students ○  For internalizing behaviors

Tier 2 Intervention Recommendation Flow

1. Check-In/Check-Out (CICO) 2.  Individualized Check-In/Check-Out 3. Social Academic Instructional Group (SAIG) 4. Behavior Assessment/Intervention Plan (BAIP) 5. Consideration for a Tier 3 Intervention

Daily Progress Report

●  All behavior interventions monitored with a Daily Progress Report (DPR)

●  Data entered into Infinite Campus (IC)

●  Success criteria receiving a 80% on their DPR 80% of the time for 4 consecutive weeks

Example Daily Progress Report

3/29/18

6

Check-In/Check-Out (CICO) - Crucial Elements

1.  Check-in with assigned adult upon arrival to school 2.  Carry a DPR to progress monitor 3.  Meet with teacher in each class/ time period 4.  Check-out at end of day 5.  Acknowledgement 6.  Monitor student’s progress 7.  Communication to families

Individualized CICO

Same as CICO framework except make one of the following adjustments for an individual student: –Change location of morning greeting –Change time of morning greeting –Adjust who holds the DPR (student, teacher, etc) –Personalized goal on DPR –Add additional meetings with greeter –Change who is morning greeter

Function-Based CICO

Finding Balance ●  Minimize changes to existing practices

○  Maintain existing procedures for identifying students for CICO

○  Continued to include 4 core components of CICO

●  Considered time and resources necessary for any potential modification

Key Modifications One page indirect FBA adapted from the FACTS

Completed prior to implementation, reviewed by Behavior Intervention Team member

Functionally relevant reinforcer list based on hypothesized function

Teacher Referral Form - Question 1 (of 3)

3/29/18

7

Teacher Referral Form - Question 2 (of 3) Teacher Referral Form - Question 3 (of 3)

Key Modifications One page indirect FBA adapted from the FACTS

Completed prior to implementation, reviewed by Behavior Intervention Team member

Functionally relevant reinforcer list based on hypothesized function

Methods

1. Can schools implement function-based CICO with fidelity? 2. Is it more effective than standard CICO?

3/29/18

8

Design

Randomly sampled 50 schools from larger group of schools (J = 98)

●  K-5 or K-8 format ●  Followed traditional calendar (not year-round schools) ●  Did not have another social-emotional initiative on-going

Half of the 50 schools were randomly assigned to the function-based CICO condition.

Training Created separate on-demand training modules for the Behavior Intervention Teams and “Greeters”.

Provided training manuals for Building Intervention Teams and Greeters.

Provided face-to-face training to school social workers.

Dependent Variables

1.  Students’ average Daily Progress Report scores (0% to 100%)

2.  Office Discipline Referrals

Fidelity Measure

We added 2 items related to function-based CICO

Fidelity Measure

12 Items related to standard CICO

Baseline Equivalence (2015-2016)

There were 933 students in grades K through 5 in CICO. Control schools had fewer students in CICO: 435 to 498

3/29/18

9

Baseline Equivalence (2015-2016)

Treatment and Control Schools Equivalent on: ●  Demographic variables ●  Math or reading test scores ●  Attendance ●  PBIS Tiers 1 or 2 fidelity ●  ODRs, Suspension Rate

Baseline Equivalence (2015-2016)

Treatment and Control Schools Equivalent on: ●  CICO Enrollment (t = .42, p = .42; d = .23)

●  Average DPR score (t = .34, p = .731, d = .02)

●  Average ODRs for students on CICO (t = -1.10, p = .27; d = -.07)

Baseline Equivalence

Approximately 34.4% of the variance in student’s average DPR scores while on

CICO was attributable to between-school differences.

Data Analysis

Treatment at school level, requires multilevel modeling

Student Level covariates were all grand-mean centered, to control for their effect on level 2 variables

Grade and attendance treated as continuous

Race (White)

Sex (Male)

Eligibility for Free or Reduced Price Lunch (Not eligible)

Special Education Status (Not Eligible)

Grade

Attendance

Data Analysis Level 2 covariates: ●  % receiving FRL ●  % eligible for SPED ●  Attendance rate ●  % Proficient in Reading ●  Tier 1 fidelity ●  Tier 2 fidelity ●  % of students on CICO ●  CICO fidelity

Level2treatmentindicatorenteredinto

modellast.

d = 0.013 Effect of assignment to function-based CICO controlling for

other level one and level predictors...

3/29/18

10

Results 1.  Is a function-based

CICO intervention feasible?

Three schools did not attempt the modified version (intent-to-treat group)

Four schools rated their fidelity on the function-based CICO components as 0 (out of 4)

11 Schools > 80%

Results

2. Is function-based CICO more effective than the standard CICO procedures?

No significant difference between groups!

Difficult to tease apart lack of effect from lack of fidelity...

Feedback Sessions 1.  Present 2015-2016 data (baseline) 2.  Review 3 key changes to function-based CICO 3.  Review school’s reported CICO fidelity 4.  Present 2016-2017 data 5.  Discuss changes from 15-16 year. 6.  Ask for feedback on:

a.  Teacher referral list b.  Reinforcer list and process of delivering them c.  Training d.  Any other issues

Issues and Challenges

Critical Components for Tier 2 ●  All staff understand the intervention and their roles ●  Quick, easily implemented (2-3 days) ●  Minimal time commitment from teachers ●  Aligned with school wide strategies ●  Consistent Implementation

Mitchell et al., 2016

Staff Understanding of Intervention and Role

●  Confusion led to poor implementation

●  Professional development and training

●  Referral forms

3/29/18

11

Teacher Referral Form - Question 1 (of 3) Teacher Referral Form - Question 2 (of 3)

Teacher Referral Form - Question 3 (of 3)

Staff Understanding of Intervention and Role

●  Need for more training

●  Behavior function

●  Function-based components

Quick, Easy Implementation

●  Referral form caused delay for intervention implementation

●  Support staff helped minimize delay and increase ease of implementation

●  Function-based components provided teachers time to reflect on student behavior

Minimal Time Commitment From Teachers

●  Referral form added time spent for teachers

3/29/18

12

Minimal Time Commitment From Teachers

●  Reinforcement feasibility

Alignment with School Wide Strategies

●  Reinforcers not aligned with school philosophy

●  Reinforcers not aligned with teacher beliefs/practices

●  Well established PBIS systems and standard CICO increased feasibility in some cases

Consistent Implementation

●  Administration support and follow through

●  School Psych and School Social Work support

Need to balance immediate implementation with adding function-based components

Summary

3/29/18

13

Implementation Issues

Many treatment schools did not implement Process identified as not feasible Implementation science critical for future success

Standard to Function-based Model

Feedback suggesting standard should be tried first

If unsuccessful, try function-based

“Tier 2.5 intervention”

May delay effective services

Conclusions and Implications

Function-based CICO makes theoretical sense Effective on small scale in literature Practical barriers must be addressed for scale-up

Thank you! Questions?

Evan Dart – evan.dart@usm.edu David Klingbeil – davidak5@uwm.edu Jon Jagemann – Jagemaj@Milwaukee.k12.wi.us