using data for effective team meetings within pb4l rob horner university of oregon

Using Data for Effective Team Meetings within PB4L

Rob HornerUniversity of Oregon

www.pbis.org

Goals• Overview of Team-Initiated Problem Solving (TIPS)

• Using data to define a problem with precision

• Building comprehensive solutions

• Using data to monitor, evaluate and adapt

Implement Solution with High Integrity

Identify Goal for Change

Identify Problemwith

Precision

Monitor Impactof Solution and

Compare against Goal

Make SummativeEvaluationDecision

MeetingFoundations

Team-Initiated Problem Solving (TIPS II) Model

IdentifySolution and

CreateImplementation

Plan with Contextual Fit

Collect and Use

Data

What, Who, When, Where, and Why, and How Often?

How do we want the problem to

change?

What are we going to do to bring about

desired change?

Did we implement with fidelity?

Has the problem been solved?

Maintain, Adapt, Stop?

8

Problem Solving• What is a problem?

o Any difference between what exists and what is desired.

• What its the difference between problems that are significant enough to warrant a solution and those that can be tolerated?o Social values o Academic expectationso Political and organizational expectations

TIPS II Training Manual (2014) www.uoecs.org

10

Problem Solving• Define Problem

o Identify problems earlyo Identify problems with PRECISION before building solutions

• Establish Goalo Where do we want to be

• Build Solutions that are comprehensive and locally relevant

• Prevent, teach, extinguish, reward, monitor• Select solutions based on local strengths, culture, organization

• Build Action Plan for Implementing SolutionsTIPS II Training Manual (2014) www.uoecs.org

11

Precise Problem Statement• How often is a behavior occurring?

• Where is the behavior most likely to occur?• What is the behavior of concern?• Who is doing the behavior?• When is the behavior most likely?

• Given Who, What, Where, When:

oWHY does the behavior keep happening in this context?


Start with Primary Problem Statements

Look at the Big Picture. Then use data to refine the problem to a Precise Problem Statement.

Move to Precise Problem Statements

Office discipline referrals for 3rd graders are above national median for schools our size.

Referrals for physical aggression among third grade students from 11:30-12:30 in the cafeteria are occurring at 2-3 per day and increasing over time. It is believed that this is happening due to insufficient supervision and access to peer attention.

Precise Statement?

PreciseNot Precise

Indicates a difference between what is

happening and what is desired.

What, Who, Where, When, Why, and How

Often

Not Precise or Not Precise?

• Students on bus #43 are physically aggressive to gain peer attention at least 5 times per bus ride

• Three 5th grade boys are name calling and touching girls inappropriately during recess in an apparent attempt to obtain attention. This is occurring at least 5 times a week.

• Boys are engaging in sexual harassment.

Precise

Not Precise

Precise

Not Precise or Not Precise?• Darin uses inappropriate language with a high

frequency in the presence of both adults and other children. This is creating a sense of disrespect and incivility in the school.

• Students are being aggressive on the bus.

Not Precise

Not Precise

Not Precise or Not Precise?

Minor disrespect and disruption are occurring at a rate of 1.5 per day/month, ( increasing over time) and are most likely during the last 15 minutes of our block periods when students are engaged in independent seat work. This pattern is most common in 7th and 8th grades, involves many students, and appears to be maintained peer attention. Precise

Not Precise or Precise?

• Danielle is 9 years old and refuses to follow parental requests to pick up toys, help with dishes, or do homework. She will sulk and pout until we get frustrated and send her to her room.

• Danielle is 9 years old and 3-4 times per day refuses to follow parental requests to do minor tasks like pick up toys, help with dishes or do homework at home. She will sulk and pout. The behaviors are maintained by escape from non-preferred activities.

Not Precise

Precise

Your Turn to practicea. Make up a PRIMARY problem statement

b. Transform that description into a PRECISE problem statement.• Who• What• Where• When• Why• How Often

Write down Primary and Precise statements Be ready to share your hypothetical example with group

5 minutes

19

Using Data to Define Problems with

Precision

• Major ODRs per day per montho Levelo Trendo Pattern (peak months?)

• Compare with national median (or average)

• Compare with local norms


SWIS summary 2013-14(Majors Only)

4858 schools; 2,476,989 students; 3,054,592 ODRsGrade Range Number of

SchoolsMean Enrollment per school

MeanODRs per 100 stud/ school day

Median ODRs per 100 per stud/ school day

25th PercentileODR/100/ school day

75th PercentileODR/100/ school day

K-6 3021 450 .31 (.40) .20 .10 .38

6-9 894 613 .48 (.67) .33 .17 .58

9-12 452 817 .57 (1.04) .36 .17 .69

PreK-8 273 418 .39 (.41) .27 .13 .51

PreK-12 76 341 .73 (1.41) .36 .16 .58

Monitoring Data;Major ODRs per day per

month

Although the months are different, an average referral per day per month count allows for true comparisons.

Average Referrals Per Day Per Month

Transforming Data into

Information• Tell the story that is present in your data.

• How often are problems occurring?• Is the trend increasing, decreasing, stable over time?• How does this level compare to last year?• Are there months with unusually high levels (peaks)?

• How do our patterns compare with others?• How do our patterns compare with prior years?• How do our patterns compare with what our students, families

and staff believe is appropriate/acceptable?

Do we have a Problem?• Review ODR per day per month

o What is the overall level?o What is the pattern?o What is typical?o What is possible?o What is needed?

• If there is a problem… then define with precision

Elementary School with 150 Students

24

Questions to Ask of the Data

What is happening?What is typical?

What is possible?What is needed?

What is the narrative?Questions to Ask

What is happening?

What is typical?

What is possible?

What is needed?

75th PercentileMedian25th Percentile

Elementary School Example

Median

Middle School Example

Median

765 Students

High School Example

Median

Multi-Year Example

Median

Year OneYear Two

What When

Who

Why

Designing Effective Behavior Support

Where

Define Problems with precision

TIP

S I

I T

rain

ing

Man

ual (

2014

)

ww

w.u

oecs

.org

31

How Often

http://www.uoecs.org/

Primary Problem Statements• Data examined at the SW level or with just one graph allow for “primary”

problem statements only, such as – Our school’s…o Average referrals per day per month for All Referrals & Minors are above

the 75th percentile for 5 of 6 monthso Average referrals per day per month for Majors show an increasing trendo Referrals for Defiance totaled 77 instances of the last three monthso Referrals in Classrooms have been too high for the last two monthso Referrals for each month of this year are higher than for the corresponding

month of the previous year TIP

S I

I T

rain

ing

Man

ual (

2014

)

ww

w.u

oecs

.org

32


ImplementSolution withHigh Integrity

Identify Goalfor Change


Precision

Monitor Impact ofSolution and

Compare AgainstGoal


MeetingFoundations

Team-Initiated Problem Solving II(TIPS II) Model

Identify Solution and Create ImplementationPlan with Contextual

Fit

Collect and Use Data

What is the problem?Who? What? Where? When? Why?

TIP

S I

I T

rain

ing

Man

ual (

2014

)

ww

w.u

oecs

.org

33


SWIS Big 4 for October 1, 2011 through December 31, 2011

ClassroomPlaygroun

d

Where?

Questions:1. What location(s) are associated with the most

ODRs?2. Sort by “structured” settings and “non-

structured” settings(Classroom & Gym vs. Commons, Cafeteria,

Hall, Playground)

SWIS Big 4 for October 1, 2011 through December 31, 2011

Defiance

What Behavior(s)

Questions:1. Are most common behavior problems (a) Student-Student, or (b) Adult-Student related?2. Are problem behaviors MAJOR or MINOR or BOTH?

What Behavior(s)

Who

Questions:1. Are there many, a few, or one student associated

with the problem?2. Are there more students emerging?

11:45-12:00

When?When?

Questions:1. Are problem behaviors more likely at some times

of the day?2. What is happening during periods when

problems are most likely?

When: Middle School

WHY?—the hardest question• What is perceived as maintaining the problem behavior?

• Always assess motivation AFTER you have defined:o Who?o What?o Where?

• You always ask WHY students misbehave in the context.

• Look for “primary” motivation if there are multiple possibilities.

Why?• Obtain positives

o Attention from peerso Attention from adultso Personal satisfactiono Physical objectso Preferred activity

• Avoid negativeso Avoid peerso Avoid adultso Avoid worko Avoid undesirable activitieso Avoid undesirable objects/food/ sounds/ etc

Don’t Get too Big

------------

------------

----

Revenge

Prestige

Power

Status

Happiness

Avoid Work

Why?Why

Why• Motivation for many elementary students engaging in disruption

in the classroom

Why• Motivation for middle school students’ non-compliance and

insubordination in the classroom




Precision




MeetingFoundations





Collect and Use

Data

Identify Solution and Create Implementation Plan with

Contextual FitWhat and When




Precision




MeetingFoundations





Collect and Use

Data

Identify Solution & Create Implementation

Plan with Contextual Fit Objectives

o Use solution option categories to brainstorm solution actions for your precise problem statement

o Define the scope of necessary solutions• SW, specific setting, grade/group/individual student

o Define an action plan for each solution action• Who does what by when

TIP

S I

I T

rain

ing

Man

ual (

2013

)

ww

w.u

oecs

.org

56


How to Build a Solution Plan• Use solution elements to guide discussion• Brainstorm options for solution elements• Select a set of actions (one plan) that

o Require the least amount of effort that will produce desired changeso Strengthen what you already do well o Fit with the precision statement and priority o Are efficient & feasibleo Are likely to have the desired impact

TIP

S I

I T

rain

ing

Man

ual (

2013

)

w

ww

.uoe

cs.o

rg

57


Solution Action Elements Solution Action Elements Defined

Prevent Focus on prevention first. How could we reduce the situations that lead to these behaviors?

Teach How do we ensure that students know what they SHOULD be doing when these situations arise?

Reward How do we ensure that appropriate behavior is recognized?

Extinguish How do we work to ensure that problem behavior is NOT being rewarded.

Correct How will you correct errors?

Safety Are additional safety precautions needed?

Solution Implementation Plan Elements

TIP

S I

I T

rain

ing

Man

ual (

2013

)

ww

w.u

oecs

.org

58


Solution Action Elements Possible Generic Solution Actions

Prevent What can we do to prevent the problem?

Adjust physical environment.Define & document expectations and routines.Assure consistent & clear communication with all staff.

Teach What do we need to teach to solve the problem?

Explicit instruction linked to school wide expectations.Teach what to do, how to do it and when to do it.Model respect .

Reward What can we do to reward appropriate behavior?

Strengthen existing school wide rewards.Include student preferences.Use function-based reinforcers

Extinguish What can we do to prevent the

problem behavior from being rewarded? Use ‘signal’ for asking person to ‘stop’.Teach others to ignore (turn away/look down) problem behavior.

Correct What will we do to provide corrective feedback?

Intervene early by using a neutral, respectful tone of voice. Label inappropriate behavior followed by what to do Follow SW discipline procedures

Safety Do we need additional safety precautions?

Separate student from others if he/she is unable to demonstrate self-control.Make sure adult supervision is available.

How much effort is needed to bring about desired change?

• When deciding how many solution elements to implement consider: o Priority for change

• Severity of the problem• Intensity of the problem• Frequency of the problem• Potential safety concerns

o Impact of solution implementation

• Confidence that solution implementation will make a differenceo Feasibility/ Availability of resources

• Professional development, support, time, tools• Existing skills and capacities of implementers• Is this doable? Is it the most efficient approach?

TIP

S I

I T

rain

ing

Man

ual (

2013

)

ww

w.u

oecs

.org

60


Is Solution Appropriate for the context?• Solution actions should fit the context

o Consider your school’s culture & expectationso Consider your school’s improvement plan goalso Consider the school calendaro Consider impact of solution implementation on staff work load &

classroom instruction schedules• Implementation time & effort need to be worth it by meeting the goal

o Consider the function of the problem behavior

TIP

S I

I T

rain

ing

Man

ual (

2013

)

ww

w.u

oecs

.org

61


Your Turn• Too many ODRs (3.4 per day) for aggression and fighting

during 3rd period recess by many boys (and some girls) wanting access to the playground equipment.

• Prevent? Extinction?• Teach? Correction?• Reward? Safety?

• Data System Needed?

Your Turn• Precise Problem:

• Solutiono Prevent Extinctiono Teach Correctiono Reward Safety

o Data System

Action PlanningWhat will be done Who will do it By When

Data Collection1. Did we implement the solution?2. Did the solution benefit students?

Teacher Completed Fidelity ScalesEstablish a fidelity check routine that relates to

Implementation• A 1-5 scale is used for questions • Up to 3 questions per week• Share scale as a poster in the faculty room, an electronic

survey, or paper and pencilWe agreed to provide “high-five greetings” to all students entering your class on time in the morning this week? How did we do?

1 2 3 4 5No Yes

How many days (we agreed to 4) during the week did you review with students the procedures for

passing in the hall?

1 2 3 4 5✓ ✓✓✓ ✓✓✓

Summary• Solve Problems• Build “meeting foundations” (roles, minutes,

systems)• Use data to define problems with precision

o Define problems early

• Use data to build solutions• Include “Action Plans” with each solution• Use data to ask (a) did we implement, and (b) did

the plan benefit students?

using data for effective team meetings within pb4l rob horner university of oregon

Documents

problem o

goal o

precision o

data slide

o respectful

schedule o

problems early o

use data