The Nuts and Bolts of Data Collection

and AnalysisPositive Behavioral Interventions and

Supports

SYST

EMS

PRACTICES

DATA

OUTCOMES

Joan Ledvina Parrjparr@bcps.org 410-887-1103

SYST

EMS

PRACTICES

DATA

SupportingStaff

Behavior

SupportingDecisionMaking

SupportingStudent Behavior

Process for Supporting Social Competence and Academic Achievement

OUTCOMES

School-wideClassroomNon-classroomIndividual

Define behavior expectationsSpecify routinesTeachAcknowledgeCorrectFollow up and feedbackReinforcementGeneralization

Office Discipline ReferralsStaff InputAcademic ProgressAttendance

Are you….

• Swimming in data?

• Drowning in data?

• In a maze of data and can’t

find your way out?

• Lost and don’t know what to do?

Data Needs to be Your Friend

Without data, you are just another person with an

opinion…..

Data Can Be Your Friend

• Key Features of Data Systems that Work:– Data are accurate

– Data are easy to collect

– Data are used for decision making• Data must be available when decisions need to

be made• People who collect the data must see the

information used for decision making

Objectives—to Help you Find your Way

• Review the types of data you will

want to consider

• Review how to access STARS data

and PBIS reports

• Learn how to use the Big 5 Generator

• Understand how to use data to

make decisions

Where to Begin……

Review End of the Year Data Reports

• Office referrals and suspensions– PBIS reports in STARS

• PBIS Self-Assessment (Staff Survey)• Benchmarks of Quality

– PBIS team completed

• Analysis of your School’s Resources– Triangle of Tier 1, 2, and 3 interventions

• Final Step: Develop an Action Plan• Bonus Step: Learn about the Big 5 Generator

What Data Should be Collected for Decision Making?

• Office Discipline Referrals– Measure of overall environment—– Referrals are affected by:

• Student behavior• Staff behavior• Administrative context

• Teacher Referrals – (Minors, Focus Room, etc.)

• Suspensions• Attendance

Office Discipline Referral Process/Form

• Coherent system in place to collect office discipline referral data

• Faculty and staff agree on categories• Faculty and staff agree on process• Office Discipline Referral Form should

include all needed information:– Name, Date, Time, Staff Referring– Problem Behavior, Location, Motivation– Others Involved, Detailed Information as needed

STARS Discipline Module

• Data Entry– Office Discipline Referrals (Majors)– Teacher Referrals (Minors)

• Reports– PBIS Reports: The Big 5

• Location• Problem Behavior• Time of Day• Student• Referrals per Day by Month

– Variable Reports

STARS—3 of the Big 5 Reports

• Location• Problem Behavior• Time of Day

• Date Range– From ……To

• Referral Type– All (major and minor)

– Major

– Minor

• Show All Categories with _____ or more referrals

STARS—4th of the Big 5 Reports

• Student • Date Range• Referral Type• Show All Students with

_____ or more referrals• Sort on:

– Student name

– Age

– Grade

– Section

– Major

– Minor

Student Report—Factors to Sort on

• Student Name—Lists students alphabetically• Age—Lists by age of students• Grade—Lists by grade of students• Section—Lists by section of students• Majors—Lists students’ names in order according to

their number of majors• Minors—Lists students’ names in order according to

their number of minors** You can limit your list by requesting only students with

_____ or more referrals or by asking for only majors or only minors

STARS—5th of the Big 5 Reports

• Average Referrals per Day by Month– Provides a chart with the number of days school

was in session, number of referrals (majors and/or minors), as well as the calculation of Referrals per Day by Month

The Big 5 Generator

• Excel spreadsheet• Record STARS data by Month• Label each document by Month• Cut and paste graphs into document

– Location– Problem Behavior– Time of Day– Students– Average Referrals per Day by Month

Analysis: What systems are problematic?

• Referrals by location?– Are there specific problem locations?

• Referrals by problem behavior?– What problem behaviors are most common?

• Referrals by student?– Are there many students receiving referrals or only a

small number of students with many referrals?

• Referrals by time of day?– Are there specific times when problems occur?

Referrals by Problem Behavior

Office Disciplinary Referral Problem Behaviors

020406080

100120140

October

Referrals by Problem Location

05

101520253035

Classro

om

Playgro

und

Commons

Hall/B

reezew

ay

Cafeter

ia

Bath/R

estro

omGym

Librar

y

Bus Load

ing

Parking L

otBus

Spec

ial Ev

ent

Off-Campus

Stadiu

mOffice

Lock

er Room

Other Lo

cation

Unknown Lo

cation

Office Disciplinary Referral Problem Locations

Series1

Referrals by Time of DayOffice Disciplinary Referrals By time

0

1

2

3

4

5

6

7

8

9

10

7:00 7:30 8:00 8:30 9:00 9:3010:0010:3011:0011:3012:0012:301:00 1:30 2:00 2:30 3:00 3:30 4:00 4:30

October

What Does the Data Tell Us

If many students are making same mistake, consider changing system….not students

Start by teaching, monitoring & rewarding…before increasing punishment

Guidelines:

IF: Focus On:

More than 35% of students receive 1 or more referrals

Average referrals per student > 2.5Schoolwide Systems

More than 35% of referrals come from non-classroom settings

More than 15% of students who receive a referral are referred from non-classroom settings

Non-Classroom Systems

More than 50% of referrals are from classroom settings

More than 40% of referrals come from less than 10% of the classrooms

Classroom Systems

If 10 or more students have 5 or more referrals Targeted Group Interventions

Less than 10 students with 10 or more office referrals

Less than 10 students continue w/referrals after targeted interv

Small number of students destabilizing overall functioning

Individual Student Systems

Looking at the Overall Numbers

• What are the trends from year to year?

Create Graphs from the Big 5 Generator, Excel or Power Point Applications

0

200

400

600

800

1000

1200

2004-2005

2005-2006

2006-2007

2007-2008

2008-2009

2009-2010

2010-2011

Office Referrals per Year

Looking at the Overall Numbers

• What are the trends within the year?

Create Graphs from the Big 5 Generator, Excel or Power Point Applications

0

5

10

15

20

25

30

35

A S O N D J F M A M J

Average Ref per Day SY 0809

Average Referrals per Day by Month

Another View of Summary Data

• Consider the triangle percentages….– Percentage of students receiving 0-1 referrals (green)– Percentage of students receiving 2-5 referrals (yellow)– Percentage of students receiving 6 or more referrals

Comparing Percentages of Students in the Green, Yellow, and Red Zones

72.0%

18.0%

10.0%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

School Name

Red

Yellow

Green 82.0%

14.0%4.0%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

School Name

Red

Yellow

Green

Next Steps: Tracking Data

Comparing academic and behavior data

1-5%

5-10% 5-10%

80-90% 80-90%

Below grade level 6+ referrals

Approaching grade level

2-5 referrals

On or above

grade level

0-1 referral

1-5%

Classroom Performance:

State-Wide Assessment

:Basic

Borderline

Proficient or

Advanced

Discipline:

STUDENT NAME

GRADE

Days Absent Days Tardy # Referrals CLASS LEVEL

MSA GRADES

Last Report

CurrentLast

ReportCurrent

Last Report

CurrentReading Math

Reading

Math 1 2 3 4

                               

                               

                               

                               

                               

                               

                               

                               

                               

                               

                               

                               

                               

Red Zone Students: (6+ referrals)

Grade

# Majors

# Minors

Total referrals

Suspensions Interventions in place

  PreK 0 6 6 0 Parent conference

  K 1 5 6 1 SST, beh chart, Couns., gradual entry

  1 3 18 21 3 IEP, FBA, beh chart, CICO,

  3 0 7 7 0 CICO, SST, beh chart, couns.

  3 0 10 10 0 CICO, beh chart,SST, couns

  4 1 7 8 1 SST, FBA, CICO, Couns.

  5 0 12 12 0 IEP, FBA, couns interv., beh chart

             

Yellow Zone Students: (2-

5 referrals)

Grade

# Majors

# Minors

Total referrals

Suspensions Interventions in place

  PreK 0 2 2 0 SST, 504, Couns. Interv.

  K 0 3 3 0 Parent contact, SST requested

  1 0 4 4 0 parent conference, beh chart

  2 0 3 3 0 IEP, couns, beh chart

  3 1 2 3 1 CICO, Couns., beh chart, referred to IEP

  4 0 2 2 0 Parent contact

  5 1 2 3 1 CICO, beh chart, parent contact

Analysis:Review Results of Self-Assessment

• Staff completed the Self-Assessment at the end of the year• Note: change of website: www.pbssurveys.org is now www.pbisassessments.org

• Generate the reports using the handout provided– Individual Summary Report (one year)– Comparative Summary Report (compares years)– Analysis of Schoolwide System Report (critical

features)– Individual Item Scores (item analysis)

Sample of Item Analysis80

%15

%5 % 10. School administrator is an active participant on the behavior support team. 16

%20

%64

%

62 %

36 %

2 % 11. Data on problem behavior patterns are collected and summarized within an on-going system.

13 %

34 %

53 %

59 %

36 %

5 % 12. Patterns of student problem behavior are reported to teams and faculty for active decision-making on a regular basis (e.g. monthly).

14 %

34 %

52 %

72 %

28 %

0 % 13. School has formal strategies for informing families about expected student behaviors at school.

19 %

32 %

49 %

41 %

34 %

25 %

14. Booster training activities for students are developed, modified, & conducted based on school data.

22 %

45 %

33 %

Analysis:Benchmarks of Quality

• Review the BOQ completed by your team in April

• Use the BOQ Planning Form (in your handouts) to guide your thoughts about improvements needed

Consider your School’s Resources

• Complete the Triangle Activity (included in your handouts)

• Consider what your school provides students at the Tier 1, Tier 2, and Tier 3 levels

• Consider what your school might want to implement to provide greater support for students

School counseling services Second Step FBAs/BIPs

School health servicesGroups targeting social skills, friendship, and/or anger management skills

Student Intervention Plans

UNIVERSAL

TARGETED

INTENSIVE

Schoolwide PBIS Check-in/Check-out

Section 504 Plans and/or IEPs Health Education Voluntary State Curriculum Behavioral contractingAlternative programs Character Education School-based mental health services

Bullying Prevention

Expanded School Mental Health Initiatives and Interagency Partnerships

IEP

Team

* Student S

upport Team

* PB

IS T

eam

SIT

Tea

m *

Gra

de L

evel

/Tea

cher

Tea

ms

Have a Vision (aka Your PBIS Action Plan)

When you know where you are headed, you can guide students toward their own success.

Without a destination in mind and plan how to get there, you may arrive at a place you don’t want to be.

Develop your School PBIS Action Plan

• Use your data• Know your resources • Consider your plan for additional interventions• Map out your plan (PBIS Action Plan included in

your handouts—electronic version available)• Develop a schedule of meetings• Delineate roles and responsibilities• Plan for data analysis and discussion• Strengthen your team with multiple levels of

talent

Final Considerations

• We can’t “make” students learn or behave

• We can create environments to increase the likelihood students learn and behave

• It is all about providing and supporting the systems so that adults can change their behavior to implement the practices that will bring about change in student behavior

SYST

EMS

PRACTICES

DATA

OUTCOMES

Some Final Thoughts

on the Road to Success

All of us will have set-backs on the journey

Allow yourself plenty of time to get there

Remember to bring the kids along

Remember, Building a PBIS Continuum is a Marathon

not a Sprint

For Additional Information

• Joan Ledvina Parr– PBIS Facilitator / School Psychologist

– jparr@bcps.org 410-887-1103

• Debely Fenstermaker– PBIS Coach / School Psychologist

– dfenstermaker@bcps.org 410-887-7566

• Margaret Grady Kidder– PBIS Coordinator / Coordinator of Psychological Services

– mkidder@bcps.org 410-887-0303