Using the Self Assessment to Expand and Sustain SW-PBS

Using the Self Assessment to Expand and Sustain SW-PBS

Susan Barrett

www.pbis.org

www.pbismaryland.org

Outcomes

Build Action Plan Using Self Assessment

State, LEA, cluster???

Introduce Topic

Discuss

Action Plan

3

PBS Systems Implementation Logic

Leadership Team

Management Team

Funding

Marketing

Visibility

Political Support

Training

Coaching

Evaluation

Active Coordination

Local School Teams/Demonstrations

Braiding

Initiatives

3

Implementers Blueprint

Self Assessment

More like guidelines

Provides common language and framework

Outcomes

Use revised self assessment to build local infrastructure

Discussion

Next Steps to ensure we sustain and move forward for expansion (more schools or up the triangle?)

Core Support Program:

Provided to all, intended to reach most.

Continuum of Supports

The required resources to address the problem increases

The need to enhance environmental structures increases

The frequency for collecting and acting upon information increases

As the magnitude of the problem increases.

5

Susan

Leadership Team

Language is important

e.g. OISM, MISI- Stayin Alive

Integration Teams? Who are the players?

Do you have folks who can assign dollars to a budget?

change policy like job descriptions, code of conduct?

Do you have Community Partners?

Management team- to do the day to day activities, planning, visiting schools, etc

Roles and Responsibilities may change over time depending on implementation phase

Establish a Partnership Agreement

Local Coordinator Identified

Implementation Phase should determine FTE

Access to Ongoing Training and Technical Assistance Support

Meeting with other coordinators is critical!!

Local Management Team

Creating protocols/standards (State v Local)

Funding

Partnership Agreements

Folks in charge have to understand 3-5 years, systems change

MD Example

Blending Initiatives

Social Marketing

Economic Benefits, Serendipity( TN example)

Grants

Be careful what you wish for

Marketing and Visibility

Who are your stakeholders?

Do you have a spokesperson?

Using the data to create newsletters, presentations, fact sheets, elevator business cards- important you can get access to what you need to make your case on the fly!!

www.pbismaryland.org

Colorado- Marketing exemplar

Be Careful

10

Multiple levels of Visibility

State and Local Level: Presentations, Trainings, Stakeholder meetings, Interagency efforts, (Transformation; Mental Health Integration; Wraparound)

Multiple Media: Visual, Face to Face, Written, Website

Multiple Audiences: School Administrations and Instructional Leaders; University staff; Legislators, Potential alternative funders; State and Local Political appointees; Juvenile Justice; Vendors in the System of Care; Parent and other advocacy organizations; Community Members

Evaluation

What are your questions?

Do you have the tools to answer?

Can you get the answer quickly?

Easy, Efficient, Relevant

Economic Benefits

Behavior

Achievement

Regular Feedback to all Stakeholders- MD example

PBIS Maryland Partner Meeting

December 10, 2007

Management Team meets with Assistant Superintendent of Student Services and Special Education

Jerry

Jerry

91% Rate of Return

Susan

Evaluation Question: Are schools in beginning or advanced stages of implementation?

Data Source: Implementation Phases Inventory

Overall, the IPI data suggest a relatively advanced level of self-reported implementation among the schools in Maryland

Susan

Jerry, Can you include percentages? Thanks!

Evaluation Question: Are schools implementing School-wide PBIS?Data Source: Team Implementation Checklist (Target Criterion = 80%)Based on the total average of the 327 checklists submitted, school teams report that 78% of items are in place.

Susan

Jerry, can you add trend line?

Evaluation Question: Are schools implementing SW-Positive Behavior Support?Data Source: SET (Objective Criterion = 80%)-All regions met objective criterion-A pre-post comparison regional average shows a 69% increase.

Susan

Schools were able to establish 80% of the school-wide critical features within the first year of implementation.

222 schools have at least two SET scores

182 SETs were completed in 2007

Evaluation Question: Do 80% of the students in MD Elementary PBIS schools using SWIS receive 0 or 1 office referrals?Data Source: SWIS PBIS Elementary schools in Maryland using SWIS report that 92% of their students receive 0 or 1 office referral.

Susan

Growing the Green

Evaluation Question: Where is the location of behavior problems in Middle Schools?Data Source: SWISThe majority of the problem behaviors reported occur in the classroom followed by hallways.

Susan

We have 500 schools that have

Problem solving teams with admin support and

teacher buy- in

Established Universal or SW Practices

Behavior Support Coach

Local Facilitator or Coordinator

Action Plan with outcome measures

Tools to assess fidelity and outcomes

Sustainability mechanisms (ongoing training, recognition)

Data Facilitator

Susan

Recommendations

SW-PBS Framework and Logic can easily be link other major education initiatives

RtI, Character Ed, Drop Out Prevention, Bully prevention

SW-PBS can also be linked to other major mental health initiatives

School-based MH, Systems of Care

We have a unique opportunity and responsibility to promote integration of services for students across a continuum that meets all students needs

What is Coaching Capacity?

Why Coach? presentation

Personnel & resources organized to facilitate, assist, maintain, & adapt local school professional development implementation efforts

Coaching is set of responsibilities, actions, & activities

Self Assessment

Roles and Responsibilities

Can you get your data easily?

www.pbismaryland.org

Coach Competencies

24

Big Ideas

#1 Schoolwide systems need to be in place to support all students prior to building secondary and tertiary systems, therefore need to focus on SW PBIS coaches (at the universal level) first!

#2. Prior Experience and Endorsement

Attendance at previous introductory PBIS team training sessions

Coaches experienced with school team implementation

team member

Supervisor endorsement

District agreements & support given

Big Idea #3

Not enough to be expert in PBIS content knowledge, you also need facilitation skills

Curriculum included in coaches PD:

Using PBIS Team Implementation Checklist

Preparing for Using the School-wide Data Management System

Using Data to Guide Decisions

Using the Tools

Using Data to Guide Instruction: Common Area Routines and Practices

Leading the Development of Teaching School-wide Behavior Expectations

Teaching Behavior

Assessing Classroom Management

Developing and Using Strategies for Generating Ideas

Learning Walk

Facilitation Skills

Assessing Committee / Workgroups

Year-end Evaluation

Action Planning

#4

On-site coaching:

Clearly identify roles of External and Internal Coach, Team Leader and Team Members

Who develops an agenda?

Feedback on agenda

Data analysis- Who prints reports?

Feedback on data discussion

Action planning

Feedback on:

Running a meeting

Opening, staying with agenda, closing, action steps

Guiding questions

Problem solving

Participant

Timelines

#5

Need to plan for sustainability and capacity building

On-going professional development available to cultivate new building coaches

Turnover and burnout

Continuous regeneration

Coaching

Focus on Role and Functions- not person

FTE/Job Description

Does your boss know what you do?

Internal v. External

Community of Practice

Funding

Buy In

Staff Turnover

Coaching

Data Facilitator- Do your schools know how to use the tools/forms?

Readiness

Computer Application

Decision Making

Primary v. Precision

Executive Coaching-

Training

Regular Training Cycle

Curriculum- Illinois, MO, VA, OR

Trainers- TOT

Focus on outcomes

Differentiated Instruction

Readiness

Follow Up

Type of Skill to be Trained

Skinner (1974) distinguishes between two types of knowledge.

Knowing About: can describe variables that influence a phenomenon.

Example: Describe principles of reinforcement.

Knowing How: can perform effectively

Example: Shape the behavior of another.

One form of knowing does not imply the other.

Final thoughts

Stay close to schools needs

Do SETs, visits, ongoing feedback from various roles etc.

Other topics need to be addressed

Sustainability

Scaling

Expansion

Sustainability

Scaling Up the Triangle

Scaling Up

Does not simply equal more schools or every school within a district/region/state

Outcome = increasing schools adoption and sustained use of evidence-based practices with integrity that lead to improved academic and social outcomes for students with accompanying organizational supports to allow replication

36

Recommendations on Promoting New Initiatives

New initiatives should be adopted with:

Formal assessment of how they may or may not connect with other initiatives

Documented evidence of effectiveness

Well defined and relevant outcome indicators

Mechanism for assessing and evaluating their fidelity of treatment

(Adelman & Taylor, 2003)

37

Research Findings on Scaling Up(Fixsen, Naoom, Blase, Friedman, & Wallace, 2005, p. 70)

Best evidence documents what doesnt work:

Information dissemination alone

Training by itself

38

Research Findings on Scaling Up(Fixsen, Naoom, Blase, Friedman, & Wallace, 2005, p. 70)

What works

Long term, multi-level approaches

Skills-based training

Practice-based coaching

Practioner performance-feedback

Program evaluation

Facilitative administrative practices

Methods for systems intervention

39

Recommendations(Fixsen, Naoom, Blase, Friedman, & Wallace, 2005, p. 77)

Develop partnerships with skilled researchers

Establish a community of practices at

implementation sites

Share lessons learned across functional purveyor

teams from different programs

40

Expansion

From 60 to 600: The Perfect Storm

Maryland 494 schools North Carolina 548 schoolsIllinois 611 schoolsColorado 405 schoolsFlorida 250 schoolsNew York 322 schoolsMichigan 181 schoolsOhio 221 schoolsNew Mexico 130 schoolsWest Virginia 215 schoolsOregon 229 schoolsLouisiana 285 schoolsMissouri 183 schoolsGeorgia 171 schools

5 Year action plan

Example

7183052578190116110310184747648211311043%56%60%90%82%79%91%97%100%

0%10%20%30%40%50%60%70%80%90%100%

020406080100120140199920002001200220032004200520062007

Trained and Implementing by Cohort

TrainedImplementing% of Trained Implementing

NOTE: Retrained schools will be countedin the most recent retraining year only.

13655072185199317275114843250783012214121013643370182176301273114543137762492131110100%98%66%97%100%40%89%67%97%14%100%60%100%94%100%97%74%100%75%80%75%100%93%92%100%

0%10%20%30%40%50%60%70%80%90%100%

01020304050607080

AlleganyAnne ArundelBaltimore CityBaltimore Co.CalvertCarolineCarrollCecilCharlesDorchesterFrederickGarrettHarfordHowardKentMontgomeryPrince GeorgesQueen AnnesSomersetSpecialSt. MarysTalbotWashingtonWicomicoWorcester

Implementing/Inactive by LSS

TrainedImplementing% of Trained Implementing

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

199920002001200220032004200520062007

% of schools submitting data

306097286

IPI Fall 2007

PreparationInitiationImplementationMaintenance

473

# of schools reporting:

Team Implementation Checklist

Spring 2007

85.6

86.8

74.8

76.7

71.9

95.0

80.8

77.0

75.5

51.9

78.3

63.1

70.6

90.0

85.085.0

90.0

100.0

83.3

94.0

72.5

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

100.0

Allegany

Anne

Arundel

Baltimore

City

Baltimore

Co.

Calvert

Caroline

Carroll

Charles

Frederick

Harford

Howard

Montgomery

Prince

Georges

Queen

Annes

Somerset

Special

St. Marys

Talbot

Washington

Wicomico

Worcester

LSS Averages

Ski

l

l Mast

e

ry

Compe

t

enc

y

L

ev

e

l

Data

Pra

c

t

i

ces

Sys

t

ems

Lev

e

l

O

n

e

(

Co

a

ches

)

:

Fluenc

y

@

U

n

iver

s

a

l

lev

e

l

L

e

ad

t

e

a

ms through

process (

D

ir

e

ct)

Mu

lt

ipl

e

d

a

t

a

c

o

l

l

e

c

tion

syst

e

ms

Proble

m

sol

v

ing

w

i

th

D

a

t

a

School

-

wi

d

e

P

B

S

es

s

en

t

i

a

l

f

e

a

tures

Effe

c

ti

v

e

In

s

truc

t

ion

C

l

assroo

m

m

anag

e

ment

Ru

l

es to inform

ado

p

tion

of

prac

t

i

c

es

Mod

e

l

s

chool ex

a

mpl

e

s

Ba

si

cs

of

A

BA

PB

S

respons

e

s to

probl

e

m

behavior

M

e

e

tings / T

e

a

m

rol

e

s

Commu

n

i

c

a

t

ion

w

it

h

in

bu

i

lding

Consu

l

t

a

tion

Targ

e

ted

t

echn

i

c

a

l as

s

is

t

an

c

e

Task

an

a

ly

z

e

t

e

am

g

ener

a

te

d

univer

s

a

l

prac

t

i

c

es

for

im

p

l

e

me

n

t

a

tion

Codify pr

a

c

t

i

c

e

s

i

n

to

po

l

icy

Lev

e

l Two

(Coac

h

e

s

/Tr

a

iners

)

:

Es

t

ab

l

ishe

d

eff

e

c

t

iv

e

school

si

te(s)

Fluenc

y

a

t

S

m

a

l

l

group and Individua

l

studen

t

l

ev

e

l

Work w

i

th

scho

o

l

t

e

am ch

a

ir

(Indir

e

ct)

Trai

n

t

e

a

ms in

univers

a

ls

Re

s

ea

r

ch

d

a

t

a

c

o

l

l

e

c

tio

n

too

l

s

(e.g.

,

S

E

T)

Dir

e

c

t

observ

a

tion

d

a

t

a a

t

individu

a

l stude

n

t lev

e

l

Advance

ABA

Functio

n

a

l

B

eha

v

iora

l

As

s

es

s

me

n

t

Soc

i

a

l

Sk

i

l

l

in

s

tru

c

tion

Syst

e

m

a

t

i

c

(d

a

ta

-

b

ased

)

s

tude

n

t

iden

t

if

i

c

a

tio

n

for

s

m

a

l

l

group/

individu

a

l supports

Cr

e

a

t

e/suppor

t

stu

d

ent support

t

e

a

m

or

process

Task

an

a

ly

z

e t

e

am

g

ener

a

te

d

s

m

a

l

l

group/individu

a

l

pr

a

c

t

i

c

e

s

for

imp

l

e

m

en

t

a

t

ion

Adapting u

n

iver

s

a

l

sys

t

e

m

s to

support

gener

a

l

i

z

a

tion

of

s

m

a

l

l

group

/indi

v

idu

a

l

p

l

ans

Consu

l

t

w

it

h

othe

r

age

n

c

i

e

s

/pare

n

ts

Codify pr

a

c

t

i

c

e

s

i

n

to

po

l

icy

Lev

e

l Thre

e

(Coordi

n

ator)

:

Work w

i

th

m

u

l

t

ip

l

e

schools

Trai

n

t

e

a

ms

uni

v

ers

a

l

individ

u

a

l

l

ev

e

l

Trai

n

co

a

ches

Es

t

ab

l

ish

d

ist

r

i

c

t

-

wi

d

e

syst

e

m

Eval

u

a

t

ion

of

d

i

st

r

i

c

t

-

wi

d

e

efforts

Da

t

a

d

e

ci

s

ion

ru

l

e

s

t

o

ide

n

tify

needed

supports

w

it

h

in

/

acros

s

schools

Link

s

chool

n

eed

s

(da

t

a

-

ba

s

ed)

to di

s

tr

ic

t

profes

s

ion

a

l

develop

m

ent

Effe

c

ti

v

e

profes

s

ion

a

l

de

v

elop

m

en

t

/

tr

a

ining

s

k

i

l

ls

M

ap

di

s

tr

i

c

t

po

l

icy to es

s

en

t

i

a

l

featu

r

es

of

P

B

S

Lea

d

ership

t

ea

m

ro

l

es / m

e

e

t

ings

Co

mmu

n

i

c

a

t

ion

a

cross

d

i

s

tr

i

ct

Resou

r

ce

b

ank

(

exper

t

s

& m

a

te

r

i

a

ls)

Codify pr

a

c

t

i

c

e

s

i

n

to

po

l

icy

Competency Level

Skill Mastery

Data

Practices

Systems

Level One (Coaches):

Fluency @ Universal level

Lead teams through process (Direct)

Multiple data collection systems

Problem solving with Data

School-wide PBS essential features

Effective Instruction

Classroom management

Rules to inform adoption of practices

Model school examples

Basics of ABA

PBS responses to problem behavior

Meetings / Team roles

Communication within building

Consultation

Targeted technical assistance

Task analyze team generated universal practices for implementation

Codify practices into policy

Level Two (Coaches/Trainers):

Established effective school site(s)

Fluency at Small group and Individual student level

Work with school team chair (Indirect)

Train teams in universals

Research data collection tools (e.g., SET)

Direct observation data at individual student level

Advance ABA

Functional Behavioral Assessment

Social Skill instruction

Systematic (data-based) student identification for small group/ individual supports

Create/support student support team or process

Task analyze team generated small group/individual practices for implementation

Adapting universal systems to support generalization of small group /individual plans

Consult with other agencies/parents

Codify practices into policy

Level Three (Coordinator):

Work with multiple schools

Train teams universal individual level

Train coaches

Establish district-wide system

Evaluation of district-wide efforts

Data decision rules to identify needed supports within/across schools

Link school needs (data-based) to district professional development

Effective professional development / training skills

Map district policy to essential features of PBS

Leadership team roles / meetings

Communication across district

Resource bank (experts & materials)

Codify practices into policy