Reducing Behavior Reducing Behavior Problems in the Problems in the

Classroom: Classroom: What we know and what we need to What we know and what we need to

know know

Michael H. Epstein, Ed.D.Michael H. Epstein, Ed.D.University of Nebraska-LincolnUniversity of Nebraska-Lincoln

mepstein1@unl.edumepstein1@unl.edu

Krista Kutash, Ph.D.Krista Kutash, Ph.D.University of South FloridaUniversity of South Florida

kutash@fmhi.usf.edukutash@fmhi.usf.edu

11

The Institute of Education Sciences (IES) 5th Annual Research Conference

June 2010 - National Harbor, MD

Krista Krista Kutash Kutash

Ph.D.Ph.D.

Mike Mike Epstein Epstein

Ph.D.Ph.D.

Today’s PresentersToday’s Presenters

How you How you can can

visualize usvisualize us

22

Released in 2008

33

Panel MembersPanel Members

Mike Epstein (Chair)Mike Epstein (Chair)– University of Nebraska-LincolnUniversity of Nebraska-Lincoln

Marc AtkinsMarc Atkins– University of Illinois-ChicagoUniversity of Illinois-Chicago

Doug CullinanDoug Cullinan– North Carolina State UniversityNorth Carolina State University

Krista KutashKrista Kutash– University of South Florida University of South Florida

Robin WeaverRobin Weaver– Harmony Hills Elementary School Principal (Silver Spring, Harmony Hills Elementary School Principal (Silver Spring,

MD)MD) 44

SRI StaffSRI Staff

Michelle Woodbridge Michelle Woodbridge (Study Director, (Study Director, author)author)

Mary Wagner Mary Wagner (Senior Consultant, author)(Senior Consultant, author)

Jennifer Yu Jennifer Yu (Researcher, author)(Researcher, author)

Carl Sumi Carl Sumi (Researcher)(Researcher)

55

WWC WWC Practice GuidesPractice Guides: : PurposePurpose Use Use expertise and judgment of a expertise and judgment of a

panelpanel to identify the most to identify the most important research relevant to the important research relevant to the recommendations.recommendations.

Bring the Bring the best available evidencebest available evidence to bear on challenges that cannot to bear on challenges that cannot currently be addressed by single currently be addressed by single interventions or programs.interventions or programs.

66

WWC WWC Practice GuidesPractice Guides: : PurposePurpose Supply discrete recommendations Supply discrete recommendations

that are intended to be that are intended to be actionableactionable.. Provide a Provide a coherentcoherent approach to a approach to a

multifaceted problem.multifaceted problem. Explicitly connect each Explicitly connect each

recommendation to the recommendation to the level of level of evidenceevidence supporting it (strong, supporting it (strong, moderate, or low)moderate, or low)

77

WWC Evidence WWC Evidence StandardsStandards

Reviewed studies receive 1 of 3 Reviewed studies receive 1 of 3 ratings:ratings:

Meets Evidence StandardsMeets Evidence Standards– Strong evidence of the intervention’s effectivenessStrong evidence of the intervention’s effectiveness

Meets Evidence Standards with Meets Evidence Standards with ReservationsReservations– Weaker evidence of the effectivenessWeaker evidence of the effectiveness

Does Not Meet Evidence StandardsDoes Not Meet Evidence Standards– Insufficient evidence of the effectivenessInsufficient evidence of the effectiveness

88

WWC Study RatingWWC Study Rating

Randomization

Equivalence

Attrition

Meets Evidence

Standards

Meets Evidence Standards with Reservations

Does Not Meet

Evidence Standards

*Figure from WWC Procedures and Version 2 Standards Handbook, December 2008

Yes

Yes

No

No

High

Low

99

WWC Study RatingWWC Study Rating

Rating affected by:Rating affected by: Rigor of Rigor of designdesign

– RCT or QED studiesRCT or QED studies– Regression discontinuity and single subject Regression discontinuity and single subject

standards under developmentstandards under development ValidityValidity and and reliabilityreliability of outcome measures of outcome measures EquivalenceEquivalence of groups at baseline of groups at baseline Overall and differential Overall and differential attritionattrition Absence of Absence of confoundsconfounds due to: due to:

– N of 1N of 1– Intervention combined with another interventionIntervention combined with another intervention– Not implemented as designedNot implemented as designed 1010

Levels of Evidence for Levels of Evidence for Practice GuidesPractice Guides

StrongStrong– High internal and external validity, including:High internal and external validity, including:

Evidence of effectiveness with no contradictory evidence, Evidence of effectiveness with no contradictory evidence, OROR

One large, well-designed, randomized, multisite One large, well-designed, randomized, multisite effectiveness trial with no contradictory evidenceeffectiveness trial with no contradictory evidence

ModerateModerate– High internal but moderate external validity, ORHigh internal but moderate external validity, OR– Moderate internal but high external validity, including:Moderate internal but high external validity, including:

Small sample sizes or conditions that limit generalizability, Small sample sizes or conditions that limit generalizability, OROR

Nonequivalent comparison groups at pretest but Nonequivalent comparison groups at pretest but consistently enhanced outcomes for participants, ORconsistently enhanced outcomes for participants, OR

Correlational research with strong controls for selection Correlational research with strong controls for selection biasbias

LowLow– Evidence not meeting standards aboveEvidence not meeting standards above

1111

Practice Guide: ScopePractice Guide: Scope

Primary audiences include:Primary audiences include:– General education elementary school General education elementary school

teachersteachers who will implement practices who will implement practices– Elementary school/district Elementary school/district

administratorsadministrators who will promote who will promote practicespractices

Final product is “more like a Final product is “more like a consensus panel reportconsensus panel report than a than a meta-analysis” in terms of breadth meta-analysis” in terms of breadth and complexity of topic addressed.and complexity of topic addressed.

1212

Identify specifics of problem behavior & Identify specifics of problem behavior &

conditions that prompt and reinforce it.conditions that prompt and reinforce it. – Level of Evidence: Level of Evidence: ModerateModerate– Implementation GuidelinesImplementation Guidelines

Concretely describeConcretely describe the behavior problem and its the behavior problem and its effect on learning.effect on learning.

ObserveObserve and and recordrecord frequency and context of the frequency and context of the problem behavior.problem behavior.

Identify what Identify what promptsprompts and and reinforcesreinforces the problem the problem behavior.behavior.

Recommendation 1Recommendation 1

1313

WhyWhy ModerateModerate??– Multiple Multiple single-subjectsingle-subject research research

studies demonstrated effectiveness of studies demonstrated effectiveness of interventions tailored to antecedents interventions tailored to antecedents and consequences of behavior and consequences of behavior problemsproblems

– OnlyOnly emerging evidence emerging evidence on feasibility on feasibility of general educator applying of general educator applying assessment-based approachesassessment-based approaches

Recommendation 1Recommendation 1

Level of EvidencLevel of Evidencee

1414

Identify specifics of problem behavior and Identify specifics of problem behavior and

conditions that prompt and reinforce it.conditions that prompt and reinforce it. – Develop screening measures that are time Develop screening measures that are time

efficient and psychometrically sound.efficient and psychometrically sound.– Develop progress monitoring systems that Develop progress monitoring systems that

are valid and reliable.are valid and reliable.– Demonstrate that teachers can independently Demonstrate that teachers can independently

implement functional assessment systems.implement functional assessment systems.

Recommendation 1 - Recommendation 1 - NeedNeed

1515

Modify the classroom learning environment to Modify the classroom learning environment to decrease problem behavior. decrease problem behavior. – Level of Evidence: Level of Evidence: StrongStrong– Implementation GuidelinesImplementation Guidelines

Revisit, re-practice, and reinforce Revisit, re-practice, and reinforce

classroom expectationsclassroom expectations.. Modify the classroom environmentModify the classroom environment

to encourage instructional momentumto encourage instructional momentum Adapt or vary instructional strategies to Adapt or vary instructional strategies to increase increase

opportunitiesopportunities for academic success and for academic success and engagement.engagement.

Recommendation 2Recommendation 2

1616

WhyWhy StrongStrong??– 3 3 RCTsRCTs, 1 , 1 QEDQED, and 6 , and 6 single-subject single-subject

research studies demonstrated research studies demonstrated empirical support for:empirical support for: Preventive classroom managementPreventive classroom management Direct and differentiated instruction Direct and differentiated instruction Peer tutoringPeer tutoring

Recommendation 2Recommendation 2

Level of EvidencLevel of Evidencee

1717

Modify the classroom learning Modify the classroom learning environment to decrease problem environment to decrease problem behavior. behavior. – Investigate the influence of student Investigate the influence of student

choice of tasks on student behavior.choice of tasks on student behavior.– Develop professional development Develop professional development

programs to teach behavior programs to teach behavior management skills to teachers.management skills to teachers.

Recommendation 2 - Recommendation 2 - NeedNeed

1818

Teach and reinforce new skills to increase Teach and reinforce new skills to increase appropriate behavior and preserve a positive appropriate behavior and preserve a positive classroom climate.classroom climate.– Level of Evidence: Level of Evidence: StrongStrong– Implementation GuidelinesImplementation Guidelines

Identify where the student needs Identify where the student needs explicit instructionexplicit instruction for appropriate for appropriate behavior.behavior.

Teach skills by providing Teach skills by providing examples, examples, practice, and feedbackpractice, and feedback. .

Recommendation 3Recommendation 3

Manage consequencesManage consequences so that reinforcers are so that reinforcers are provided for provided for

appropriate behavior and withheld for inappropriate appropriate behavior and withheld for inappropriate

behavior. behavior. 1919

WhyWhy StrongStrong??– 5 5 RCTsRCTs and 3 and 3 single-subjectsingle-subject research research

studies demonstrated effectiveness of studies demonstrated effectiveness of teaching and reinforcing replacement teaching and reinforcing replacement behaviors to reduce inappropriate behaviors to reduce inappropriate behaviorsbehaviors Attention seekingAttention seeking Social skillsSocial skills Problem solvingProblem solving Self managementSelf management Self controlSelf control

Recommendation 3Recommendation 3

Level of EvidencLevel of Evidencee

2020

Teach and reinforce new skills to increase Teach and reinforce new skills to increase appropriate behavior and preserve a positive appropriate behavior and preserve a positive classroom climate.classroom climate.– Develop manualized programs that teach Develop manualized programs that teach

social behavior skills based on sound social behavior skills based on sound instructional practices. instructional practices. 

– Investigate the contribution of degree of Investigate the contribution of degree of implementation and dosage of treatments to implementation and dosage of treatments to student outcomes. student outcomes.

– Conduct research investigating the value Conduct research investigating the value added effects of multi-component programs.added effects of multi-component programs.

Recommendation 3 - Recommendation 3 - NeedNeed

2121

Draw on relationships with colleagues Draw on relationships with colleagues and families for guidance and support.and families for guidance and support.– Level of Evidence: Level of Evidence: ModerateModerate– Implementation GuidelinesImplementation Guidelines

CollaborateCollaborate with other with other teachersteachers for continued for continued guidance and support.guidance and support.

Build Build collaborativecollaborative professional partnerships professional partnerships with with schoolschool, district, and , district, and community behavior community behavior expertsexperts who can consult with teachers when who can consult with teachers when problems are serious enough to warrant help problems are serious enough to warrant help from outside the classroom. from outside the classroom.

Encourage Encourage parentsparents and and other family membersother family members to participate as active partners in teaching to participate as active partners in teaching and reinforcing appropriate behavior.and reinforcing appropriate behavior.

Recommendation 4Recommendation 4

2222

Why Why ModerateModerate??– 1 1 QEDQED and 1 and 1 single-subjectsingle-subject study study

demonstrated peer teacher relationships (i.e., demonstrated peer teacher relationships (i.e., coaching) improved student social skills and coaching) improved student social skills and engagementengagement

– 1 1 RCTRCT confirmed effectiveness of teachers’ confirmed effectiveness of teachers’ consulting with behavioral expertsconsulting with behavioral experts

Limitation: study conducted specifically with ADHD Limitation: study conducted specifically with ADHD populationpopulation

– 2 2 RCTsRCTs confirmed positive effect of teacher- confirmed positive effect of teacher-parent partnershipsparent partnerships

Limitation: study conducted with specific teacher-Limitation: study conducted with specific teacher-parent programparent program

Recommendation 4Recommendation 4

Level of EvidencLevel of Evidencee

2323

Draw on relationships with colleagues and families Draw on relationships with colleagues and families for guidance and support.for guidance and support.– Focus consultation research on student Focus consultation research on student

outcomes as opposed to teacher-centered outcomes as opposed to teacher-centered outcomes.outcomes.

– Investigate teacher assistance teams decision Investigate teacher assistance teams decision making process. making process. 

– Test efficacy and effectiveness of consulting Test efficacy and effectiveness of consulting models.models.

– Investigate the value added effects of school Investigate the value added effects of school based mental health programs.  based mental health programs.  

– Understand parent engagement process.Understand parent engagement process.– Develop alternative delivery models of Develop alternative delivery models of

parenting programs.parenting programs.

Recommendation 4 - Recommendation 4 - NeedNeed

2424

Implement schoolwide strategies Implement schoolwide strategies to reduce negative and foster to reduce negative and foster positive interactions.positive interactions.– Level of Evidence: Level of Evidence: ModerateModerate– Implementation GuidelinesImplementation Guidelines

Address schoolwide behavior issues by involving a school Address schoolwide behavior issues by involving a school improvement improvement teamteam

Collect informationCollect information on the on the hot spotshot spots throughout the throughout the school, such as the frequency of particular schoolwide school, such as the frequency of particular schoolwide behavior problems and when and where they occurbehavior problems and when and where they occur

Recommendation 5 Recommendation 5

Monitor implementation and outcomes using an efficient Monitor implementation and outcomes using an efficient method of method of data collectiondata collection and allow ample time for the and allow ample time for the program to work.program to work. If warranted, If warranted, adopt a packaged interventionadopt a packaged intervention program program that that fits well with identified behavior problem(s) and the fits well with identified behavior problem(s) and the school school context.context.

2525

Why Why ModerateModerate??– 1 1 QEDQED study demonstrated positive effects of study demonstrated positive effects of

schoolwide changes (e.g., structure, schoolwide changes (e.g., structure, organization) on student social relationships organization) on student social relationships and acceptanceand acceptance

– 4 4 RCTsRCTs and 1 and 1 single-subjectsingle-subject study documented study documented impact of schoolwide intervention program on impact of schoolwide intervention program on reduced problem behaviorsreduced problem behaviors

Limitation: RCTs support only specific schoolwide Limitation: RCTs support only specific schoolwide program, not all components of recommendation.program, not all components of recommendation.

Recommendation 5Recommendation 5

Level of EvidencLevel of Evidencee

2626

Implement schoolwide strategies to reduce Implement schoolwide strategies to reduce negative and foster positive interactions.negative and foster positive interactions.– Test the efficacy and effectiveness of Test the efficacy and effectiveness of

schoolwide programs. schoolwide programs. – Understand how school climate/culture Understand how school climate/culture

influences the implementation and outcomes influences the implementation and outcomes of schoolwide programs.of schoolwide programs.

– Study the economic benefits of schoolwide Study the economic benefits of schoolwide programs.programs.

Recommendation 5 Recommendation 5 NeedNeed

2727

2828

To download and printTo download and printhttp://ies.ed.gov/ncee/wwc/publications/practiceguides/http://ies.ed.gov/ncee/wwc/publications/practiceguides/

Thank You Thank You

Thank you for participating todayThank you for participating today

2929