3/5/20

1

Implementing the Pyramid Model Across Settings: What, Where and How?

Erin BartonKathleen Artman Meeker

Angel FettigHedda Meadan

Mary Louise Hemmeter

CRIEIFebruary, 2020

Overview of Session

• Introduction to Session• Overview of Pyramid Model• Efficacy studies• Issues in implementation

• Program-Wide Implementation of the Pyramid Model in Community-Based Programs• Tiered Coaching Model to Support Implementation of

the Pyramid Model• Family Behavior Support Mobile Software App

(FBSApp)• Discussion and Questions

The Pyramid Model: Promoting Social-Emotional Competence

and Addressing Challenging Behavior

Universal prom otion: All children

Secondary prevention: Som e children

Tertiary intervention: Few children

Research on the Pyramid Model

• Teacher outcomes– PD including workshops, 16 weeks of practice-based coaching, and

implementation materials resulted in higher TPOT scores– Up to a year after the PD intervention ended, teachers continued to use

the practices at levels similar to those observed at the end of the PD year

– Teachers reported the intervention to be feasible and effective• Child outcomes– Intervention group teacher ratings of children’s social skills and

challenging behavior improved when compared to teachers in the control group

– Observations of target children’s social skills improved when compared to children in the control group

Hemmeter, Snyder, Fox, & Algina, 2018

3/5/20

2

What do we do to take the PM to the next level?

• Support implementation at scale–Effective PD approaches that can be implemented within

systems– Identify other program supports needed to sustain

implementation• Examine implementation in a variety of early childhood

settings• Expand to home visiting programs• Identify effective ways to support families• Integrate Pyramid Model with instruction on content in

other domains

Institute of Education Sciences, Grant Number R305A150141

Supporting Implementation of the Pyramid Model in Community

Early Childhood Programs

Mary Louise Hemmeter, Erin Barton, & Lam PhamVanderbilt University

Lise Fox & Chris VatlandUniversity of South Florida

Gary HenryUniversity of Delaware

Child Care Context

• Limited resources– Staffing issues

–Materials• Staff turnover

• Competing priorities– Capacity of the program and

workforce

– Scheduling

• Staff with limited training– Teaching staff

– Behavior support staff– Coaches

• Hours– Time for Leadership team

meetings

– Time for PD

Child Care Context

3/5/20

3

Rationale for Implementing Pyramid Model in Child Care Settings

• Reduce suspensions and expulsions of children with challenging behavior

• Build capacity of programs to support children• Link to other community resources• Program specific needs:– Reduce need for administrative staff to put out fires– Provide support for teaching staff– Build confidence of teaching staff– Facilitate family engagement

Program-Wide Implementation

Program-Wide Supports for Pyramid Model Implementation (PWS-PMI)

Procedures for Responding to Challenging

Behavior

Data Decision-Making

Examining Implementation and Outcomes

• Leadership Team• Staff Buy-In• Program-wide

Expectations• Family Engagement– Individual child–Program-wide

implementation

• Professional Development and Support • Responding to

Challenging Behavior• Collecting and Using

Data to inform Program Improvement

Critical Elements of Program-Wide Implementation

3/5/20

4

PILOT STUDY: Methods and Outcomes

Overview of Pilot Study • 16 Programs– Head Start, Child Care, Pre-k– Serving low income children

• Randomly assigned to control and intervention conditions• Intervention group:– Leadership academy– Training – staff, behavior support personnel, coaches– External coaching weekly

• Control group:– All training events at the end of the study

External Coaching Visits

• Support leadership team meetings (M = 9, range = 6-10)• Weekly visits (M = 25, range = 14-36)–Discuss coaching–Data support–Coach in classroom without coach–Coach in classroom with coach–Behavior support out of classroom–Behavior support in classroom–Conduct TPOT with coach–Family event

Measures

• Program Level–Supporting Program-Wide Implementation Fidelity Instrument

(SPIFI)–Preschool-Wide Evaluation Tool (PreSET)

• Classroom Level–Teaching Pyramid Observation Tool (TPOT)–Classroom Observation and Scoring System (CLASS)

• Child Level–Social Skills Intervention System (SSIS)

3/5/20

5

Cascading Design Issues

External Coaching

Program-wide Implementation

TeachersStaff

Children

Results

• Program–Significant effect on SPIFI and PreSET–No effect on disciplinary actions

• Classroom–Significant effect on TPOT–No effect on CLASS

• Child–No effect on SSIS

Readiness for PWS-PMI• Motivation– Lack of understanding of what was involved• Staff lacked commitment, overwhelmed by the work involved in

implementation– Philosophical alignment to program• Ranged from being passionate about the Pyramid Model to being comfortable

with punishment in response to behavior• Leadership– Strong investment by administration (e.g., program director) linked to teacher

buy-in and excitement

• Resources– Lack of staff for allocated roles (e.g., classroom coach), class coverage, turnover

– Physical resources (e.g., materials, equipment, space)

Implementation Growth

Challenges

• Teacher stress about competing demands

• Teacher innovation fatigue “wait this out”

• Teaching team relationship challenges

• Classroom quality • Lack of time for coaching• Coaching skill and confidence• Staff turnover

Supports

• Strong leader investment• Administrator participation on the

leadership team• Regular leadership team meetings• Previous Pyramid Model experience• Strong relationships with families• Core quality practices• Delivery of coaching

3/5/20

6

Vanderbilt

• Mary Louise Hemmeter - PI• Erin Barton – co-PI• Kym Horth– External Coach• Abby Taylor – External Coach• Taylor Puckett– data• Rebecca Murray- data• Gary Henry and Lam Pham –

design and data analysis

USF

• Lise Fox – PI• Chris Vatland - co-PI• Myrna Veguilla – USF Study

Coordinator• Meghan von der Embse – External

Coach• Denise Perez Binder – External

Coach• Shanel Greene – data• Telisha McKenzie - data

Study Team

24

Comments and Questions

Tiered Coaching Model to Support Implementation of

the Pyramid ModelCRIEI 2020 TCM Project

Goals 1. Create a data-driven decision model related to

PD and teacher coaching profiles2. Develop a tiered coaching model that matches

teachers to different types of coaching3. Evaluate the efficacy of the tiered model

This project is funded by Institute of Education Sciences (R324A170019).

3/5/20

7

Project Overview

Year 3● Pilot fully developed

tiered coaching model

YEAR 2

● Develop and field test a tiered coaching model based on coaching profiles

YEAR 1● Develop coaching profiles

based on TPOT and Survey Data

Framework and Measures

Coac

hing

Foc

us

Latent Profile Analysis: Method

● 97 preschool teachers

● Teacher survey:○ Demographics○ PD experiences and satisfaction○ Job stress and satisfaction○ Work engagement and commitment○ Disciplinary efficacy

● Teaching Pyramid Observation Tool (TPOT)

3/5/20

8

Development of the Tiered Coaching ModelField Test: Method and Sample

● 16 lead preschool teachers● TPOT and Survey completed in Fall 2018● Teachers initially assigned to Tiers 1-3:○ Tier 1: 7 teachers○ Tier 2: 6 teachers (two groups)○ Tier 3: 3 teachers

● Midpoint TPOT collected, teachers moved to new Tier (as needed)

● Post-test TPOT and focus groups conducted in Spring 2019

Field Test: Quantitative Results

Initial Placement

N Average Change

Tier 1 7 2.50Tier 2 6 5.69Tier 3 3 3.45

Field Test: Movement Across Tiers

Coaching Type Initial N Mid-Year NTier 1 (Self) 7 10

• 6 stayed• 4 entered from higher tier• 1 moved to Tier 3

Tier 2 (Group) 6 4• 3 stayed• 1 entered from Tier 3• 3 moved to Tier 1

Tier 3 (Individual)

3 2• 1 stayed• 1 entered from Tier 1• 1 moved to Tier 1• 1 moved to Tier 2

3/5/20

9

Lessons Learned & Successes in Childcare

• Online small group coaching seemed effective and had positive reviews from teachers. • Integration with existing

accountability and quality systems was helpful (STARS credits, etc.)• Challenging behavior was often

the “game changer” for coaching (requests to move between tiers, etc.)• Leadership support was

essential.

Challenges & Questions

Implementation• Turnover & role shifts• Teaming, co-teaching &

“floating” staff• Integration with existing

coaching resources• Misalignment with philosophies

or program models

Conducting Research

• Coaching requests & project scope• Recruitment in Year 3 Pilot:• Intensity of data collection• Control group• Leader buy-in

Development and Testing of the FBSApp for Improving Parent and

Child Outcomes

Grant R324A160086

Cross-site Collaboration

Angel Fettig Hedda MeadanErin Barton

3/5/20

10

Need for effective family supports

Homes• Home-based supports for

families of children with challenging behaviors.

Technology • Mobile applications (apps)

might provide an innovative, effective option for delivering ongoing support to families. • Internet and smart phones are

ubiquitous Mobile technology provides an opportunity to increase the accessibility and efficiency of support for parents of these children.

Need for effective family supports

FBSApp is a mobile application to support parents in implementing functional assessment (FA)-based interventions, which work by determining the purpose (function) that a particular behavior serves for a child and using this information to develop a plan for reducing and preventing the behavior.

Secure Login

3/5/20

11

How to use this app tutorial Professional Login

Demographic Information Child’s Communication Repertoire

3/5/20

12

Challenging Behavior Topography

FBA Tutorial Collecting A-B-C Data

3/5/20

13

A-B-C Log Hypothesis Statement

Behavior Support Plan

Monitoring Progress

3/5/20

14

But will it work??

Three Categories of Successes

1. Pilot under-powered RCT was conducted 2. Results are promising • Statistical significance in favor of FBSApp group • Challenging behaviors reduced for both groups

3. Iterative process allowed for continual revisions to app • We have an iPhone version! • We have a new algorithm! • We changed almost all the language. • We revised all fidelity supports.

3/5/20

15

Under-powered RCT

Research Questions:1. Does using the FBSApp (including the webinars) effectively increase

parents’ use of FA-based intervention practices compared to parents who have access to the webinars alone?

2. Does using the FBSApp (including the webinars) effectively decrease parent-reported stress over time compared to parents who have access to the webinars alone?

Conditions

• FBSApp + Webinar • Webinar Only

Participants, Setting, Materials

• Children ages 2 – 5 with disabilities • One parent • Across the US

HOMES• iPad • Access to the internet

Results Summary

•Parent Fidelity improved•Parents with access to the FBSApp+Webinars

improved more than parents with Webinars alone•Parent stress decreased across both groups•Child challenging behaviors decreased across both

groups • Satisfaction was high for both groups

3/5/20

16

Four Categories of Challenges

1. Gathering data from families in homes 2. High rates of attrition 3. Complex measurement system4. Lack of professional involvement

Wave Family Procedures Measures Collected Type

W1Demo REDCapPSI-4, CBCL, MAS, TPQ PaperPre/post Interview Interview

W2 Upload 5 videos Child, Parent Observational Measures ObservationW3 RANDOMIZATION

W4 Upload 3 videos Child, Parent Observational Measures ObservationSatisfaction Survey Redcap

W5 Satisfaction Survey RedcapW6 Upload 3 videos Child, Parent Observational Measures Observation

Satisfaction Survey RedcapW7 Upload 3 videos Child, Parent Observational Measures Observation

W8Demo, Satisfaction Survey RedcapPSI-4, CBCL, MAS, TPQ PaperPre/post Interview Interview

W9 Upload 3 videos

Child, Parent Observational Measures ObservationDemo, Satisfaction Survey Redcap, PSI-4, CBCL, MAS, TPQ PaperPre/post Interview InterviewChild, Parent Observational Measures Observation

Participants

Cohort1 2 3 4 Total

Completed W1 19 34 26 12 91Dropped pre-randomization (W3) 9 6 8 2 25Completed 10 24 14 8 56

Treatment 5 13 6 4 28Control 5 11 8 4 28

Dropped randomization-Time 2 (W8) 0 3 4 2 9Dropped Time 2-Time 3 (W9) 0 1 0 0 2

Participants

Cohort1 2 3 4 Total

Completed W1 19 34 26 12 91Dropped pre-randomization (W3) 9 6 8 2 25Completed 10 24 14 8 56

Treatment 5 13 6 4 28Control 5 11 8 4 28

Dropped randomization-Time 2 (W8) 0 3 4 2 9Dropped Time 2-Time 3 (W9) 0 1 0 0 2

3/5/20

17

Parent Fidelity

• 7 point rating scale• Prevent• Positive attention• Predictable routines• Choices• Clear expectations

• Teach• Function-based opportunity• Reinforcement of new skills

• New Response• Function-based response to challenging behavior

Child Challenging Behavior

• Estimated rate or frequency• 5s partial-interval recording• Target and nontarget behaviors• Families could can create multiple behavior

support plans

Child Replacement Behavior

• Estimated rate or frequency• Timed-event sampling• Function-based appropriate verbal replacement behaviors

Issues and Ideas With Ongoing Research

1. SCR study underway! • When to measure, How to measure, What to measure???• Do families really have routines?

2. Changes to measurement system to individualize per family • Complexity versus reliability • Validity versus sensitivity

3. Recruited professionals • Buy in• Who to invite?

3/5/20

18

Issues and Ideas With Ongoing Research

4. How do we scale-up individualized, Tier 3 supports for ALL families?

QUESTIONS??

Research Team

Beth Pokorski

Marina Velez

Claire WinchesterBrandy Locchetta

Bhavya

Hidy Kong

Gabrielle Trimlett

Kaitlin Greeny

Gounah Choi

Shawna HarbinMoon Chung

Rebecca Hacker

Karrie Karahalios

Mollie Todt

Emma Skiba

Gabriela Robinson Lauren Schulte

Jonathan Gerth

Caroline Baum

Financial disclosure: Preparation of this presentation was supported in part by the U.S. Department of Education, Institute for Education Sciences, National Center for Special Education Research Grant R324A160086. However, the opinions expressed do not necessarily reflect the policy of the U.S. Department of Education and no official endorsement should be inferred.

3/5/20

19

Panel Discussion

•How might we extend existing evidence-based practices across early childhood contexts?•How might professional development

systems meet the needs of professionals across contexts?•How can research focus on replications and

feasibility of existing evidence-based practices to novel contexts and implementers?