3/5/20
1
Implementing the Pyramid Model Across Settings: What, Where and How?
Erin BartonKathleen Artman Meeker
Angel FettigHedda Meadan
Mary Louise Hemmeter
CRIEIFebruary, 2020
Overview of Session
• Introduction to Session• Overview of Pyramid Model• Efficacy studies• Issues in implementation
• Program-Wide Implementation of the Pyramid Model in Community-Based Programs• Tiered Coaching Model to Support Implementation of
the Pyramid Model• Family Behavior Support Mobile Software App
(FBSApp)• Discussion and Questions
The Pyramid Model: Promoting Social-Emotional Competence
and Addressing Challenging Behavior
Universal prom otion: All children
Secondary prevention: Som e children
Tertiary intervention: Few children
Research on the Pyramid Model
• Teacher outcomes– PD including workshops, 16 weeks of practice-based coaching, and
implementation materials resulted in higher TPOT scores– Up to a year after the PD intervention ended, teachers continued to use
the practices at levels similar to those observed at the end of the PD year
– Teachers reported the intervention to be feasible and effective• Child outcomes– Intervention group teacher ratings of children’s social skills and
challenging behavior improved when compared to teachers in the control group
– Observations of target children’s social skills improved when compared to children in the control group
Hemmeter, Snyder, Fox, & Algina, 2018
3/5/20
2
What do we do to take the PM to the next level?
• Support implementation at scale–Effective PD approaches that can be implemented within
systems– Identify other program supports needed to sustain
implementation• Examine implementation in a variety of early childhood
settings• Expand to home visiting programs• Identify effective ways to support families• Integrate Pyramid Model with instruction on content in
other domains
Institute of Education Sciences, Grant Number R305A150141
Supporting Implementation of the Pyramid Model in Community
Early Childhood Programs
Mary Louise Hemmeter, Erin Barton, & Lam PhamVanderbilt University
Lise Fox & Chris VatlandUniversity of South Florida
Gary HenryUniversity of Delaware
Child Care Context
• Limited resources– Staffing issues
–Materials• Staff turnover
• Competing priorities– Capacity of the program and
workforce
– Scheduling
• Staff with limited training– Teaching staff
– Behavior support staff– Coaches
• Hours– Time for Leadership team
meetings
– Time for PD
Child Care Context
3/5/20
3
Rationale for Implementing Pyramid Model in Child Care Settings
• Reduce suspensions and expulsions of children with challenging behavior
• Build capacity of programs to support children• Link to other community resources• Program specific needs:– Reduce need for administrative staff to put out fires– Provide support for teaching staff– Build confidence of teaching staff– Facilitate family engagement
Program-Wide Implementation
Program-Wide Supports for Pyramid Model Implementation (PWS-PMI)
Procedures for Responding to Challenging
Behavior
Data Decision-Making
Examining Implementation and Outcomes
• Leadership Team• Staff Buy-In• Program-wide
Expectations• Family Engagement– Individual child–Program-wide
implementation
• Professional Development and Support • Responding to
Challenging Behavior• Collecting and Using
Data to inform Program Improvement
Critical Elements of Program-Wide Implementation
3/5/20
4
PILOT STUDY: Methods and Outcomes
Overview of Pilot Study • 16 Programs– Head Start, Child Care, Pre-k– Serving low income children
• Randomly assigned to control and intervention conditions• Intervention group:– Leadership academy– Training – staff, behavior support personnel, coaches– External coaching weekly
• Control group:– All training events at the end of the study
External Coaching Visits
• Support leadership team meetings (M = 9, range = 6-10)• Weekly visits (M = 25, range = 14-36)–Discuss coaching–Data support–Coach in classroom without coach–Coach in classroom with coach–Behavior support out of classroom–Behavior support in classroom–Conduct TPOT with coach–Family event
Measures
• Program Level–Supporting Program-Wide Implementation Fidelity Instrument
(SPIFI)–Preschool-Wide Evaluation Tool (PreSET)
• Classroom Level–Teaching Pyramid Observation Tool (TPOT)–Classroom Observation and Scoring System (CLASS)
• Child Level–Social Skills Intervention System (SSIS)
3/5/20
5
Cascading Design Issues
External Coaching
Program-wide Implementation
TeachersStaff
Children
Results
• Program–Significant effect on SPIFI and PreSET–No effect on disciplinary actions
• Classroom–Significant effect on TPOT–No effect on CLASS
• Child–No effect on SSIS
Readiness for PWS-PMI• Motivation– Lack of understanding of what was involved• Staff lacked commitment, overwhelmed by the work involved in
implementation– Philosophical alignment to program• Ranged from being passionate about the Pyramid Model to being comfortable
with punishment in response to behavior• Leadership– Strong investment by administration (e.g., program director) linked to teacher
buy-in and excitement
• Resources– Lack of staff for allocated roles (e.g., classroom coach), class coverage, turnover
– Physical resources (e.g., materials, equipment, space)
Implementation Growth
Challenges
• Teacher stress about competing demands
• Teacher innovation fatigue “wait this out”
• Teaching team relationship challenges
• Classroom quality • Lack of time for coaching• Coaching skill and confidence• Staff turnover
Supports
• Strong leader investment• Administrator participation on the
leadership team• Regular leadership team meetings• Previous Pyramid Model experience• Strong relationships with families• Core quality practices• Delivery of coaching
3/5/20
6
Vanderbilt
• Mary Louise Hemmeter - PI• Erin Barton – co-PI• Kym Horth– External Coach• Abby Taylor – External Coach• Taylor Puckett– data• Rebecca Murray- data• Gary Henry and Lam Pham –
design and data analysis
USF
• Lise Fox – PI• Chris Vatland - co-PI• Myrna Veguilla – USF Study
Coordinator• Meghan von der Embse – External
Coach• Denise Perez Binder – External
Coach• Shanel Greene – data• Telisha McKenzie - data
Study Team
24
Comments and Questions
Tiered Coaching Model to Support Implementation of
the Pyramid ModelCRIEI 2020 TCM Project
Goals 1. Create a data-driven decision model related to
PD and teacher coaching profiles2. Develop a tiered coaching model that matches
teachers to different types of coaching3. Evaluate the efficacy of the tiered model
This project is funded by Institute of Education Sciences (R324A170019).
3/5/20
7
Project Overview
Year 3● Pilot fully developed
tiered coaching model
YEAR 2
● Develop and field test a tiered coaching model based on coaching profiles
YEAR 1● Develop coaching profiles
based on TPOT and Survey Data
Framework and Measures
Coac
hing
Foc
us
Latent Profile Analysis: Method
● 97 preschool teachers
● Teacher survey:○ Demographics○ PD experiences and satisfaction○ Job stress and satisfaction○ Work engagement and commitment○ Disciplinary efficacy
● Teaching Pyramid Observation Tool (TPOT)
3/5/20
8
Development of the Tiered Coaching ModelField Test: Method and Sample
● 16 lead preschool teachers● TPOT and Survey completed in Fall 2018● Teachers initially assigned to Tiers 1-3:○ Tier 1: 7 teachers○ Tier 2: 6 teachers (two groups)○ Tier 3: 3 teachers
● Midpoint TPOT collected, teachers moved to new Tier (as needed)
● Post-test TPOT and focus groups conducted in Spring 2019
Field Test: Quantitative Results
Initial Placement
N Average Change
Tier 1 7 2.50Tier 2 6 5.69Tier 3 3 3.45
Field Test: Movement Across Tiers
Coaching Type Initial N Mid-Year NTier 1 (Self) 7 10
• 6 stayed• 4 entered from higher tier• 1 moved to Tier 3
Tier 2 (Group) 6 4• 3 stayed• 1 entered from Tier 3• 3 moved to Tier 1
Tier 3 (Individual)
3 2• 1 stayed• 1 entered from Tier 1• 1 moved to Tier 1• 1 moved to Tier 2
3/5/20
9
Lessons Learned & Successes in Childcare
• Online small group coaching seemed effective and had positive reviews from teachers. • Integration with existing
accountability and quality systems was helpful (STARS credits, etc.)• Challenging behavior was often
the “game changer” for coaching (requests to move between tiers, etc.)• Leadership support was
essential.
Challenges & Questions
Implementation• Turnover & role shifts• Teaming, co-teaching &
“floating” staff• Integration with existing
coaching resources• Misalignment with philosophies
or program models
Conducting Research
• Coaching requests & project scope• Recruitment in Year 3 Pilot:• Intensity of data collection• Control group• Leader buy-in
Development and Testing of the FBSApp for Improving Parent and
Child Outcomes
Grant R324A160086
Cross-site Collaboration
Angel Fettig Hedda MeadanErin Barton
3/5/20
10
Need for effective family supports
Homes• Home-based supports for
families of children with challenging behaviors.
Technology • Mobile applications (apps)
might provide an innovative, effective option for delivering ongoing support to families. • Internet and smart phones are
ubiquitous Mobile technology provides an opportunity to increase the accessibility and efficiency of support for parents of these children.
Need for effective family supports
FBSApp is a mobile application to support parents in implementing functional assessment (FA)-based interventions, which work by determining the purpose (function) that a particular behavior serves for a child and using this information to develop a plan for reducing and preventing the behavior.
Secure Login
3/5/20
11
How to use this app tutorial Professional Login
Demographic Information Child’s Communication Repertoire
3/5/20
12
Challenging Behavior Topography
FBA Tutorial Collecting A-B-C Data
3/5/20
13
A-B-C Log Hypothesis Statement
Behavior Support Plan
Monitoring Progress
3/5/20
14
But will it work??
Three Categories of Successes
1. Pilot under-powered RCT was conducted 2. Results are promising • Statistical significance in favor of FBSApp group • Challenging behaviors reduced for both groups
3. Iterative process allowed for continual revisions to app • We have an iPhone version! • We have a new algorithm! • We changed almost all the language. • We revised all fidelity supports.
3/5/20
15
Under-powered RCT
Research Questions:1. Does using the FBSApp (including the webinars) effectively increase
parents’ use of FA-based intervention practices compared to parents who have access to the webinars alone?
2. Does using the FBSApp (including the webinars) effectively decrease parent-reported stress over time compared to parents who have access to the webinars alone?
Conditions
• FBSApp + Webinar • Webinar Only
Participants, Setting, Materials
• Children ages 2 – 5 with disabilities • One parent • Across the US
HOMES• iPad • Access to the internet
Results Summary
•Parent Fidelity improved•Parents with access to the FBSApp+Webinars
improved more than parents with Webinars alone•Parent stress decreased across both groups•Child challenging behaviors decreased across both
groups • Satisfaction was high for both groups
3/5/20
16
Four Categories of Challenges
1. Gathering data from families in homes 2. High rates of attrition 3. Complex measurement system4. Lack of professional involvement
Wave Family Procedures Measures Collected Type
W1Demo REDCapPSI-4, CBCL, MAS, TPQ PaperPre/post Interview Interview
W2 Upload 5 videos Child, Parent Observational Measures ObservationW3 RANDOMIZATION
W4 Upload 3 videos Child, Parent Observational Measures ObservationSatisfaction Survey Redcap
W5 Satisfaction Survey RedcapW6 Upload 3 videos Child, Parent Observational Measures Observation
Satisfaction Survey RedcapW7 Upload 3 videos Child, Parent Observational Measures Observation
W8Demo, Satisfaction Survey RedcapPSI-4, CBCL, MAS, TPQ PaperPre/post Interview Interview
W9 Upload 3 videos
Child, Parent Observational Measures ObservationDemo, Satisfaction Survey Redcap, PSI-4, CBCL, MAS, TPQ PaperPre/post Interview InterviewChild, Parent Observational Measures Observation
Participants
Cohort1 2 3 4 Total
Completed W1 19 34 26 12 91Dropped pre-randomization (W3) 9 6 8 2 25Completed 10 24 14 8 56
Treatment 5 13 6 4 28Control 5 11 8 4 28
Dropped randomization-Time 2 (W8) 0 3 4 2 9Dropped Time 2-Time 3 (W9) 0 1 0 0 2
Participants
Cohort1 2 3 4 Total
Completed W1 19 34 26 12 91Dropped pre-randomization (W3) 9 6 8 2 25Completed 10 24 14 8 56
Treatment 5 13 6 4 28Control 5 11 8 4 28
Dropped randomization-Time 2 (W8) 0 3 4 2 9Dropped Time 2-Time 3 (W9) 0 1 0 0 2
3/5/20
17
Parent Fidelity
• 7 point rating scale• Prevent• Positive attention• Predictable routines• Choices• Clear expectations
• Teach• Function-based opportunity• Reinforcement of new skills
• New Response• Function-based response to challenging behavior
Child Challenging Behavior
• Estimated rate or frequency• 5s partial-interval recording• Target and nontarget behaviors• Families could can create multiple behavior
support plans
Child Replacement Behavior
• Estimated rate or frequency• Timed-event sampling• Function-based appropriate verbal replacement behaviors
Issues and Ideas With Ongoing Research
1. SCR study underway! • When to measure, How to measure, What to measure???• Do families really have routines?
2. Changes to measurement system to individualize per family • Complexity versus reliability • Validity versus sensitivity
3. Recruited professionals • Buy in• Who to invite?
3/5/20
18
Issues and Ideas With Ongoing Research
4. How do we scale-up individualized, Tier 3 supports for ALL families?
QUESTIONS??
Research Team
Beth Pokorski
Marina Velez
Claire WinchesterBrandy Locchetta
Bhavya
Hidy Kong
Gabrielle Trimlett
Kaitlin Greeny
Gounah Choi
Shawna HarbinMoon Chung
Rebecca Hacker
Karrie Karahalios
Mollie Todt
Emma Skiba
Gabriela Robinson Lauren Schulte
Jonathan Gerth
Caroline Baum
Financial disclosure: Preparation of this presentation was supported in part by the U.S. Department of Education, Institute for Education Sciences, National Center for Special Education Research Grant R324A160086. However, the opinions expressed do not necessarily reflect the policy of the U.S. Department of Education and no official endorsement should be inferred.
3/5/20
19
Panel Discussion
•How might we extend existing evidence-based practices across early childhood contexts?•How might professional development
systems meet the needs of professionals across contexts?•How can research focus on replications and
feasibility of existing evidence-based practices to novel contexts and implementers?