team-initiated problem solving (tips) overview presented by: dale cusumano, ph.d., ncsp research...
DESCRIPTION
Problem Solution Organizing for an Effective Problem Solving Conversation Use DataTRANSCRIPT
Team-Initiated Problem Solving (TIPS)
OverviewPresented by:
Dale Cusumano, Ph.D., NCSPResearch Associate
University of North Carolina at Charlotte
Presented to: Seventh Annual Response to Intervention Best Practices Institute
October 15, 2013
90,000 public schools in the United States
Each school has 1+ teams to address
challenges and build solutions
Each team meets at least monthly
On average there are 5 people on each
team
810,000 hours of meetings
4,050,000 hours of personal
time annually
Problem
Solution
Organizing for an Effective Problem Solving Conversation
Use Data
Bob Algozzine, Kate Algozzine, and Dale CusumanoUNC at Charlotte
Rob Horner, Anne Todd, and Steve NewtonUniversity of Oregon
Data+
Outcomes
To what extent do teams follow problem solving steps and include
critical components?• Include a behavioral definition of
target behavior• Have a direct measure of the target
behavior prior to intervention• Include a step-by-step intervention
plan• Graph intervention results• Compare pre-intervention and post-
intervention performance• Develop a hypothesized reason for
the problem• Gather evidence that the
intervention was implemented as designed
Behavioral definition of target behavior, baseline
data, step-by-step plan, or graphic or numeric data
comparison from pre to post intervention were not
included (Flugum & Reschly, 1994)
Lowest implementation for hypotheses for problem,
treatment fidelity, and pre and post data comparison(Telzow, McNamara, & Hollinger,
2000)
Team members rated implementation as higher than observers with observers rating identifying antecedents and consequences for
behavior, identifying data to monitor progress, scheduling a follow up meeting as unmet
(Telzow, McNamara, & Hollinger, 2000)
People aren’t tired from solving problems – they are tired from solving the same problem over and over.
6Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished training manual.
Efficient
Effective
Logical
Thorough
Team-Initiated Problem Solving II(TIPS II)
Decisions are more likely to be effective and
efficient when they are based on data
Quality of decision-making depends most on the first step (defining the
problem to be solved)
Data help us ask the right question…they do not
provide the answers: Use data to identify problems, refine the problems, and define the questions that
lead to solutions
Data help place the “problem” in the context rather than in students
Main Ideas
Build “decision systems” not “data systems”• Problem solving is driven by
decision systems, which are fueled by data
Use data in “decision layers”
(a) Is there a problem? (b) Localize the problem
(location, problem behavior, students, time of day), and
(c) Get specific
Do not drown in data
Be efficient
It is OK to be doing well!
More Main Ideas
What do we need?
• A clear model with steps for problem solving
• Access to the right information at the right time in the right format
• A formal process that a group of people can use to build and implement solutions.
10Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports,
University of Oregon, unpublished training manual.
Eight Keys to Effective Meetings
1. Organization (team roles, meeting process, agenda)2. Data (right information at right time in right format)3. Separate (a) Review of On-going Problem Solving (b)
Administrative Logistics and (c) New Problem Solving4. Define Problems with Precision5. Build Comprehensive Solutions that “fit”6. Add “Action Plans” for all solutions7. Review Fidelity and Impact regularly8. Adapt Solutions in response to data.
Implement Solution with High Integrity
Identify Goal for Change
Identify Problemwith
Precision
Monitor Impactof Solution and
Compare against Goal
Make SummativeEvaluationDecision
MeetingFoundations
Team-Initiated Problem Solving (TIPS II) Model
IdentifySolution and
CreateImplementation
Plan with Contextual Fit
Collect and Use
Data
TIPS ModelTIPS Training
• One full day team training• Two coached meetings
Team Meeting• Use of electronic meeting minute system• Formal roles (facilitator, recorder, data analyst)• Specific expectations (before meeting, during meeting, after meeting)• Access and use of data• Projected meeting minutes
Research tool to measure effectiveness of TIPS Training• DORA (decision, observation, recording and analysis)• Measures “Meeting Foundations” & “Thoroughness of Problem Solving”
Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished manual.
9
TIPS I Study: Todd et al., 2011
OctNov Dec Ja
nFeb Mar Apr
May0%
20%
40%
60%
80%
100%
OctNov Dec Ja
nFeb Mar Apr
May0%
20%
40%
60%
80%
100%
OctNov Dec Ja
nFeb Mar Apr
May0%
20%
40%
60%
80%
100%
OctNov Dec Ja
nFeb Mar Apr
May0%
10%20%30%40%50%60%70%80%90%
100%
School A
School B
School C
School D
Baseline Coaching TIPS%
DOR
A Fo
unda
tions
Sco
re
Solid = SW PBIS meetings using SWISOpen = progress monitoring meeting using DIBELS
Journal of Applied School Psychology
Pre Post/Exp0
10
20
30
40
50
60
70
80Exp Control
N = 17 N = 17TIPS Control TIPS Control
DORA
Tho
roug
hnes
s of D
ecisi
on M
akin
g Sc
ore
(Sim
ple)
Newton et al., (2012):Effects of TIPS Training on Team Decision-making
Pre TIPS Training Post-TIPS Training
Problem-Solving Meeting
Foundations
Structure of meetings lays foundation for efficiency & effectiveness
Implement Solution with High Integrity
Identify Goal for Change
Identify Problemwith
Precision
Monitor Impactof Solution and
Compare against Goal
Make SummativeEvaluationDecision
MeetingFoundations
Team-Initiated Problem Solving (TIPS II) Model
IdentifySolution and
CreateImplementation
Plan with Contextual Fit
Collect and Use
Data
PREDICTABILITYPARTICIPATION
ACCOUNTABILITYCOMMUNICATION
PREDICTABILITYPARTICIPATION
ACCOUNTABILITYCOMMUNICATIONRoles & Responsibilities
The process a
team uses is
important
Roles
FacilitatorRecorder
Data analystActive Member
Organization
Agenda, old business, new business, action plan for
decisionsWhat happens BEFORE a
meetingWhat happens DURING a
meetingWhat happens AFTER a
meeting
Foundations of an Effective and Efficient Meeting
Who is Responsible?Action Person Responsible
Reserve Room
Recruit items for Agenda
Review data prior to the meeting
Reserve projector and computer for meetingKeep discussion focused
Record Topics and Decisions on agenda/minutesEnsure that problems are defined with precisionEnsure that solutions have action plans
Provide “drill down” data during discussionEnd on time
Prepare minutes and send to all members
PREDICTABILITYPARTICIPATION
ACCOUNTABILITYCOMMUNICATIONRoles & Responsibilities
Start and end times
Agenda is used
TIPS Meeting Minutes and Problem-Solving Action Plan FormToday’s Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst:
Next Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst: Team Members (bold are present today________________________________________________________________
Information for Team, or Issue for Team to Address
Discussion/Decision/Task (if applicable) Who? By When?
Administrative/General Information and Issues
Implementation and Evaluation
Precise Problem Statement, based on review of data
(What, When, Where, Who, Why)
Solution Actions (e.g., Prevent, Teach, Prompt, Reward, Correction, Extinction,
Safety)Who? By When?
Goal, Timeline, Decision Rule, & Updates
Problem-Solving Action Plan
Agenda for NEXT Meeting
1. 2.3.
Implementation and EvaluationPrecise Problem Statement, based on
review of data(What, When, Where, Who, Why)
Solution Actions (Prevent, Teach, Prompt, Reward,
Correction, Extinction, Adaptations, Safety)
Who? By When? Goal with Timeline
Fidelity of Imp measure
Effectiveness of Solution/Plan
Not started Partially Imp Imp Fidelity Done
Goal Met Better Same Worse
Agenda for Today:1. 3. 5.2. 4. 6.Previously Defined Problems/Solutions (Update)
TIPS Meeting Minutes and Problem-Solving Action Plan FormToday’s Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst:
Next Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst: Team Members (bold are present today________________________________________________________________
Information for Team, or Issue for Team to Address
Discussion/Decision/Task (if applicable) Who? By When?
Administrative/General Information and Issues
Implementation and Evaluation
Precise Problem Statement, based on review of data
(What, When, Where, Who, Why)
Solution Actions (e.g., Prevent, Teach, Prompt, Reward, Correction, Extinction,
Safety)Who? By When?
Goal, Timeline, Decision Rule, & Updates
Problem-Solving Action Plan
Agenda for NEXT Meeting
1. 2. ‘3.
Implementation and EvaluationPrecise Problem Statement, based on
review of data(What, When, Where, Who, Why)
Solution Actions (Prevent, Teach, Prompt, Reward,
Correction, Extinction, Adaptations, Safety)
Who?
By When? Goal with Timeline
Fidelity of Imp measure
Effectiveness of Solution/Plan
Not started Partially Imp Imp Fidelity Done
Goal Met Better Same Worse
Agenda for Today:1. 3. 5.2. 4. 6.
Previously Defined Problems/Solutions (Update)
Where in the Form would you place:
1. Planning for next PTA meeting?
2. There have been five fights on the playground in the past 3 weeks.
3. Update on CICO implementation
4. Increasing gang recruitment as an agenda topic for today.
5. Next meeting report on lunch-room status.
PREDICTABILITYPARTICIPATION
ACCOUNTABILITYCOMMUNICATIONRoles & Responsibilities
Start and end times
Agenda is used
Meeting schedule
PARTICIPATION
Engaged in Topic
Decision Makers
COMMUNICATIONNorms for the meeting
Access to Meeting Minutes
ACCOUNTABILITY
Prepared for meetings
Are solutions working?
Documentation of decisions
Differences in lives of students
PREDICTABILITYPARTICIPATION
ACCOUNTABILITYCOMMUNICATION
Implement Solution with High Integrity
IdentifyGoal for Change
Identify Problemwith
Precision
Monitor Impactof Solution and
Compare against Goal
Make SummativeEvaluationDecision
MeetingFoundations
Team-Initiated Problem Solving (TIPS II) Model
Identify Solution and
CreateImplementation
Plan withContextual Fit
Collect and Use
Data
Problem = A difference between what is expected and what is observed that a team feels is worthy of addressing.
Start with Primary Problem Statements
Look at the Big Picture. Then use data to refine the problem to a Precise Problem
Statement.
Office discipline referrals for 3rd graders are above national medians for schools our size.
Morphing Data into Useful Information
Develop Primary Problem Statements• Look first at your patterns (tell the story)
• Level, Trend• Peaks• Match data to current perceptions
• Compare your data• With national median or percentiles• With last year• With what your faculty/students/ families want
Start with Primary Problem Statements
Look at the Big Picture. Then use data to refine the problem to a Precise Problem
Statement.
Move to Precise Problem Statements
Office discipline referrals for 3rd graders are above national medians for schools our size.
Referrals for defiance among third grade students from 11:30-12:30 in the cafeteria are increasing over time. It is believed that this is happening because students want
to avoid silent reading that happens after lunch.
What When
Who
Why
Where
What
WhoWhy
Precision Components for Behavior Problem Statements
Precision Components for Academic Problem Statements
Examples: Primary to Precise
Gang-like behavior is increasing.
Our fourth graders cannot comprehend when reading!
• Bullying (verbal and physical aggression) on the playground is increasing during “first recess,” is being done mostly by four 4th grade boys, and seems to be maintained by social praise from the bystander peer group.
• Forty-seven percent of 4th grade students did not meet reading comprehension targets on AIMSweb Maze benchmark assessments when 80% of students at a grade level should meet this target. It appears that weak vocabulary skills are lowering students’ comprehension skills.
Examples: Primary to Precise
Carly is having reading difficulties.
• Carly is reading 20 words correctly per minute (goal is 60), skips or guesses at words she doesn’t know, mostly during language arts.
• Carly can not decode and struggles to read words containing R controlled vowels, digraphs, & long vowels.
• Jack’s math CBM scores fell at the 10th percentile as compared to national norms. Jack is not fluent in his knowledge of basic math facts and often does not attend to addition or subtraction signs on written math problems.
Jack cannot add or subtract.
Precise or Primary Statement?
Minor disrespect and disruption are increasing over time, and are most likely during the last 15 minutes of our block periods when students are engaged in independent seat work. This pattern is most common in 7th and 8th grades, involves many students, and appears to be maintained peer attention.
Precise or Primary Statement?
James is instructional in reading content that falls two grades below his current placement. His reading fluency scores fall below the 25th %ile and his rate of improvement is predicting that he will not meet end of the year goals for his grade. He has difficulty decoding unfamiliar words and does not track accurately from left to right when reading.
The boys in third grade are performing well below the 3rd grade girls in reading.
Three 5th grade students are having great difficulty expressing their thoughts in writing. Their fine motor skills are well developed but writing test scores are well below target. Since they can verbally share well-constructed and grammatically correct stories, it is thought that their spelling skills are hindering their writing skills with average weekly spelling test scores of C’s and D’s obtained.
Next, We Need Goals…
We need measurable and realistic SMART goals.
• SMART Goals• Specific • Measurable • Achievable• Relevant• Timely
Precision Problem StatementMinor disrespect and disruption are increasing over time, and are most likely during the last 15 minutes of our block periods when students are engaged in independent seat work. This pattern is most common in 7th and 8th grades, involves many students, and appears to be maintained by peer attention.
Precision GoalThe rate of disrespect and disruption among 7th and 8th graders (as measured by SWIS) will decrease to 2 per week by the 15th of next month (~.40 per day per month) and decrease to one or fewer per week (<.20 per day per month)by the end of the following month to be maintained across the remainder of the school year.
Examples
Trend
Our average Major and Minor ODRs per school day per month are higher than national median for a school of our enrollment size for all months except June. We have peaks in frequency of problems in Nov, Dec, Jan, and March with an increasing trend from September to March.
What When
Who
Why
Where
What
WhoWhy
Precision Components for Behavior Problem Statements
Precision Components for Academic Problem Statements
SWIS Big 4 for October 1, 2011 through December 31, 2011
Defiance
SWIS Big 4 for October 1, 2011 through December 31, 2011
ClassroomPlaygroun
d
11:30-12:15
Avoid Work
SWIS Big 4 for October 1, 2011 through December 31, 2011
Possible Problem Precision Statement
Average referrals per day per month for All Referrals & Minors are above the 75th percentile for 5 of 6 months with an increasing trend. Defiance (what) in classrooms and on the playground (where) for a group of 15 fifth graders (who) between 11:30 and12:15 (when) is the most frequently reported behavior problem and occurs as a means of avoiding work (why).
How does this work for academics?
80% meeting
expectation
15% need more
5% need the most
Is there a problem? Academic Reference Points
50th Percentil
e on national norms
Low Risk indicator
DIBELS Universal Screening
Primary Problem StatementOur DIBELS Distribution summary shows that 49% of our kindergarten students at Adams Elementary fall in the strategic and intensive range. We have over 50% of our students requiring strategic and intensive supports for ISF, LNF.
Primary Goal At least 80% of our Kinders will be in Benchmark range at Winter Universal Screening Time
1 2 3 4 50%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of Students Meeting Expectation in Reading Comprehension (AIMSweb Maze)
FallWinterSpringGoal
Grade Level
Only 62% of 4th graders and 65% of 5th graders are meeting expectations in reading comprehension at the winter
benchmark; although a slight improvement from fall to winter is noted, it is below the goal of having 80% of students meet
expectation.
What?
For today, we will focus on 4th
graders
Four 4th graders have the lowest scores on measures of reading comprehension. Other
students are below expectation but their scores are within + 1 correct responses (CR) of the target (14 CR). Scores for these four students fall in the
At Risk range, which is below the 10th percentile on national norms.
Who?
Low Risk > 14 CR
All close
to goal of 14 CR
Why?
Why?
ICEL
Skill Developm
ent
Functional
Drivers
Three 4th grade students are performing well below expectation comprehending written text that is presented at grade level (scores on Maze measures fall below the 10th percentile on national norms). These students also have weak reading
fluency skills that fall well below expectation, which is thought to be the reason for their low reading comprehension skills. A
fourth student (Sally) also is performing well below expectation in reading comprehension, but her reading fluency skills are in expected ranges. Weak vocabulary skills may be lowering her
comprehension skills.
Why?
Let us look at reading fluency as a possible reason for these
four students’ low reading comprehension.
Harris Word list for Sally = 2nd grade
Implement Solution with High Integrity
IdentifyGoal for Change
Identify Problemwith
Precision
Monitor Impactof Solution and
Compare against Goal
Make SummativeEvaluationDecision
MeetingFoundations
Team-Initiated Problem Solving (TIPS-II) Model
Identify Solution and
CreateImplementation
Plan withContextual Fit
Collect and Use
Data
Using Precision Problem Statements to Build Solutions, Action & Evaluation
PlansSolutions
Prevention: How can we avoid the problem context? Teaching: How can we define, teach, and monitor what we want? Recognition: How can we build in systematic reward for desired behavior?Extinction: How can we prevent problem behavior from being rewarded?Consequences: What are efficient, consistent consequences for problem behavior?
Action PlanWho will do each task & when will it be completed?
Evaluation
How will we collect and what data will we use to evaluate: • Implementat
ion fidelity?• Impact on
student outcomes?
Prevent “Trigger”
Define & Teach
Reward/Reinforce
Withhold Reward
Corrective consequence
Other
Safety62
Example for Precise Behavior Problem Statement6th and 7th graders are engaging in inappropriate language, harassment, disrespect and aggression in two classrooms at 9:45 and 12:45 to get peer and adult attention and to escape the work. There are 175 total instances of problem behavior in 6th and 7th grade classrooms, for 2010-11 school year.
Re-review 6th and 7th graders classroom expectations/Respecting others, daily
Focus on Respect, Re-teach stop-walk routine
Set up “Daily Double”: Class period without problem behavior occurrence receives extra 2 mins, at end of period to talk.Provide specific feedback for using stop-walk routineEnsure staff use routine for responding to a report when student comes to talk.Use School Defined Process
Prevent “Trigger”
Define & Teach
Reward/Reinforce
Withhold Reward
Corrective consequence
Other
Safety63
Example for Precise Academic Problem StatementFour 4th grade students are performing below expectation in reading comprehension. For three of these students, their performance is being impacted by low reading fluency skills. For one student, weak vocabulary skills are keeping her from comprehending written text.
Provide instructional level materials for independent reading activities.
Participation in a Repeated Readings intervention group.Self monitoring of reading fluency skills with gains tied to rewards.Self monitoring of known and unknown vocabulary words and accuracy rates.
Participation in a Drill Sandwich vocabulary practice.
Every solution/task needs an action plan• Who will do the task?• When will the task be completed?
Action Planning
Implement Solution with High Integrity
IdentifyGoal for Change
Identify Problemwith
Precision
Monitor Impactof Solution and
Compare against Goal
Make SummativeEvaluationDecision
MeetingFoundations
Team-Initiated Problem Solving (TIPS II) Model
IdentifySolution and
CreateImplementation
Plan withContextual Fit
Collect and Use
Data
Evaluation Planning
Evaluation Plan for monitoring fidelity of implementation AND impact on student behavior
• Evaluate fidelity of implementation compared to the goal
• Define how, when, criteria• Evaluate effect of solutions on student behavior
(impact) as compared to the goal• Define data to be used, how often and criteria
• Data analyst with data summaries and data access
Fidelity of ImplementationMeasure the degree in which the intervention was implemented as defined/expected
• Use percent/absolute value/ rate/scale as metric• Strive for 80% fidelity of implementation as measured
weekly (bi-weekly) on scale of 1-5Make easy for staff to record data
• Fidelity Check Board: X on number line• Fist of five• Fidelity check basket• Direct observation
Are we implementing the plan?
1 2 3 4 5 No Yes
Other Data for Monitoring Fidelity of Implementation
Fidelity Check RoutineWe do what we say we will do and we do it
with 80% fidelityEstablish a fidelity check routine that relates to Implementation
A 1-5 scale is used for all questions, with up to three questions per weekAt staff meeting, use fist of five while asking questionsIn staff room, create number line poster with questions
Did you provide instructional level reading to students?
1 2 3 4 5No Yes Did you acknowledge 5 students, not in
your classroom, daily? 1 2 3 4
5No
Yes
Evaluation PlanningEvery problem needs to be monitored and evaluated
• Fidelity of Implementation• Effectiveness of Implementation
Impact of SolutionFor Behavior Example
Did we make a difference?
Impact of SolutionFor Academic Example
Evaluate the Problem: Did we make a difference?
1 2 3 4 50%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of Students Meeting Expectation in Reading Comprehension (AIMSweb Maze)
FallWinterSpringGoal
Grade Level
✔
Evaluate the Problem: Did we make a difference?
✔ ✔
✔
Identify Problemwith
PrecisionMake Summative
EvaluationDecision
Implement Solution with High Integrity
IdentifyGoal for Change
Identify Problemwith
Precision
Monitor Impactof Solution and
Compare against Goal
Make SummativeEvaluationDecision
MeetingFoundations
Team-Initiated Problem Solving (TIPS II) Model
IdentifySolution and
CreateImplementation
Plan withContextual Fit
Collect and Use
Data
TIPS Team
Training
Readiness
Team Membershi
p
Team Data Access
Team Commitme
nt
Coaching Commitme
nt
Team Membership1. Representation needed for meeting their purpose 2. Inclusion and presence of administrator with authority to
make decisions
Team Data Access3. Data available for problem solving & decision-making
before and during the meeting4. Consistent process & procedures for documenting &
entering data exists5. Team member is fluent in generating basic and custom
reports from data set(s) being used
TIPS Team Training Readiness10 readiness guidelines
TIPS Team Training Readiness10 readiness guidelinesTeam Commitment
6. Implementation of TIPS Meeting Foundations7. Team & coach attendance at TIPS Team Training
• ** one full day or two half day team trainings8. Application of the TIPS model through the school year &
annual TIPS boosters
Coaching Commitment9. Team has access to a coach who knows the TIPS system
& who is available before, during, & after meetings to support fidelity of implementation
10. Commitment to attend team training and provide coaching before, during and after the meetings
Where are you in the implementation process? Adapted from Fixsen & Blase, 2005
• We think we know what we need so we are planning to move forward (evidence-based)
Exploration & Adoption
• Let’s make sure we’re ready to implement (capacity infrastructure)
Installation
• Let’s give it a try & evaluate (demonstration)Initial Implementation
• That worked, let’s do it for real (investment)Full Implementation
• Let’s make it our way of doing business (institutionalized use)Sustainability & Continuous Regeneration
Putting “it” in place.
Should we do it?
Now make it better
Initiative is way of work
TIPS II: Contacts and ReferencesUniversity of
OregonRob Horner
Anne [email protected]
Steve [email protected]
UNC at CharlotteBob [email protected]
Kate [email protected]
Dale [email protected]