overview of longitudinal evaluation research pbs teacherline station meeting october 27, 2008
TRANSCRIPT
Overview of Longitudinal Evaluation Research
PBS TeacherLine Station MeetingOctober 27, 2008
ContextContext
Two two-year studies:• Peer Connection use among
coaches and their coachees
• Online course – Teaching Reading Fluency
Experimental designs Nationwide reach 2008-2009 and 2009-2010 Mixed methods
Coaching Study Research QuestionsCoaching Study Research Questions
1. What impact does PBS TL Peer Connection have on instructional coaching programs and teacher attitudes, knowledge and instructional practices?
2. To what extent do the effects of PBS TL Peer Connection depend on mediating factors?
3. How do the effects of the PBS TL Peer Connection change over time?
4. To what extent do the effects of PBS TL Peer Connection vary according to mediating factors?
5. To what extent does the implementation of PBS TL Peer Connection correlate with improved student achievement?
Coaching Study DesignCoaching Study Design
Matched-pair random assignment to:
a) “business as usual”
b) Unlimited access to PBS TL Peer Connection, and professional development
Coaching Study ParticipantsCoaching Study Participants
1 24 coaches are in the process of joining the study2 36 teachers are in the process of joining the study
Located in SC, AR, and throughout the NortheastFocus on Title I schoolsRecruitment is ongoingStipend for participation
Goal Current
Coaches 150 921
Teachers 500 3192
Coaching Study MeasuresCoaching Study Measures
Electronic tracking of PC use Content analysis of coach-teacher
dialogue Surveys Field studies Document analysis
Coaching Study Part IICoaching Study Part II
Research question:
• To what extent are the effects of PC on student literacy or math achievement mediated by Peer Connection’s effects on teacher instructional practice?
Study within a study Focused intervention Literacy, then math Subsample of 30 coaches for each content area Same methods as Part I, plus student
benchmark testing data
Online Course Research QuestionsOnline Course Research Questions
1. What impact does the PBS TL course have on teacher outcomes?
2. What impact does the PBS TL course have on student reading fluency and engagement in reading?
3. To what extent do the effects that the PBS TL course has on teaching depend on mediating factors?
4. How do the effects of the PBS TL course on teaching change over the course of the school year?
Online Course Study DesignOnline Course Study Design
Switching replications designTimeline
Teachers
Fall 2008
Spring 2009
May 2009
Sept 2009 Late Fall 2009
January 2010
May 2010
Group A
Baseline teacher surveys
Online course
Teacher surveys
Teacher surveys and baseline student assessments
Teacher surveys and student assessments
Teacher surveys and student assessments
Group B
Baseline teacher surveys
Teacher surveys
Teacher surveys and baseline student assessments
Online course
Teacher surveys and student assessments
Teacher surveys and student assessments
Online Course Study ParticipantsOnline Course Study Participants
150 teachers• 75 Group A
• 75 Group B Located in NV, CO, VA, and MD Grades 2-3; ELA or general ed Title I schools Stipend for participation IRB approval
Online Course Study MeasuresOnline Course Study Measures
Teacher surveys: background, beliefs, efficacy, knowledge, practices. Collected at 5 points between October 2008 and May 2010
Student measures: reading fluency and attitudes; DIBELS. Collected at 3 points during 2009-2010 school year
Why does this matter?Why does this matter?
Research provides PBS stations with information on the effectiveness of PBS TeacherLine courses and products
The quality of PBS TeacherLine products improves based on research results
Ways you can helpWays you can help
Support efforts to recruit and/or maintain study participants
Make contacts between Hezel and the district’s research approval team
Maintain lists of interested teachers, principals, and school districts that can be tapped for future research projects
Barb StorandtSenior Project Manager, PK-12
1201 East Fayette StreetSyracuse, NY 13210
315-422-3512 [email protected]
Hezel AssociatesHezel Associates