gathering feedback for teaching combining high-quality observations with student surveys and...
Post on 13-Dec-2015
Embed Size (px)
- Slide 1
Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains Slide 2 Closing the effectiveness gap 2 Slide 3 Progressing beyond The Plateau 3 Slide 4 Ensuring Reliable & Trustworthy Observations CalibrateRefine TestTrain Video examples for anchor points Rater certification: Dont pass, dont rate Periodic tuning: Out of tune, dont rate Adjustments to observation framework based on data Slide 5 The First of Three Questions How many distinct levels of teacher performance do you think an evaluation system should recognize? 5 Slide 6 The Second of Three Questions What performance levels would you assign to what fraction of your districts teachers on the following competencies? A.Classroom Management (time, behavior, materials) B.Goals & Tasks (clear, appropriate, rigorous, interesting) C.Supporting Student Understanding (content depth, feedback, questioning and discussion, instructional dialogue) 6 Slide 7 The Third of Three Questions How closely associated are teaching behaviors with student outcomes (academic growth over time)? 7 Student Performance Teachers Classroom Observation Score highly moderately weakly Slide 8 8 Validation Teachers with Higher Observation Scores Had Students Who Learned More Slide 9 Observation Score Distributions Framework for Teaching 9 Slide 10 PLATO Prime, CLASS and MQI Lite 10 Slide 11 Observation Score Distributions UTeach Observation Protocol 11 Slide 12 12 Slide 13 13 Slide 14 14 Slide 15 Measures have different strengths and weaknesses 15 Slide 16 16 The Importance of Multiple Measures Slide 17 Compared to What? Compared to MA Degrees and Years of Experience, the Combined Measure Identifies Larger Differences 17 on state tests Slide 18 Compared to What? and on low stakes assessments 18 Slide 19 Compared to What? as well as on student-reported outcomes. 19 Slide 20 The Measures of Effective Teaching Project Two school years: 200910 and 201011 >100,000 students Grades 48: ELA and Math High School: ELA I, Algebra I and Biology Participating Teachers Slide 21 Our primary collaborators include: Mark Atkinson, Teachscape Nancy Caldwell, Westat Ron Ferguson, Harvard University Drew Gitomer, Educational Testing Service Eric Hirsch, New Teacher Center Dan McCaffrey, RAND Roy Pea, Stanford University Geoffrey Phelps, Educational Testing Service Rob Ramsdell, Cambridge Education Doug Staiger, Dartmouth College Other key contributors include: Joan Auchter, National Board for Professional Teaching Standards Charlotte Danielson, The Danielson Group Pam Grossman, Stanford University Bridget Hamre, University of Virginia Heather Hill, Harvard University Sabrina Laine, American Institutes for Research Catherine McClellan, Clowder Consulting Denis Newman, Empirical Education Raymond Pecheone, Stanford University Robert Pianta, University of Virginia Morgan Polikoff, University of Southern California Steve Raudenbush, University of Chicago John Winn, National Math and Science Initiative Research Partners Slide 22 22 The MET Project is ultimately a research project. Nonetheless, participants frequently tell us they have grown professionally as a result of their involvement. Below is a sampling of comments we received. From Teachers: The video-taping is what really drew me in, I wanted to see not only what Im doing but what are my students doing. I thought I had a pretty good grasp of what I was doing as a teacher, but it is eye opening I honestly felt like this is one of the best things that I have ever done to help me grow professionally. And my kids really benefited from it, so it was very exciting. "With the videos, you get to see yourself in a different way. Actually you never really get to see yourself until you see a video of yourself. I changed immediately certain things that I did that I didn't like. I realized I learned more about who I actually was as a teacher by looking at the video. I learned of the things that I do that I think that Im great at I was not so great at after all. Even the things I did well, I thought, ok that's pretty good, why do I do that, and where could I put that to make it go farther. So it was a two-way road, seeing what you do well, and seeing the things that have become habits that you don't even think about anymore." From Raters: Being a rater has been a positive experience for me. I find myself watching my own teaching more and am more aware of the things I should be doing more of in my classroom. I have to say, that as a teacher, even the training has helped me refine my work in the classroom. How wonderful! I have loved observing teachers, reflecting on my own teaching and that of the teachers teaching in my school. What the Participants Said. Slide 23 MET Extension: A Library of Teaching Practice Additional Data Collection Subset of 360 MET Teachers (disproportionately highly effective) 50 lessons taped ( 18,000 total lessons) 100% teacher & parental consent (allowing for broader public use) Cheaper cameras with potential for scale Library of Practice Searchable database (tagged by Common Core standards, teaching practices, etc.) Tagging to be done in partnership with schools of education (w/ teachers-in-training) Potential Uses: Rater training, certification, and calibration School districts professional development Teacher training institutions teaching methods Observation instrument developers validation of new & existing tools Slide 24 MET Logical Sequence 24 Measures predict Measures fairly reflect teacher Measures combine Effective Teaching Index Teaching Effectiveness Dashboard ResearchUse Measures reliable Measures improve effectiveness Measures stable under pressure Measures communicated effectively Slide 25 Validation Engine System picks observation rubric & trains raters Raters score MET videos of instruction Software provides analysis of: Rater consistency Rubrics relation to student learning Slide 26 26 Four Steps Four Steps to High-Quality Classroom Observations Slide 27 27 Slide 28 Step 1: Define Expectations Framework for Teaching (Danielson) 28 Four Steps Slide 29 Step 2: Ensure Accuracy of Observers 29 Four Steps Slide 30 Step 3: Monitor Reliability 30 Four Steps Slide 31 31 Four Steps Multiple Observations Leads to Higher Reliability NOTES: The numbers inside each circle are estimates of the percentage of total variance in FFT observation scores attributable to consistent aspects of teachers practice when one to four lessons were observed, each by a different observer. The total area of each circle represents the total variance in scores. These estimates are based on trained observers with no prior exposure to the teachers students, watching digital videos. Reliabilities will differ in practice. See the research paper, Table 11, for reliabilities of other instruments. Slide 32 32 Students with Most Effective Teachers Learn More in School Slide 33 Student Perceptions 33 Care Control Clarify Challenge Captivate Confer TestPrep Consolidate Care My teacher makes me feel that s/he really cares about me My teacher seems to know if something is bothering me My teacher really tries to understand how students feel about things My teacher makes me feel that s/he really cares about me My teacher seems to know if something is bothering me My teacher really tries to understand how students feel about things Control Students in this class treat the teacher with respect My classmates behave the way the teacher wants them to Our class stays busy and doesnt waste time Students in this class treat the teacher with respect My classmates behave the way the teacher wants them to Our class stays busy and doesnt waste time Clarify If you dont understand something, my teacher explains it a different way. My teacher knows when the class understands, and when we do not. My teacher has several good ways to explain each topic that we cover in the class. If you dont understand something, my teacher explains it a different way. My teacher knows when the class understands, and when we do not. My teacher has several good ways to explain each topic that we cover in the class. Challenge My teacher asks students to explain more about the answers they give. My teacher doesnt let people give up when the work gets hard. In this class, we learn to correct our mistakes. My teacher asks students to explain more about the answers they give. My teacher doesnt let people give up when the work gets hard. In this class, we learn to correct our mistakes. Captivate My teacher makes learning enjoyable My teacher makes learning interesting I like the way we learn in this class My teacher makes learning enjoyable My teacher makes learning interesting I like the way we learn in this class Confer My teacher wants us to share our thoughts Students get to decide how activities are done in this class My teacher wants us to share our thoughts Students get to decide how activities are done in this class Consolidate My teacher takes the time to summarize what we learn each day The comments that I get on my work in this class help me understand how to improve My teacher takes the time to summarize what we learn each day The comments that I get on my work in this class help me understand how to improve TestPrep I have learned a lot this year about [the state test] Getting ready for [the state ] test takes a lot of time in our class I have learned a lot this year about [the state test] Getting ready for [the state ] test takes a lot of time in our class Slide 34 Student Perceptions Top 5 Correlations 34 Survey Statement Category Rank 1 1 2 2 3 3 4 4 5 5 Students in this class treat the teacher with respect My classmates behave the way my teacher wants them to Control Our class stays busy and doesnt waste time Challenge In this class, we learn a lot every day Challenge In this class, we learn to correct our mistakes 33 I have learned a lot this year about [t